Marraige is a state contract, not a federal power. Just because federal law REFERS to a state contract status does not make it into a federal issue (or power). What denies the right of the people to polygamy? What denies the right to vote to convicted felons, but bestows most other rights to them without restriction? Many things simply are, yet that is why we have constitutional amendment, to change the things that simply are.
You assert that the SCOTUS can simply say something and it serves as an amendment, like Roe vs. Wade, making abortion kind of a right (yet it specifically said 'during the first trimester' but that just got blown off). It took constitutional amnendment to ban the recreational drug alcohol, yet none to ban herione, marijuana, cocaine, LSD or many other recreational drugs. We now seem to accept "close" or "related" to mean "stated" as it applies to constituional law.
How many state referemdums and laws have been passed relating to this SSM 'issue'? If something can not pass the test of the voters, we now seek to find a 'like minded' judiciary to "git-r-done", rather than actually admit that this is beyond the control of the constitution as stated.
The constituion envisioned "postal roads", yet made not provision that the states may charge license fees for or regulate travel on them, it just sort of happened, limitting our freedom as citizens to travel. Many rights can be implied or infered, especially given judges with the 'correct' attitude. I would bet that if the SCOTUS was 5 to 4, leaning to the right, then Roe vs. Wade would not have ever happened and that a constitutional "abortion rights" amendment will NEVER happen. We need to stick to making the constitutional law work as intended, not simply do "what seems right at the current time".