Neither side in an argument can find the truth when both make an absolute claim on it.
As for modern day marriage, we're talking about two distinct things--marriage as a religious ceremony and marriage as a contract. I don't believe that we have an argument here beyond the already well-worn discussion about "marriage vs. civil union," but the OP already suggests that creating a bright line distinction between these is highly unlikely. I hate getting bogged down in semantic arguments. I'd stipulate that "marriage," in the context of a discussion about "rights" has to be a legal concept.
I don't think anyone wants to force churches to violate their tenets by forcing anyone to perform a rite that doesn't make sense within that particular religion.
But it's important to remember that the whole question of "rights" is a matter for the courts to decide, based on the Constitution. This question hasn't received a full airing before the courts, but it will.
So I suppose the answer is as far as an athiest and the law is concerned they are married, but as far as a church is concerned, they aren't.
Interestingly, strictly speaking, a divorcee can't get remarried in a Catholic church without getting an annulment, for much the same reason. Only the other way around.