Unless the effect of that vote is an active subversion of the principles of freedom and privacy that inform the operation of the republic.And those rights INCLUDE the right to practice ones religion and the right to vote. And if my religion has taught me that homosexuality is wrong and thus shouldn't be endorsed, then I am well within my rights to vote against the government allowing gay marriage and you'd be the one attempting to violate rights if you attempted to tell me I'm not allowed to use my religion as a basis for my rational on how I vote.
The basic logic in your post is that people have a right to vote against things on the terms that it offends their personal moral sensibilities. Since that generally isn't believed about EVERY behavior (such as owning guns, or belonging to a specific religion), a good rationale must be provided to add moral authority to the law necessary for the law to be maintained.
There's never been a logically satisfying argument for why gay marriage should be banned. It "upsets traditional marriage" isn't a good argument because traditions enjoy no constitutional protections, particularly not above behaviors that actually seem consistent with the rights promised by the U.S. Constitution.
The criticism is that religious people have a civic duty to use their votes responsibly and non-hypocritically, which doesn't consistently occur.