And there were those who claimed the same thing about interracial marriage. Once you start hitting them where it hurts, like forcing them to pay attorney fees or other costs incurred for having to postpone weddings/marriages or for having to fight the matter in court, and forcing them to pay, they won't fight that hard (and yes, this happened with interracial marriage).
https://www.aclu.org/news/nc-interracial-couple-denied-marriage-license-1970s-speaks-out-against-sb-2
"In 1977, a federal court ordered a magistrate to perform their marriage, and the two magistrates who originally refused were ordered to pay legal fees."
How exactly is "I will not be involved in the marriage of a
same sex couple due to my religious beliefs" different from "I will not be involved in the marriage of an
interracial couple due to my religious beliefs", besides "well I agree with one belief but not the other"?