Divorce is the only way other people's unions damage me. It behooves me and every other citizen to promote healthy unions, and that's accomplished by helping couples avoid the leading causes for divorce so that they stay happily married with loving, stable homes. Beyond that, ssm doesn't effect me and I have no interest in the topic.
Yes, you are setting them up for failure.
IMO there should be a broad marital health and equality act, where, yes, same-sex couples are accepted into the fold, and where all couples have to participate in personal and financial pre-marital counseling. I would even go so far as to specifically allow polygamy up to 4 wives (to be inclusive of both Native Americans and Muslims). You have to learn a basic level of skill before you're allowed to drive. In most states you have to learn a basic level of familiarity before you can hunt or own a gun. I'm saying that every couple should have to learn how to, frankly, conduct their lives together, before marrying. Pre-marital counseling clears up a lot of communication and money problems, so it would drastically reduce the divorce rate, thus reducing the harm onto you and me.
And no, pre-marital counseling is not a cure-all. There will a % of unions which can pass any counseling and still fall apart. But many more can be saved then who fall through the cracks.
My problem with the modern pro-ssm political movement (as distinguished from 2 real people of the same sex who wish to marry) has less to do with the genitalia involved and more to do with ignoring existing problems.