I contend they are not similar. Every contract requires the free-will consent of all affected parties.
2 homosexuals have agreed to have an intimate relationship. All parties are notified and in agreement. This is ethical and moral.
Adultery involves 3 parties. One party is defrauded when the other 2 parties engage in conduct that is concealed and for which consent has not been obtained.
If a married couple agreed to an open relationship, then it would not be adultery. Therefore, there is no comparison. As for your personal opinion, I can respect it but completely disagree. What if I thought that gun ownership was immoral? Would that make it immoral? I don't think so.