The way I see it, unless the government can show demonstrable harm of an activity they don't have a right to ban it. So, yes, you might "have to" allow polygamy. Oh dear, that slippery slope of civil rights. I'm shaking.
Unless there's demonstrable harm in polygamy. The only possibility I can think of is child custody in the event of a divorce. Say 5 people are a married group, they all have children raised in that home, but one of the biological parents wants to split. Who gets custody? A definitely complicated issue, legally speaking. But is that harm?
You need to move away from the historical view of polygamy. It's often harmful not because of its nature, but because of the manner in which people have gone about it. Polygamy is often equated with goofy cultists where one man has several (often underage) wives who are basically brainwashed. Using that as a basis for law, however, would be like banning firearms because some people shot up a bank. Oh wait. :P