Chris... a prison is a part of society. Why you argue against this is beyond me. A prison is simply a part of society that falls under the state's jurisdiction. It is as much a part of society as a monastery is. There are parts of air ports that are locked to the public, have locks on them, etc and it is still a part of society.
so·ci·e·ty noun \sə-ˈsī-ə-tē\
: people in general thought of as living together in organized communities with shared laws, traditions, and values
: the people of a particular country, area, time, etc., thought of especially as an organized community
Society - Definition and More from the Free Merriam-Webster Dictionary
If the prison were isolated on an island in a distinct place or on the moon then it probably wouldn't be a part of the society... but as most prisons are just in a different part of town they are a part of society. A prison is of a particular country, area, time. A prison is a part of organized communities with shared laws, traditions, and values. A prison has workers that go in and out of the prison on a daily basis to homes and other jobs that are equally a part of society.
You are letting the common term: Remover them from society... to mean everything. It doesn't literally mean remove them from society but rather to isolate them from other members of society in a more secure manner so as to effectively not have to worry about them.