Yeah but you idea of responsible could have nothing to do with freedoms rights and the constitution.
"responsible" is very subjective.
Not saying you think this way at all but IM sure there are people that think its irresponsible for a white person to marry a black person, or its irresponsible to let women vote, or let muslims on an airplane. Etc etc
But stopping any 3 of those things would definitely infringe on rights and freedoms.
Again I'm not saying you think any of these are true I'm just pointing out how using responsibility (totally subjective) as a deciding factor could also mean a person does NOT care about your fellow american.