Within? A US passport is a ticket to anywhere you want, buy a visa at the door and jump a couple hoops to stay as long as you like. It's near-instant world mobility (with double luggage allowance).
It is wrong that physical mobility is so restricted, but the hard truth is letting people into the country that have little chance of returning home is a little too much Statue of Liberty. Face it, living poor on the streets of a developing country is worse than doing so in a developed country. If a plane ticket (not completely out of the realm financially, perhaps with donations) could buy that... yeah, that doesn't work. So, we must have immigration law if we are not to become the receptacle of every street person that their government is willing to buy them a ticket. Now, let's calculate the cost of a street person. 1k? Gone, forever, welfare program called "Get your ass outta here" and they buy them a ticket wherever, no return. Like Castro, but a better chance of survival.