Congress has power to enact laws that affect the lives, liberty, and property of people. Those laws can deprive you of your life, your property and your liberty and put you in prison or even kill you. So honestly, I think it's pretty ridiculous to say the Congress has no such power under the Constitution. The Constitution gives the Federal government the power to tax citizens. When I pay income tax, they are taking my property. Now Wesley Snipes thought the government had no such power, but we know how well that went for him.
Moreover, the 13th amendment states in no ambiguous terms that slavery shall not exist in the United States. Again the exact text
That is not a merely a restriction on government, it is a blanket restriction on individuals as well. It's an absolute statement that slavery shall not exist. Therefore your assertion that the Federal government does not have power over peoples lives, liberty, and property is false.Neither slavery nor involuntary servitude, except as a punishment for crime whereof the party shall have been duly convicted, shall exist within the United States, or any place subject to their jurisdiction.