Um.... sort of yes but not really.
Everyone has various potentials, and inborn limits to those potentials, yes.
However, most people never put in the work to reach their full potential and come up against that hard limit. I could point to any number of people I know who had talent and potential and never did as much with it as they could have.
OTOH some people are more stringently limited. Some have low intelligence or disabilities of various sorts.
However there are many ways to compensate for this and succeed anyway. I know a successful dyslexic doctor (surgeon in fact)... a successful plumber who can't read a newspaper... more like that.
Bruce Lee and Chuck Norris were sickly children who didn't exhibit any athletic brilliance until later in life. Steven Hawking is in a wheelchair but the whole world listens when he speaks.
About the only thing you can't compensate for is a lack of even trying.
Now, I've been around, and I've spent more than a little time on the bad side of town. I've known people from what is termed "generational poverty". While most of them didn't win the genetic lottery on brains, looks or talents, from what I see most of them "rot" because no one taught them not to. No one instilled a "can do" attitude in them, told them they could be something, encouraged them to do more and overcome. Often the exact opposite.
Yet amid the unkempt yards and un-maintained houses you'll still come across a family here and there who has some pride. They're not brilliant but they work, pay their bills and manage their money, and they may be working-class poor but they're far better off than their neighbors. Walk in the house and talk to the folks and the children and ask "what is different here?"
To me the answer is "hope". They have hope than if they work at it they can better their situation, and they DO work at it. The parents teach the children to work at life; they make sure they go to school and do their homework and chores; the atmosphere in the house says "it's possible" instead of "why bother".
Sometimes one of their kids makes it to college or opens a small business.
That only happens if you try.