I would actually argue that the average person in modern, western contexts is actually dumber than the average aborigine. They are less than equal in basic problem solving (using tools to an end), less physically capable, and are placed into societies with ideological constructs they choose to side with and cannot see out of.
Intelligence, is rather subjective, however. We could be addressing booksmarts, in which the subjects seem more intelligent after acquiring higher knowledge, or we could be addressing the smarts on standardized army entrance testing. Cause if you get less than ace on those, you're a dumb dumb.
I teach tae kwon do and I've done TKD in a third world country. The western students arent as coordinated, sharp, quick to learn or capable of following directions or controlling themselves like the third world students are.
(Though the question is whether people have gotten smarter over the past few thousand years)
The answer would be no. The potential for intelligence is the same. Dolphins have larger brains than us, yet they cannot read.
Elaborate on this underclass you speak of.