You can't have your cake and eat it too. If you want blacks and Hispanic students to be judged as individuals, then you can't have a system which allows unqualified blacks and Hispanics to be admitted as though they met the qualification bar. If you admit unqualified blacks and Hispanics then it is only rational for all the other qualified students to presume that when they encounter a black or a Hispanic student that that student is not qualified to be a student.
Look at the data I provided on admission into the law schools in Arizona and Nebraska. Every single black student, including the brightest among them, scored lower on the LSAT than the dimmest bulb among the white and Asian students. Students know the score. They'll judge rationally.
Now take a look at the culture that dominates the Military and Professional Sports. The Military institutes IQ tests before enlistment and refuses to enlist unqualified applicants. The racial aspect with respect to competence is pretty much non-existent in the military. The same too with pro-sports. Every athlete is on the team because they earned it by being a stand-out athlete. The race-competence-stigma mess doesn't exist.
Get rid of affirmative action and stop cooking the admissions process to achieve diversity and you'll likely find the same change in attitudes. Every black and Hispanic who is enrolled in university will be seen as being there because they had the competence to qualify for admission on equal terms with every other student. There would be no basis to assume that they're as, now how did you so elegantly put it, oh yes, "dumb as ****."
Affirmative Action stigma is quite
real.