And in the case of the UBS scandal, voluntary disclosure and payment of tax would go on your tax return as a special item. So tax records would show that he either got caught or voluntary came forth and paid the taxes he would have owed had he paid them on time plus interest and penalties. Penalties/interest generally come with some form of form showing what they were related to. Same deal for the KPMG but he might be able to sue KPMG for restitution if he could prove he didn't know they were illegal.
Merely because the IRS knows doesn't mean we know. Tax records for the most part are private information for everyone but non-profits who are legally required to release them.
You can check guidestar for free to see many nonprofits. This year will be especially interesting as many of the Superpacs will file and release their returns to the general public.
Romney's got to be hiding one of those if he's willing to actively add fuel to the rumor he paid little taxes legally. You don't add fuel to a rumor that hurts your image unless disproving that rumor results in an even bigger dent to how voters view you. I've never heard of a campaign anywhere where the subject of the bad image rumor actively helped his opponent in propagating the rumor against him. If Romney has one or both, it makes perfect sense to let the Democrats go hog wild on him. The alternative of letting them know he got caught is soooo much worse.
I've brought this issue up with hard core partisans and they just stop critically thinking about it. Apparently HELPING your opponent tear you down is a NORMAL activity in a campaign. It seems a great many people have simply just stopped thinking.