Saratoga's track/dirt was said to be deeper and slower than most other tracks. And some horses don't like that, so a lot of the champions did get beat up there. I don't think it bothered Secretariat though. IIRC he ran there as a 2 year old and won. I think that day he lost to Onion it was simply he was sick, and they shouldn't have run him.
BTW a friend of mines father bet $2,000 on Secretariat to win that day. Boom!