"The most important thing is for us to find Osama bin Laden. It is our number one priority and we will not rest until we find him."
- G.W. Bush, 9/13/01
If Rumsfeld ignored the President's prime directive (and it seems pretty clear he did) he should have been removed from office immediately following the Torabora screwup.
Of course, we know now that putting bin Laden out of business never really was the president's number one priority. It's been documented that Bush had a giant hard on for Saddam Hussein and wanted to invade Iraq since well before 9/11.