Yes, but with certain conditions
I'm not arguing about the affair affecting his ability to do his job. We don't know the whole facts just yet, but there has been talk of classified information possibly having been passed along by him to his mistress. Until everything is all cleared up, I am reserving judgement on whether or not he should get his job back.
Hail to the King baby!
The poll's option: "Yes, with conditions". I'm wondering what those conditions might be. Like, no more bangin' chicks without telling the CIA?
"Those who do not learn from history and condemned to relive it".
"There are those who will debate the necessity of wilderness, I will not, either you know it in your bones or you are very very old". Aldo Leopold - Sand County Almanac
The head of the CIA is a not a regular occupation. You can't simply say "I am off the clock", do something stupid, and then claim that it doesn't effect your job performance. A single careless statement said in the most casual setting has the potential to reveal the wrong information and get someone killed. Foreign intelligence services are probably actively looking for vulnerabilities in such a powerful person to exploit.
Petraeus demonstrated his job is not his number one priority when he took actions he knew would torpedo his career. He judgement was so terrible he couldn't even be bothered to put any effort into concealing his wrongdoings.