Obama spoke separately to Seattle's KING television and answered a question that the reporter said was given to her by an Alaska Airlines passenger who wrote it on a napkin.
The questioner wanted to know whether, when he deals with his Republican opponents, "are you altogether too reasonable?"
Some Democrats believe Obama has been too accommodating to Republicans on some issues. Those worries have grown as the White House seeks to negotiate a debt and deficit deal with Republicans that may include slowing the growth of popular entitlement programs.
Chuckling, Obama said he has sought to tamp down the political polarization of recent years.
"Are there times where I say, 'Boy, these folks are driving me nuts?' Absolutely."
"Are there times where I think that they are just trying to score some political points instead of solving problems? Absolutely."
He said his job is not to be "goaded into a food fight."
"There are times where I'm pulling my hair out. You notice it's gotten much grayer than it was," he said.