Concerning Yemen, the Yemeni government does not have control over all of the provinces. Tribes control some provinces, and they are historically not very cordial to any central government authority. Elements of al-Queda control the Saudi border region, and skirmishes have been going on there for well over a year.
I don't believe the Yemeni government can capture this individual, nor do I believe a law enforcement agency such as the FBI could do so either. As far as Special Ops forces it could possibly be done (if intelligence was airtight with feeds in real-time), but quite iffy. Such a Yemeni operation would be equivalent to doing likewise in the lawless Afghanistan/Pakistan border region.
Under Bush and now Obama, the US has opted for deadly Predator strikes against terrorists. It negates combat deaths and injuries, and also the possible capture of US military personal which would be a political hot potato/nightmare.
Concerning treason, IIRC the US State Department has the legal power to determine that certain odious activities of US citizens abroad are treasonous and strip such an individual of US citizenship. The individual would have to return to the US to dispute such a finding, which in this case is extremely unlikely to occur.