The idea that the death penalty does not deter crime I find mostly invalid - the death penalty is not structured in such a way as to deter crime. Deterring crime requires that a punishment be swift, sure, and public. The death penalty as it is currently structured is none of these things. It could be, which means that the lack of deterrence falls (as does the cost and time period) on workings of the government, not the death penalty itself.
Like you, I was formerly strongly death penalty, and am since moving away from it.
The strongest arguments I find in favor is that it could
theoretically be restructured to provide deterence to crime in a manner that is not cost-prohibitive nor lengthy, and that victims and their families deserve closure. There are people who deserve to die, and there are killings that are not wrong.
The strongest arguments I find against is that we are all sinners, and deserve more than we probably get, and are in desperate need of a chance at redemption. If one who hates his brother is a murderer, do not all murderers need the same forgiveness and redemption that those who have hated a brother have access to? It is hard to find pity for some people. But that is precisely what I am called by Christ to do (He's got you coming and going, that guy does) - love even your enemy, He says. The other argument I find compelling is the one you mention - that while the process is infallible, the execution of it is final; it is a putting of total power into a fallible structure when doing so is not necessary (as it can often be in self-defense, crime fighting, or war).