You are making the argument that he should not have been a Pope, not that he was not a Pope. To the Catholics (which I am not), who get to decide who is Pope and who is not, he is considered to be the Rock upon which the church was built; the first Pope. Pope has no meaning, except to Catholics... they created the function. It is an office created by a religion. The office, by definition, is the lineage of Peter, hence he is the first.
I can make an argument that the Rosenbergs should not have been executed for spying. I can not make an argument (well, if I do, it would be a ridiculous one) that they were not executed for spying. Similarly, you really can't make the argument that he wasn't a Pope....and, really what difference does it make? Why would you care if the Catholics consider him a Pope or not?
Of course Peter was a Jew, which was my original point: Peter, the who is widely acknowledged as the first Pope, was a Jew. What is a Christian anyway, but someone that believes that Christ is the Messiah and thus is either a completed Jew or was grafted onto God's promise for his chosen people.