Where by "failure" I mean its gradual displacement from the center of the moral and intellectual life of they civilization.
To define these options bit:
Poll option one is the conservative answer. It holds that Christian belief would be as predominant today in the West as it was in 1913 if it were not for the conscious, deliberate machinations of a small group of secularizing elites promoting atheism and amorality.
My thoughts: This is the least tenable of the four options I've provided, in part because 'the elite' in the West has never been anti-Christian. To be sure, they are opposed to fundamentalism, but only because it is at odds with liberal-capitalist notions of 'progress'. The invocation of the defense of Occidental Christianity during the Cold War is proof-positive that Western elites want generally to employ Christianity to their own ends.
Poll option two is the liberal answer, the "secularization thesis". According to this theory, Christianity is doomed to deplacement, as are all religions eventually, by the gradual and wholly unconscious forces of mental and mechanical progress.
My thoughts: This is almost as problematic a solution to the question posed as the first answer. It assumes a great deal of the structure of Christian ideology - progress towards a "new Heaven and a new Earth", an eventual end to history, and so on - while draining it of its metaphysical content.
Option three is what I call the Nietzscheite option: Christianity has failed because it is inherently flawed. It can exist only among theoppressed, and as soon as a people become strong enough to shirk ofc a collective sense of inferiority it will abolish the correspondent notimon of individual existential guilt that informs Christianity.
My opinion: This is the view I hold closest to. Christianity, in a very real sense, requires weakness to thrive (it is little wonder that Christianity is ascendant today only in the impoverished Third World nations of sub-Saharan Africa, Latin America, and the American South). A strong people wants a religion of strength and severity.
Option four: The Marxist solution. Christianity belongs at the historical latest to the age of feudalism; the rising capitalists of the eighteenth and nineteenth centuries sought initially to do away with it altogether, as a reminder of the hated age of the nobility, and retain it only as a matter of practicalg politica expedience.
My opinion: This is superficially similar to the liberal answer, relying on notions of deterministic 'progress', but avoids some of its problems by acknowledging the fact of necessity and human action in historical processes, rather than ascribing all history to forces largely independent of men.
Whether or not Christianity has "failed" at all is frankly a matter of some debate. However, going off of what you have presented, I think the reason behind the relative decline in Western Christian influence in recent centuries can be primarily attributed to a combination of options one (Conservative), three (Nietzscheite), and four (Marxist).
The Conservative explanation is correct in pointing out that a deliberate and persistent movement to marginalize and discredit the view of ethics and the world in general presented by Christianity in favor of more "progressive" ideas can be observed to exist among the Western "intellectual elite." The Marxist explanation, for its own part, is correct in pointing out that this movement originated during the Enlightenment era among Liberals, Libertines, and Capitalists who sought to remake the prevailing European social and political order of the 18th and 19th centuries in their own, overwhelmingly secular and materialist, image.
While the popularity of this goal has waxed and waned over the course of the decades since in the face of the different challenges Western Civilization has come to face (i.e. as you pointed out yourself, the threat of atheistic, anti-Liberal, and anti-Capitalist Communists resulted in a strong resurgence of pro-Christian messages in mainstream culture during the mid 20th Century), it has never gone away completely among the ranks of the Western intelligentsia. In today's world, for example, the banner of the anti-Christian, anti-theist, and anti-traditionalist cause is unsubtly (and some might even say,
proudly) borne by academia and the mainstream media alike.
However, at the end of the day, you are correct in pointing out that the plots and machinations of society's elite can only carry things so far by themselves. This is where the Nietzscheite explanation comes into play.
It is absolutely no secret that Christianity, and even religion in general, thrive most in times of want and adversity. The New Testament itself admits to this fact.
Matthew 19:24 said:
"Again I tell you, it is easier for a camel to go through the eye of a needle than for someone who is rich to enter the kingdom of God."
The simple fact of the matter is that human beings are shallow and selfish creatures who will always look to their own gratification before any loftier goal. As such, most of them have no real need to look to the supernatural for validation or comfort when earthly goods are present in abundance. The idea of heavenly paradise in the hereafter often seems trite when one can find hedonistic pleasures aplenty right here on earth.
In case you haven't noticed, earthly goods and hedonistic pleasures ad infinitum are precisely what our modern consumerist society seeks to, and often does, provide its citizenry.
As standards of living and leisure in the Western World continue to climb ever higher, so too has the relative importance of religion begun to fade into the background of social consciousness for many people. After all, no one wants to be lectured on the value of such concepts as "temperance," "responsibility," or "morality" when living in a society which seemingly allows one to bypass them all entirely.
This is a phenomena which, as I have already pointed out, academia and the media have been only too happy to support in the interests of furthering their own agenda. They encourage and enable such behavior with the increasingly crass and risqué material they continue to churn out year after year.
Keep in mind, however; that none of this is to say that any of these developments have necessarily been for the better. Given the general state of the Western World in our current era, it would actually seem to have been much for the worse.
Rampant apathy and hedonistic excess are slowly but surely bringing Western Civilization to its knees.