It is all in perception. Patrick Henry pretty well summed up all the comments and stated beliefs of the Founders on the subject when he said:
"It cannot be emphasized too strongly or too often that this great nation was founded, not by religionists, but by Christians; not on religions, but on the gospel of Jesus Christ. For this very reason peoples of other faiths have been afforded asylum, prosperity, and freedom of worship here."
--The Trumpet Voice of Freedom: Patrick Henry of Virginia, p. iii.
If you ask people to name the Muslim countries of the world, we would all rattle off most of the middle eastern countries plus others that we would easily recall.
If you ask people to name the Jewish countries of the world, we all would immediately name Israel.
If you ask people to name the Atheist countries of the world, most would name China and North Korea.
And if you ask people to name the Christian countries of the world, the USA would head the list followed by Canada, Mexico, all the central and South American countries, all of Europe, and many others.
Even in the most recent polls including all the big polling groups, more than 90% of Americans profess a belief in God and roughly 78% label themselves Christian. The next largest group are the no religion people and all the rest comprise very low percentage.