It's a long story.
The word Yule, mean "wheel" (lit. "to turn"), and was the feast of light, taking place on the winter solstice, when a turn of the world, taking one year, was complete.
In Nordic mythology, the one who turned the wheel was Thor. The constellation the big dipper is called "The Carl's Wagon" (not the name Carl, but as in "a carl" i.e. a free man), and the Carl was Thor, pulling the world around the world axis; variously represented as the world tree "Yggdrasil", the ancient German world pillar "Irminsul", and the Swedish mid-summer pole (don't know if it has a name) among others. Interestingly, the Big Dipper was called "The Plough" in medieval England, and Thor was the god of freemen and farmers as well as thunder, so there may be some old pagan connection there. The Christmas tree is most likely an old representation of the world tree, with a star added to symbolize the star over Betlehem, and at the same time de-paganizing it.
Goats and rams play a large part in various ancient cosmologies from the northern hemisphere, likely because it was the first astrological sign of the year until around 1 AD (a fact I'm sure did not escape Christendom). We see an earlier similar focus on bulls in older cosmologies after the age of Taurus (ended c. 1800 BC). Goats draw the wagon of Thor, a goat eats of the world tree and provides mead for the fallen heroes. Particularly for Christimas, the world-serpent Jormungandr (lit. "huge monster") is often depicted with goat or rams' horns. The story of Thor feeding a bulls head into the mouth of the horned world serpent was of huge cosmological importance, as this tied together the last and first signs of the year together, completing the turning of the year and ushering in a new one. For some not too obscure reason goats also represented fertility (in Christendom; the sin of Lust), so a goat was a natural symbol for a pagan new year's celebration.
So as you can see, we have at least three connections between goats and old christmas. Astrological, mythological (Heidrunn the mead goat), and as a generic fertility symbol.