On the other side, even very sophisticated music can be enjoyed on a non-intellectual level (you can like and love classic music and good jazz, yet not know much about it... that's what I do most of the time when I listen to that kind of music
). Sure, a lot of clasical music and a good part of jazz offer more than just that, unlike generic commercial music, so you can study it deeply and yet find new interesting twists in it amateurs don't recognize. But you don't need to be an expert to love Mozart.
And then, when jazz music was new, there was a huge outcry by lovers and experts of classical music. I remember having read that one very famous professor and critic for classical music said "it's not music, it's vulgar noise, the sound of uncivilized sexual passions" and "negro music" or something to that extent. There was a similar outcry in the establishment when certain classical music was new, like when ... was it Mendelssohn? made the shift from classicism to romanticism, or when Wagner came up with his pompous operas. Now they're considered titans of classical music, and jazz a sophisticated style. Probably it always takes a generation that grew up with a new style to get old, before a new style is generally accepted.