The lazy genius janitor might not keep the windows as clean on a daily basis, but when the and the building catches on fire or something, they have the potential to understand the situation and respond effectively.
You also give them a huge head start in what they are good at or interested in. Being good at one thing is what matters in the real world. These subjects do not have general practicalities. Sure, you can use your knowledge of gerunds while twittering or your calculus skills at the grocery store, but they aren't necessary. Very few jobs require both higher math and knowledge of literature. It is a waste of time to teach both to someone who is only good at or interested in one.That's not true. Science and advance Maths can be very useful in everyday life if only people are aware enough to employ them. Like understanding the dynamics of forces, how weathers are formed etc Understanding algebra makes decisions much easier, you can use it when you go grocery shopping - comparing prices and planning your budget etc. So it's a "waste of time" because people don't apply them, not because they are useless.
School teach all these things because they have general practicalities, and it's not certain in the future what each students will do. By narrowing the education early, you take away their choices in the future.