No, it really means you care. Use the proper grammar and you won't look ignorant.
You COULDN'T care less, is what you mean.
Just because the world is full of ignorant fools who wouldn't know a how to use a semicolon if their life depended on it, doesn't mean you need to join their ranks.
If you want to clearly elucidate your thoughts, then do so.