You're not wrong, and ISPs are already doing it everyday. The only reason Skype works in the US, for instance, is because Microsoft shells out big $$ to ensure that Skype packets aren't selectively blocked.
What the ISPs need to recognize here is that the Internet is not just the bad ol' USA, but other countries have networks (i. e. fiber) through which IP traffic runs, and a content provider's market consists of those countries as well.
So if an ISP tries to block/slow a content provider's packets, that content provider will still claim a large market share in countries (i. e. Iceland, Norway) where its packets
aren't slowed.
During the time Skype became big, it was still very much a European craze, and it was not until later that it became popular in the US, and only at that point can US ISPs step in to address it.
BOTTOM line: the ISPs geographic limitation gives it limited power to exploit its oligopoly to stifle competition from content providers