I don't have a problem with it. Politicians are still human, the last time I checked. They goof up. The vast majority of Americans swear, so I don't see a problem with Biden swearing, especially if it's not an ad hominem directed at someone. They already have a Secretary of State for that.
If anything, I encourage Democrats to swear. They're not the religiously pious, holier-than-thou folks who like to shove morality in your face like a banana cream pie at a circus. Just dodge the Bible-thumping Republican who will chase you with holy water afterward and yell, "The power of Christ compels you!".
Damn, that would be funny.