That's a solid point. I've generally avoided password managers because not knowing my (unique-per-service, strong) passwords makes me nervous in exactly the same way as not actually knowing the phone numbers of the most important N people in my life.
You'll get over that little hurdle once you realize that you can dump the anxiety of remembering a hundred password variants for different sites. And realistically speaking, you're probably not even using a hundred variants...or possibly even 10. If you're memorizing passwords, chances are your re-use frequency is nonzero.
What's important is to keep a backup of your password database in a few places. I use KeePass because I have no desire to keep passwords, encrypted or not, in a cloud service. I also don't find value in browser integration (possible attack vector?). I'm generally very DIY-inclined anyway. Your preferences may vary.
I guess you aren't familiar with KeePass. If your KeePass database is pwnd, that means your box has been pwnd since the database is stored locally and not any cloud provider (unless YOU put it there). This means you have much bigger problems and is not a shortcoming of KeePass, itself.
As a full disclaimer, there are some issues with KeePass [1], but known issues are detailed in full by the project and are available for review.
