Present day EVs don't take that long to charge (basically the time to go to the bathroom and check email), but they don't have enough parallelism so at a busy location you can end up waiting for an open charger. There are orders of magnitude more gas pumps than public chargers.
when you need a charger there is rarely any choice of location. Charging at home doesn't help when you are not home. Those rare public chargers is what you need and they can be hard to find.
This varies a lot based on where you are. For the trips I usually take around in Texas where I need public chargers, I actually usually do have a choice of chargers. I understand this isn't universally true in other parts of the US though.