This is also one of my pet peeves. I keep thinking “people do understand progressive taxation, right?”, but in discussions like this, I realize: no, maybe they don’t. If you’re a single filer who makes $180K in the US in 2022, you’re in the 32% tax bracket. But your effective federal tax rate is just over 21% — and that’s assuming you’re only taking the standard deduction.
Are you sure that they're mistaking marginal tax rates for effective tax rates, rather than you mistaking gripes about taxes in general for gripes about federal income tax in particular? In my state, a single filer making $180k pays 34.6% effective tax on income - 19.1% federal income, 6.4% FICA, and 9.2% state income tax. (And maybe 4% more from property taxes if they own a house in a major city, although that's a somewhat different subject.)
This is not to say that rich people are actually poor because of taxes, obviously, but it seems more likely that people would talk about these things in a somewhat inaccurate way than that they would be totally wrong about what they actually pay.
