A stint of physical labor is certainly a valuable experience, I long for it too, but it DOES eventually break the body if it is done year after year for decades.
I suspect that years of meaningless work just shuffling money around for wealthy people and institutions has a similar effect on the psyche, even if it is obscenely compensated.
I've concluded when people say "break your body", they usually refer to the slowly accumulating string of little slip-ups. You turn the wrench too hard and bash your knuckle. You grab something that was still too hot. You drop something on your foot. You lift something the wrong way and pull something. No one slip-up is a real big deal, but you can't help but make a few, and they slowly add up.
I suspect that years of meaningless work just shuffling money around for wealthy people and institutions has a similar effect on the psyche, even if it is obscenely compensated.