> I'm skeptical of this logic. I think nobody would be able to learn mathematics if they had to keep it all in their head at once.

That's what functions are for.

> Instead we learn some, we internalise it, and then we can appreciate the new notation

Those new notations are functions, simplified abstractions of more complex processes that allow us to think in larger concepts and not sweat the details.

> Mathematics is beautiful. If anything, it's programming languages that have the catching up to do.

Perhaps, perhaps not.