> I don't get why it seems like languages are designed for convenience then later down the road billions of hours are spent trying to get better performance.
It's easy to explain.
1. Most languages are designed for personal satisfaction of the author, not for commercial success. Perl, PHP, and Ruby were all just personal projects. Python was a language intended for education, then pivoted towards Perl's niche in the 90s.
2. The vast majority of languages that are designed will fail and be abandoned without anyone having ever heard of them.
3. Designing a pretty and elegant language is hard, but fun. You don't even need to implement it.
4. Designing a language that is deliberately amenable to optimization is also hard, but not fun (especially when optimization has to be prioritized over elegance). It's also hard to prove that your design actually yields good performance, which is hard and not fun. Furthermore, the only point of designing such a language is to hope to someday fully implement it, which is very hard and extremely not fun. And hoping to implement such a high-performance language in a cross-platform way is, for any single person, basically impossible.
The modern resurgence of high-performance languages like Rust is because, thanks to retargetable backends like LLVM, the idea of a hobbyist designing a new high-performance cross-platform language has gone from "basically impossible" to "unbelievably tedious though possible but also on some level your language has to basically pretend that it's still C". And also the consolidation of platforms for the past few decades now mean that there's fewer targets to support in the first place, and hence fewer C compilers than ever, so if you just want to transpile directly to C you can, though you'll still spend the rest of your life adding hacks to your compiler to work around bugs in every combination of platform/C compiler/C compiler version out there.
> Designing a pretty and elegant language is hard, but fun. You don't even need to implement it.
The parts that make Python slow make it easier to hack on, but they don't make it elegant in any meaningful sense.
> Designing a language that is deliberately amenable to optimization is also hard, but not fun (especially when optimization has to be prioritized over elegance).
Elegant languages are usually amenable to being efficiently implemented without the implementor having to do anything special. That's because elegant languages guide you towards using the simplest, least powerful features that are powerful enough to solve your problem. For example, in Standard ML, I don't use higher-order functions that much, not because they suck (they don't), but rather because first-order functions and functors (which can't be recursive) are powerful enough for most things I want to do.
On the other hand, in a language like Python, it can be very difficult to find features that solve exactly your problem and nothing more, so often you have to reach for super-powerful hammers (operator overloading, metaprogramming, etc.). These hammers are almost invariably more general than the intrinsic demands of your problem, but, since the language designer can't provide an implementation that only works for your use cases, you must pay the price of the unnecessary generality of the feature.
It's easy to explain.
1. Most languages are designed for personal satisfaction of the author, not for commercial success. Perl, PHP, and Ruby were all just personal projects. Python was a language intended for education, then pivoted towards Perl's niche in the 90s.
2. The vast majority of languages that are designed will fail and be abandoned without anyone having ever heard of them.
3. Designing a pretty and elegant language is hard, but fun. You don't even need to implement it.
4. Designing a language that is deliberately amenable to optimization is also hard, but not fun (especially when optimization has to be prioritized over elegance). It's also hard to prove that your design actually yields good performance, which is hard and not fun. Furthermore, the only point of designing such a language is to hope to someday fully implement it, which is very hard and extremely not fun. And hoping to implement such a high-performance language in a cross-platform way is, for any single person, basically impossible.
The modern resurgence of high-performance languages like Rust is because, thanks to retargetable backends like LLVM, the idea of a hobbyist designing a new high-performance cross-platform language has gone from "basically impossible" to "unbelievably tedious though possible but also on some level your language has to basically pretend that it's still C". And also the consolidation of platforms for the past few decades now mean that there's fewer targets to support in the first place, and hence fewer C compilers than ever, so if you just want to transpile directly to C you can, though you'll still spend the rest of your life adding hacks to your compiler to work around bugs in every combination of platform/C compiler/C compiler version out there.