A rather large sector of software where you don't throw hardware at the problem is embedded software. Shipping hardware that is 25€ more expensive times a million is often much more expensive than optimizing the software.
I've also seen the opposite, quite often actually: Cheaping out on hardware that is going to ship maybe 1-10k (very expensive) units, then spending hundreds of thousands on optimizing software to make it not even good, just less painfully slow. The i.MX6 chip with its weak GPU and corresponding wonky drivers is an especially popular way to get user interfaces that can't keep 60 fps.
The i.MX6 is certainly a poor choice today but do you know of any good contemporary alternatives? Honest curiosity because personally I can't think of any medium-power (or at least thermal dissipation), (Mainline-) Linux capable, well (and openly) documented processor available long-term in low/medium quantities from a proven vendor.
I've recommended Toradex Tegra 2 modules to a customer (I got involved early enough to recommend hardware - a rare case) and they seem to be quite happy with it. Just a few euros more per module than i.MX6 from the same module vendor. The GPU is unsurprisingly (with nVidia making the whole SoC) pretty good. Most everything needed for software support except the user-space graphics driver is even open source. I am not an nVidia fan because of their desktop Linux driver shenanigans, but I strongly prefer Tegra 2 over i.MX6. By the way, i.MX6/6+/7 with the etnaviv driver might also be alright. I've just never used anything but the proprietary "gal3d" driver.
Regarding mainline Linux support, AFAIK you don't get mainline Linux support anyway with i.MX6 and the gal3d driver. You can only use mainline with etnaviv, which is semi-officially(?) supported by Pengutronix. I've heard others say good things about etnaviv.
I've also seen the opposite, quite often actually: Cheaping out on hardware that is going to ship maybe 1-10k (very expensive) units, then spending hundreds of thousands on optimizing software to make it not even good, just less painfully slow. The i.MX6 chip with its weak GPU and corresponding wonky drivers is an especially popular way to get user interfaces that can't keep 60 fps.