If you take two speed-ups which individually would be 5% over the initial performance and apply them both, you get 10.25%, not 10%. The point is that it can scale.
A number of discrete improvements is also more resilient. If a single perf improvement needs to be removed, only a 5% gain is removed. If the big 50% gain is monolithic it is much more susceptible to disruption.
you're statement needs a lot of qualifications. It is in only very specialized cases where this is true.
Let's say my program takes time 1.0 and is made up of part A taking 0.5 and part B taking 0.5
Defining 5% speedup is weird in the first place, but let's say 5% speedup is taking 0.95 for the program and 10% speedup is taking 0.9 for the program.
5% speedup by improving part A => 0.45 + 0.5 => 0.95 runtime
5% speedup by improving part B => 0.5 + 0.45 => 0.95 runtime
making both improvements:
0.45 + 0.45 => 0.9
So, two speed-ups which individually would be 5% over the initial performance gives us a 10% speed up.
percentage speed ups do not necessarily compound. And in practice, usually they do not.
