It does sound like a lot but it depends on the actual wall clock time. If your run time goes from 10 days down to 1 day then yes, it matters a lot. If it goes from 1s down to 0.1s it might not matter so much.
Of course, but if you observe 3x to 10x performance gains across the board, you will have some programs that run in more than 1 second where it may be worthwhile.
