I mainly care about games when it comes to high-performance programming, so this is not an issue for me. Consoles are obviously fixed platforms, and on PC any game is likely to have high enough GPU requirements that you can compile your executable against a SSE3 min-spec target without excluding users.
The instruction set available is often not the most relevant thing. In many cases, the optimal code will differ wildly between CPUs regardless of the available instruction sets. Let's look at some quick examples of performance characteristics:
Athlon 64: Slow SSE2 unit, MMX is often faster than SSE2 for many functions.
Phenom: Very fast SSE2 unit, but missing SSSE3 support, so can't take use the same optimized functions in many cases as the Core 2 and above. Significantly higher instruction latency than Core 2, so needs significantly more pipelining.
Core 2 Conroe: SSE shuffle operations (punpck, etc) are excruciatingly slow: 4/2 for shuffles that use both arguments as input data (e.g. punpckldq), 2/2 for those that use only one argument (e.g. pshufb). Cacheline-split loads are extraordinarily painful (equivalent to an L1 cache miss, or ~14 cycles).
Core 2 Penryn: Same as Conroe, but shuffles are fast now (1/1 for everything basically).
Nehalem: Cacheline-split loads are cheap now (2 cycles), and shuffles have doubled throughput compared to Penryn (1/0.5).
This doesn't even get into the more instruction-specific messiness, like how some CPUs like movddup and movhlps on integer data while others don't. It's not uncommon to have 3 or 4 assembly functions just to cover the latest CPUs -- not even counting old ones. Fortunately for our sanity, these are usually templated from a single function, with small changes to the relevant areas created via macros.
A gaming PC is already so fast compared to consoles that I'm generally not super concerned about those discrepancies. Compared to the big gains in going from no SIMD to SSE3-level SIMD, it's very minor. As I already said, if you have a few speed-critical inner loops (not generally the case in games but more the domain of programs like x264 and Bink 2) then go ahead and write the whole loop with a good macro assembler. But even then, Bink 2's encoder uses SSE in parts and is highly multi-threaded, and yet Jeff hasn't at all bothered with your level of per-CPU customization of the SSE code, and I doubt he ever will.
Most programs aren't like codecs or even like games but could still make great gains from a properly administered dose of SIMD intrinsics.
The instruction set available is often not the most relevant thing. In many cases, the optimal code will differ wildly between CPUs regardless of the available instruction sets. Let's look at some quick examples of performance characteristics:
Athlon 64: Slow SSE2 unit, MMX is often faster than SSE2 for many functions.
Phenom: Very fast SSE2 unit, but missing SSSE3 support, so can't take use the same optimized functions in many cases as the Core 2 and above. Significantly higher instruction latency than Core 2, so needs significantly more pipelining.
Core 2 Conroe: SSE shuffle operations (punpck, etc) are excruciatingly slow: 4/2 for shuffles that use both arguments as input data (e.g. punpckldq), 2/2 for those that use only one argument (e.g. pshufb). Cacheline-split loads are extraordinarily painful (equivalent to an L1 cache miss, or ~14 cycles).
Core 2 Penryn: Same as Conroe, but shuffles are fast now (1/1 for everything basically).
Nehalem: Cacheline-split loads are cheap now (2 cycles), and shuffles have doubled throughput compared to Penryn (1/0.5).
This doesn't even get into the more instruction-specific messiness, like how some CPUs like movddup and movhlps on integer data while others don't. It's not uncommon to have 3 or 4 assembly functions just to cover the latest CPUs -- not even counting old ones. Fortunately for our sanity, these are usually templated from a single function, with small changes to the relevant areas created via macros.