- High speed copper physical layers are cheap - Thunderbolt today has 40Gbps PHYs, and Infiniband/10GBE both extremely inexpensive compared to a few years ago.
- Their chips are getting smaller and more power efficient, and the Atom finally has an ECC part.
- As clockspeed is not longer hockeysticking up, people are transitioning in droves to multi-system parallel task model, which favors lots of medium speed, efficient cores.
In short, in the near future general purpose computing is going to look more like HPC clusters, because the software side of things has, in most cases, caught up.
Latency is a significant issue with this approach. A SSD doing 100,000 IOPS per second that's 3 feet from the CPU is noticeably faster than one that's 100 feet from the CPU.
HPC tends to solve this with Infiniband, which is optimized for very low latency.
I'd assume that Intel's long term trajectory is to have CPU/cache/memory/flash all on one system-on-a-chip "module" that is the smallest replaceable subunit. Larger dedicated storage is delivered over the network interconnect in a tiered architecture.
Since the speed-of-light delay for 100 feet is 100 ns, and SSD latencies are on the order of 10-100 us, this shouldn't be a fundamental issue. Maybe it's caused by the latency to the disk cache (DRAM)? In which case perhaps you could separate that from the disk, and move it closer to the CPU.
At 2Ghz the speed of light in a vacuum delay is a clock cycle every 15cm. Current fiber and copper transmission is about 70% of c, so that's a clock cycle every 10.5 cm.
A few clock cycle might not matter for bulk storage, but Intel is also talking about separating main memory from individual processors. There individual clock cycles do matter. Witness the rise of low latency premium RAM.
100 feet / speed of light = 101 nanoseconds, round trip that's 202 nanoseconds, but electricity is ~66% of speed of light ~= 300 nanoseconds or .3us. (Fiber is also ~1/3 slower than the speed of light both because it's not a vacuum and the path is not strait.)
Now 10 us vs 10.3 us might not sound like much but it's still 3% slower. And it get's worse when you look at high end DRAM based SSD's which can be faster than 10 us.
According to Wikipedia, it depends on the insulation:
== SNIP ==
Propagation speed is affected by insulation, so that in an unshielded copper conductor ranges 95 to 97% that of the speed of light, while in a typical coaxial cable it is about 66% of the speed of light.
At data rates, transmission lines are used in copper. This is why CAT-5e has higher requirements on the twisted pairs than CAT-3 (or plain old phone line).
66% turns out to be a surprisingly consistent approximation for both copper and fiber.
In a world of commoditized storage isn't latency always the issue (rather than capacity)? In a traditional setup it surely harder to tune the latency requirements for a particular app.
- High speed copper physical layers are cheap - Thunderbolt today has 40Gbps PHYs, and Infiniband/10GBE both extremely inexpensive compared to a few years ago.
- Their chips are getting smaller and more power efficient, and the Atom finally has an ECC part.
- As clockspeed is not longer hockeysticking up, people are transitioning in droves to multi-system parallel task model, which favors lots of medium speed, efficient cores.
In short, in the near future general purpose computing is going to look more like HPC clusters, because the software side of things has, in most cases, caught up.