The price, power, and size. Make it cheap, low power, and small enough for mobile. One way to do this is inference in 4, 2, 1 bit. Also GPUs are parallel, most tasks can be split on several GPUs. Just by adding they you can scale up to infinity. In theory. So datacenters aren't going anywhere, they will still dominate.
Another way is CPU+ + fast memory, like Apple does. It's limited but power efficient.
Looks like with ecosystem development we need the whole spectrum from big models+tools running on datacenters to smaller running locally, to even smaller on mobile devices and robots.
