Apple has been stingy with RAM in consumer hardware. RAM prices will continue escalating, some analysts say into 2030, and this will make it more difficult to build next generation phones with sufficient memory for meaningful ML workloads.
I hope there's some kind of inversion in the current chip economics, because I love distributed/democratized/private compute, but currently cloud based LLM inference seems to be much more viable. I don't see local llms meaningfully viable for the general usecase in the near future.
I'm not sure about that. I can run Qwen 3.8 27B at acceptable speed on an M1 MacBook Pro from 5 years ago. Thats Opus 4.6 Quality, on 5 year old consumer hardware. I think the trend is the opposite - smaller models that can run on hardware people already have are getting better and better.
There’s something I’d don’t understand about this. Apple’s RAM is on their own die. Is the price of RAM an indicator for the cost of die space in general?
I hope there's some kind of inversion in the current chip economics, because I love distributed/democratized/private compute, but currently cloud based LLM inference seems to be much more viable. I don't see local llms meaningfully viable for the general usecase in the near future.