Isn’t this more a problem of running inference for closed-source AI services? Considering we have open models that are now within throwing distance of GPT-4, it would make sense to do those workloads outside data centers running a single service (or even on people’s devices). Of course training still requires lots of resources, but that doesn’t have to happen nearly as frequently.
The rate of efficiency optimizations that have occurred in the open source community over the past two years I think call that into question. Services like OpenAI and Anthropic have the highest performing models, but since we have no ideas about what they really are or how they do inference, we can’t say that they’re necessarily more efficient than open source. In fact, people doing things in walled gardens, motivated by maximizing market share, subsidized by big players, are more likely to be doing things inefficiently than research being done out in the open.
