My cynical take is that it'll works out just fine for the data centers, but the neighbouring communities won't care for the constant rolling blackouts.
Okay but even in that case the hardware suffers significant under utilisation which massively hits RoI. (I think I read they only achieve 30% utilisation in this scenario)
That article appears to be stuck behind a paywall, so I can't speak to it.
That's good for now, but considering the federal push to prevent states from creating AI regulations, and the overall technological oligopoly we have going on, I wonder if, in the near future, their energy requirements might get prioritized. Again, cynical. Possibly making up scenarios. I'm just concerned when more and more centers pop up in communities with less protections.
Current US debt to gdp is 124%, 38.6 trillion.
Japan too at 230-240%.
Bond markets in both are looking seriously unhealthy (Japan going via a Liz Truss moment at present).
If the AI bubble falls over, the US government is going to have to print 5 trillion to cover the bubble at least. The only option there is inflate away anyone holding cash.
If hte AI succeeds and people are replaced, the US government faces a massive fiscal cliff of a loss of tax receipts. They won't be able to service the debt and again will be forced to inflate away.
To service current debt projects, AI growth needs to return some 3.2-3.5%, it is currently 0.5%.
Bonds, equities, USD, and housing are all risk assets right now.
I am still a little skeptical about utilisation rates. If demand is so extreme, wouldn't we see rental prices for H100/A100 prices go up or maintain? Wouldn't the cost for such a gpu still be high (you can get em 3k used).
On "runpod community cloud" renting a 5090 costs $0.69/hour [1] and it consumes about $0.10/hour electricity, if running at full power and paying $0.20/kWh.
On Amazon, buying a 5090 costs $3000 [2]
That's a payback time of 212 days. And Runpod is one of the cheaper cloud providers; for the GPUs I compared, EC2 was twice the price for an on-demand instance.
That ASUS motherboard is far from the cheapest available. If using it makes the user liable for failure, a large part of the market is unsuitable.
For both the cooler and the motherboard, AMD have too much control to look the other way. The chip can measure its own temperature and the conceit of undermining partners by moving things on chip and controlling more of the ecosystem is that things perform better. They should at least perform.
The biggest discussion I have been on having this is the implications on Deepseek for say the RoI H100. Will a sudden spike in available GPUs and reduction in demand (from efficient GPU usage) dramatically shock the cost per hour to rent a GPU. This I think is the critical value for measuring the investment value for Blackwell now.
The price for a H100 per hour has gone from the peak of $8.42 to about $1.80.
A H100 consumes 700W, lets say $0.10 per kwh?
A H100 costs around $30000.
Given deepseek, can the price of this drop further given a much larger supply of available GPUs can now be proven to be unlocked (Mi300x, H200s, H800s etc...).
Now that LLMs have effectively become commodity, with a significant price floor, is this new value ahead of what is profitable for the card.
Given the new Blackwell is $70000, is there sufficient applications that enable customers to get a RoI on the new card?
Am curious about this as I think I am currently ignorant of the types of applications that businesses can use to outweigh the costs. I predict that the cost per hour of the GPU dropping such that it isn't such a no-brainer investment compared to previously. Especially if it is now possible to unlock potential from much older platforms running at lower electricity rates.
Why is there this implicit assumption that more efficient training/inference will reduce GPU demand? It seems more likely - based on historical precedent in the computing industry - that demand will expand to fill the available hardware.
We can do more inference and more training on fewer GPUs. That doesn’t mean we need to stop buying GPUs. Unless people think we’re already doing the most training/inference we’ll ever need to do…
Historically most compute went to run games in peoples homes, because companies didn't see a need to run that much analytics. I don't see why that wouldn't happen now as well, there is a limit to how much value you can get out of this, since they aren't AGI yet.
This just seems like a very bold statement to make in the first two years of LLMs. There are so many workflows where they are either not yet embedded at all, or only involved in a limited capacity. It doesn’t take much imagination to see the areas for growth. And that’s before even considering the growth in adoption. I think it’s a safe bet that LLM usage will proliferate in terms of both number of users, and number of inferences per user. And I wouldn’t be surprised if that growth is exponential on both those dimensions.
> This just seems like a very bold statement to make in the first two years of LLMs
GPT-3 is 5 years old, this tech has been looking for a problem to solve for a really long time now. Many billions has already been burned trying to find a viable business model for these, and so far nothing has been found that warrants anything even close to multi trillion dollar valuations.
Even when the product is free people don't use ChatGPT that much, making things cheaper will just reduce the demand for compute then.
> It has basically replaced search for most people.
Not because it's better than search was, though.
They lost the spam battle, and internally lost the "ads should be distinct" battle, and now search sucks. It'll happen to the AI models soon enough; I fully expect to be able to buy responses for questions like "what's the best 27" monitor?" via Google AdWords.
Over the long run maybe, but for the next 2 years the market will struggle to find a use for all this possible extra gpus. There is no real consumer demand for AI products and lots of backlash whenever implemented eg: that Coca Cola ad. It's going to be a big hit to demand in the short to medium term as the hyperscalers cut back/reasses.
In a thread full of people who have no idea what they're talking about either from the ML side or the finance side, this is the worst take here.
OpenAI alone reports hundreds of millions of MAU. That's before we talk about all of the other players. Before we talk about the immense demand in media like Hollywood and games.
Heck there's an entire new entertainment industry forming with things like character ai having more than 20M MAU. Midjourney has about the same.
Definitely. An industry in its infancy that already has hundreds of millions of MAU across of it shows that there's zero demand because of some ad no one has seen.
Seems like your reasoning for how the next 2 years will go is a little slanted. And everyone in this thread is neglecting any demand issues stemming from market cycles.
Is anyone looking at this and the CBO figures and not just realising the government is straight lying about the figures?
Gonna believe Powell and Waller on this one.
reply