Yes. The CPU and GPU demand has nothing to do with it. The reason is the car industry.
For some reason in early 2020 all the car industry execs were convinced that people would buy dramatically fewer cars in 2020, due to pandemic crashing demand. Because they have a religious aversion to holding any stock they decided to shift the risk over to their suppliers, fucking said suppliers over, as the car industry normally does when they expect demand shifts. The thing that made this particular time special as opposed to business as usual is that the car execs all got it wrong, because people bought way more cars due to pandemic rather than less, due to moving out of cities and avoiding public transit. So they fucked over their suppliers a second time by demanding all those orders back.
Now, suppose you're a supplier of some sort of motor driver or power conversion chip (PMIC) in early 2020. You run 200 wafers per month through a fab running some early 2000s process. Half your yearly revenue is a customized part for a particular auto vendor. That vendor calls you up and tells you that they will not be paying you for any parts this year, and you can figure out what to do with them. You can't afford to run your production at half the revenue, so you're screwed. You call up your fab and ask if you can get out of that contract and pay a penalty for doing so, and you reduce your fab order to 100 wafers per month, so you can at least serve your other customers. The fab is annoyed but they put out an announcement that a slot is free, and another vendor making a PMIC for computer motherboards buys it, because they can use the extra capacity and expect increased demand for computers. So far so normal. One vendor screwed, but they'll manage, one fab slightly annoyed that they had to reduce throughput a tiny bit while they find a new buyer.
Then a few months later the car manufacturer calls you again and asks for their orders back, and more on top. You tell them to fuck off, because you can no longer manufacture it this year. They tell you they will pay literally anything because their production lines can't run without it because (for religious reasons) they have zero inventory buffers. So what do you do? You call up your fab and they say they can't help you, that slot is already gone. So you ask them to change which mask they use for the wafers you already have reserved, and instead of making your usual non-automotive products, you only make the customized chip for the automotive market. And then, because they screwed you over so badly, and you already lost lots of money and had to lay off staff due to the carmaker, you charge them 6x to 8x the price. All your other customers are now screwed, but you still come out barely ahead. Now, of course the customer not only asked for their old orders back, but more. So you call up all the other customers of the fab you use and ask them if they're willing to trade their fab slots for money. Some do, causing a shortage of whatever they make as well. Repeat this same story for literally every chipmaker that makes anything used by a car. This was the situation in January 2021. Then, several major fabs were destroyed (several in Texas, when the big freeze killed the air pumps keeping the cleanrooms sterile, and the water pipes in the walls of the buildings burst and contaminated other facilities, and one in Japan due to a fire) making the already bad problem worse. So there are several mechanisms that make part availability poor here:
1. The part you want is used in cars. Car manufacturers have locked in the following year or so of production, and "any amount extra you can make in that time" for a multiple of the normal price. Either you can't get the parts at all or you'll be paying a massive premium.
2. The part you want is not used in cars, but is made by someone who makes other parts on the same process that are used in cars. Your part has been deprioritized and will not be manufactured for months. Meanwhile stock runs out and those who hold any stock massively raise prices.
3. The part you want is not used in cars, and the manufacturer doesn't supply the car industry, but uses a process used by someone who does. Car IC suppliers have bought out their fab slots, so the part will not be manufactured for months.
4. The part you want is not used in cars, and doesn't share a process with parts that are. However, it's on the BOM of a popular product that uses such parts, and the manufacturer has seen what the market looks like and is stocking up for months ahead. Distributor inventory is therefore zero and new stock gets snapped up as soon as it shows up because a single missing part means you can't produce your product.
So here we are. Shameless plug - email me if you are screwed by this and need help getting your product re-engineered to the new reality. There's a handful of manufacturers, usually obscure companies in mainland China that only really sell to the internal market, that are much less affected. Some have drop-in replacement parts for things that are out of stock, others have functionally similar parts that can be used with minor design adaptation. I've been doing that kind of redesign work for customers this whole year. Don't email me if you work in/for the car industry. You guys poisoned the well for all of us so deal with it yourselves.
My dad was in manufacturing. He always told the car companies to fuck off because of this. If you start doing business with a car company, the bullying starts right away. You eventually are forced to choose between them and your other customers. It’s probably best to just never start down that road if you can avoid it.
My company refuses to work with automotive primarily because of the billing. They require you to front the money for all R&D, and tooling, and then they literally pay whenever they want for actual delivery (they completely ignore invoice terms such as Net 30 days - and will pay at 60, 90, or 111.8843 days)
They’re not the only ones to do that, but yeah, they have a rep. I’ve heard of FIAT (now Stellantis) routinely paying suppliers 3 to 6 months after a contract is fulfilled. Absolutely ludicrous.
It's actually worse for us since we are capitol equipment suppliers. It could literally be 3-4 years before we get paid since they won't pay until the equipment has a proven production rate/quality/other random metric. They will find just about any excuse to not sign off the IQ/OQ/PQ and if there are any delays in other equipment, or materials, or whatever then they don't pay.
This sounds a lot like how big distribution chains work around here... Maybe it's a thing with economies-of-scale-based businesses. Like 'we'll be moving MILLIONS of your gadget but you'll have to sell your and your employees' and providers' souls. And self-respect.
If they know they can get away with their suppliers having to front all the cash, why not do it? If all their competitiors get away with it, they'd be fools not to as that is a lot of expense and interest they're eating and they'll need to make up for that somewhere.
Yes. And still not everyone wants to work for them. I'd sure never ever ever try to sell them anything... I'm thinking with such reputations they get shafted out of good opportunities and this has a cost. I'm not sure how Costco treats their partners & providers but I'd guess they're still shrewd without bullying and sell better quality. I know the business model is different but clearly there's room and money to win in other ways.
Very interesting breakdown. It sounds to me like the core of the problem is that everyone needs fab slots to build their things, but fab slots are a limited resource and car manufacturers have bid up their price to levels few others are willing to match. Does that seem about right?
No, that's wrong. Fab slots are not a limited resource (except at the very high end). They just have a lead time of 12-18 months and it takes a lot of time and a lot of lost production to change what they produce any faster than that.
The way your original comment phrased it I assumed phab slots are not actually locked in 12 months in advance, and if you pay someone enough they might agree to swap the masks they were going to use for yours. That would mean fab slots are acting like a scarce resource in the short term (3--12 months?). Is that also not right?
I'm not saying the system isn't broken and/or that car manufacturers aren't jerks for monopolizing a scarce resource and putting other demands in tight spots. I'm just reading it as regular free market capitalism level broken, but it sounds like you're saying it's even more broken than that. This is a topic very relevant to my current interests so I truly do wish to understand it.
There is a cost to switching, and a time delay. If you switch faster than the optimal interval, you lose some of the stock that was in-process. Because of this, a large industry changing their mind back and forth rapidly with a particular alignment to that interval can cause disproportional disruption. This is happening here. They changed their mind on short notice, causing one wave of disruption as everyone adapted, then they changed their mind on short notice again in the opposite direction, causing a second, even bigger wave of disruption on top of that. Suppose you were on a plane from London to Dubai, and someone tells you to get to New York as fast as possible. You can't turn the plane around, so you wait until you're in Dubai and take the next plane in the opposite direction to New York. But halfway there you get a phone call saying you need to be in Singapore instead. Again, you have to wait until you land before you can do anything about it, and your entire current trip is wasted. Getting from where you were originally headed (Dubai) to your ultimate destination (Singapore) is a short trip, but by timing the instructions at especially inconvenient moments, you end up having to add enormous cost and resource waste to reach the same target. The car industry is definitely paying for their fuckup at the moment, but everyone else who uses electronics is stuck on that plane with them going the wrong way.
Ah, okay, that makes sense. Do I get it right that there are then essentially these two additional problems going on here, beyond short-term slots being a scarce resource:
- When manufacturers change their minds quickly and pay the price to buy up a lot of slots last minute, they are effectively also "burning" some slots that will simply no longer add to worldwide production of anything? and
- Being such a big customer with a high variance in demand, the car industry have are making it nearly impossible for the rest of the market to make rational decisions based on long-term forecasts?
(It might sound like I'm stating the obvious here but I know how often I think I understand something but later turn out to have missed the mark completely anyway.)
here's the wafer agreement that i found between TSMC and Altera. there's probably more to this. i think auto maker's chip suppliers didn't sign the wafer agreement with fabs.
" 5.12 ADJUSTMENT IN CAPACITY RIGHTS AND OBLIGATIONS. Unless otherwise
agreed in writing by the Parties, any adjustment in the Buyers' Percentage
Interests which results in a change in their Adjusted Percentage Interests (as
defined in Subsection 1.1.3), shall not become effective for purposes of the
Buyers' Basic Purchase Shares until the thirteenth (13th) week after the date
upon which the adjustment in Percentage Interests occurs pursuant to the LLC
Agreement, and unless agreed otherwise by TSMC and each Buyer under a Purchase
Order, no such adjustment in Percentage Interests, when so effective, shall
operate to cancel, amend or otherwise affect any Purchase Orders accepted by
TSMC prior to the date that such adjustment in Percentage Interests is so
effective."
I think what you write maybe is what has broadly been going on (across all industries - not just cars). Sudden drop in demand followed by a sudden surge in demand. Plus disruption is suppliers ability to deliver (due to lockdowns and border closures).
It is somewhat the same story as being described here - which is an article about a company that makes display drivers:
I hate cars for so many reason already, and here you are telling me that they are literally squeezing global production of chips too. Cherry on top.
I really want us to stop being so obsessed with cars, because no matter how useful they are to transportation, in my mind they just can't be worth the pain and damage they cause to our society...
I also hate cars for so many reasons but surely "their manufacturers run an efficient operation and can outbid competitors for a scarce resource" isn't one of them?
That's not the case here - the case is that they are big and powerful enough to bully suppliers so that suppliers assume a lot of the risks that any normal-sized company with fewer political connections would have to assume itself.
From your original comment I gathered that they "will pay literally anything" for you to swap the mask in your remaining 100 slots. That sounds to me like regular free market-enabled bullying.
Are you saying it's a different kind of bullying? What makes it different? You original comment really piqued my desire to understand this.
Customized car parts are often a major part of a subvendor's revenue, but they are stuck in a monopsony situation - nobody but audi will buy an audi rear seat height adjustment lever. The market power of large automakers mean that they can very quickly become the primary revenue source of a small vendor, and there's no collective action that various vendors can take against the automaker. The automaker can then get away with conditions like having the vendor pay massive penalties for not delivering a number of parts within a 15 minute time window, changing delivery volumes on short notice, or payment terms where they pay for parts months and sometimes years after they are delivered, and the vendor has to sit on their costs in the meantime. The vendor can't afford to lose the automaker as a customer, but they also can't afford to have enough margin to swallow a back-and-forth variation in demand without adjusting production volumes. In this case, the automakers ended up in a tight spot because they had fucked over all their vendors in this way simultaneously, and so ended up in a situation where their production lines were literally standing still because the vendors could not deliver as they had just finished adjusting to the previous change. This is when the automakers shifted to offering way-above-market money for product, because the alternative was massive losses from stuck production lines.
Oh. You're saying they normally wouldn't pay the cost of the mask change, only say to their vendors "do it and absorb the cost or we'll pull our business?"
What a great comment. Does anyone know if Tesla went the old automaker way or do they stockpile chips and/or had better forecasting on demand during pandemic? It seems their production is doing great.
Selfish reason, I want to know if my Tesla order will be delayed.
I'm not 100% certain, their suppliers could concievably have been affected by the same problems everyone else has. But my impression from following Tesla's quarterly reports before and during the pandemic is distinctly that they just kept plowing on, business as usual. No changes in production or their investments for production capacity, apart from production delays due to government-mandated shutdowns.
Hard to underestimate the impact of Musk's leadership when you see that result.
I agree it's sad to see things disrupted/destroyed, but it certainly lit a fire under a ton of people who have the power to make dramatic shifts. I think this will mark the beginning of a new era in semiconductor abundance and innovation.
My dramatic shift - never work on a project that helps the car industry. They're now on the same shitlist as weapons, coal, and surveillance capitalism.
It seems overly naive to assume any company's actions don't have pros and cons, or aren't part of the larger society/ecosystem you're enabling by existing? You can't take an action (or exist and take no action) without a side effect, and at the scale of any significant company, it is impossible to do what someone somewhere considers to be harm to someone or something.
Picking and choosing the scope, scale, and amount of the type of harm you're comfortable with certainly seems worthwhile, but pretending there is any option that is harm free seems like self deception. Even refusing to play (not living?) has side effects most would consider harmful - at least to those around you.
That is a good point. Perhaps a different statement makes more sense: it's hard to believe that there are no options where the benefits outweigh the harm.
In my experience, the benefits do outweigh the harm in pretty much all cases where it lives awhile - if you pick the right group of people to care about, and the right things or people to be ok harming.
We all do it, directly or indirectly. It’s necessary to survive. It’s common for instance to not worry too much about the earthworms dying on the sidewalk after the sprinklers run. Most people would consider someone pretty weird if they didn’t drive because of the body count of insects on their windshield.
For example - everyone who drives is implicitly or explicitly saying the benefit to them outweighs the harm to those insects. Everything has a trade off.
Most life survives by eating other life (yes even Vegans), and even pure photosynthetic algae has to kill or crowd out (and starve) competitors and things that would kill and eat it, or they would no longer be alive.
It’s easy to be isolated from this reality, especially with modern life meaning we don’t need to gather, raise and/or kill our own food.
Nature is not (just) a national park with carefully curated trails and animals to snap pictures of when they grace us with their presence. It’s wild fight for survival and dominance - full of growth, birth, gore, beauty, and death.
Recognizing that someone’s trade off to make another country less strong (a ‘other’) in exchange for making themselves richer or their country more capable of doing the same in the future is important (one example) - say a defense contractor making a new weapon.
It is a specific mindset and trade off that we might not agree with it even find repugnant, but it is keyed off important survival needs and a fundamental part of the world and nature we are foolish to ignore.
What do you consider ok to harm, to get the benefits for the people or things you care about? At some point it might be worth making it concrete, it can be illuminating.
My point was not about nature though, rather it was about being sustainable. Looking at the classic prisoner's dilemma, there are activities which bring localized benefit, but an overall loss in a non-zero-sum game.
Presuming that human activity is not zero-sum regarding whatever system of values we take, it's clear cut that some activities are bad, period.
This makes the grandparent's examples clearer: coal production benefits some people now but hurts future generations disproportionately. Wars hurt all sides, and allow some of them partially recover, and never as far as if they joined forces. Car companies throw the whole semiconductor industry under the bus in exchange for not having to keep inventory.
In a huge simplification, it's self-destruction I'm talking about, not sacrificing little things to gain greater ones, like exchanging the lives of insects to feed a human cvilization (presuming you value humans more).
Of course, it's possible to construct an arbitrary system of values to justify your own arbitrary deeds as "survival" or "natural", but I don't find such systems convincing.
Nature is the only known example we have of a sustained environment, no? Everything else is artificially constructed, and no social or artificial environment I know of has lasted in any meaningfully sustained and stable way for more than a few generations.
My point is that you are constructing your own arbitrary system relying on projections of the results of actions far into the future, and impacts that are purely hypothetical at this point - they might be true/accurate, or they might not. Someone is staking their own success or happiness based on if they think it’s true or not, and I can point you to many examples in the recent past where that was a severe evolutionary negative for many people.
Prisoners dilemma is a thought experiment/theoretical problem because in the real world, what people value, their own understanding of and perceived trustworthiness of others, and their own dispositions and habits means it might as well be talking about spherical cows.
If we get into a shooting war with China next year for instance, global warming is going to be pretty low on the problematic outcomes one could face. If a pile of coal (or billion barrels of oil) is what it takes for the US (or China, or any other nation) to survive, they will burn it without a moments hesitation.
We can all hand wring about if that hurts the global balance, and how it’s terrible, and game theory and all that - but if it’s a fight for dominance, survival, and resources it won’t matter. So far we’ve found enough common ground/everyone is getting rich enough helping each other out it hasn’t come to that. We can hope it never will.
Pretending that folks who are building tools, or capabilities, or working in the areas that would be necessary to survive if something like that happened seems willfully ignorant of the reality we’re in or incredibly naive, IMO.
Some folks make short sighted choices, lose a ton of money, screw people over (intentionally or not), etc. In my experience it’s usually due to environmental factors (regulation, market, competitive forces) strongly incentivizing it, or just plain not being able to predict the future.
That chaos and dumbness is not only inevitable, it happens everywhere. It’s fundamental to the human experience. It’s easy to be high and mighty, much harder to do it differently - and survive.
> Nature is the only known example we have of a sustained environment, no?
No, not really. Looking at the large scale, the Earth will burn up together with the Sun, and then we'll reach the death of the universe one way or the other.
Small scale, we have sustained businesses, economies, families, ecosystems. In those, prisoner's dilemma is real and testable.
I'm getting the impression that you're trying to draw some equivalency between all classes of harms, regardless of their magnitude. I do agree to a certain extent, but I can nary see an argument that would convince me that some options are just not singularly harmful.
As an example, me and my neighbour being paid to grow a field, at the cost of the field insects, versus me and my neighbour being paid to drive each other away and take over the land. One is clearly better than another if human well being is more important than that of insects.
If we fear the neighbour fighting us and we value survival, we must fight ourselves. That's the curse of the prisoner's dilemma, because giving up the fight yields a better outcome. I would call such a greedy "survival" approach naive and unfortunate.
If we value dominance above all, then fighting the neighbour is the better choice. I believe we have a word for such value systems: egoism.
Giving up the fight yields the better outcome only when the other party doesn’t come over
and eat you or kill you no?
It isn’t about dominance over all - it’s about survival and the complex intertwining of all the various strategies (good, bad, failing) being tried simultaneously with the hope that theirs wins. Sometimes it’s domination. Sometimes it’s teamwork. Sometimes it’s passivity.
As any individual player, you have to pick one - and sometimes it wins, sometimes it loses. Most strategies exist not because they are always ideal, but because having the selection is essential and necessary to long term survival in a fundamentally unpredictable environment. A species that only does one strategy at once is going to go extinct when the environment changes suddenly.
I’m definitely not equating all actions as equal, or saying everything has equal cost. I’m noting that nothing, including no action, has no cost, and that assuming a player in the game (you or someone else) will definitely lose or definitely win for a given survival criteria has a very low degree of predictive accuracy in the real world.
Of course, you're right about survival. Sometimes the rational strategy is to fight. What I'm contesting is the view that predicting outcomes is useless.
I gave the example of economies and ecosystems together with prisoner's dilemma to illustrate that we do indeed have a models that are accurate in predicting at least first order effects.
First order effects are not everything, of course, but it doesn't seem sensible to throw away our knowledge coming from game theory (or cybernetics? or plain life experience) and settle on not being able to predict which choices are net positive and which are net negative.
As with any estimation, there will be those that can't be evaluated with any certainty, but also clear winners and losers.
I work in this industry, I talk to vendors, I see this stuff daily. If you want the manufacturing-side background, nikkei's semiconductor news has lots of material on this ( https://asia.nikkei.com/Business/Tech/Semiconductors/ ) including the fabrication disasters that ruined NXP and Renesas. Go through the last few months to see it unfolding. For the other stuff, I can't give you public evidence, but I've been working with customers adapting their designs, and they've had confirmed orders from manufacturers delayed or cancelled, some citing prioritization of the car industry, others not giving any reason, but then auctioning their remaining stock out, with the final bids going to automakers at 8x the 2019 price. Microchip, a major vendor, introduced a program where they will provide prioritized product a year from now in exchange for non-cancelable non-refundable month-by-month demand visibility into the future ( https://www.microchip.com/content/dam/mchp/documents/announc... ). I don't know what sort of link you need, this is literally what I do every day.
For some reason in early 2020 all the car industry execs were convinced that people would buy dramatically fewer cars in 2020, due to pandemic crashing demand. Because they have a religious aversion to holding any stock they decided to shift the risk over to their suppliers, fucking said suppliers over, as the car industry normally does when they expect demand shifts. The thing that made this particular time special as opposed to business as usual is that the car execs all got it wrong, because people bought way more cars due to pandemic rather than less, due to moving out of cities and avoiding public transit. So they fucked over their suppliers a second time by demanding all those orders back.
Now, suppose you're a supplier of some sort of motor driver or power conversion chip (PMIC) in early 2020. You run 200 wafers per month through a fab running some early 2000s process. Half your yearly revenue is a customized part for a particular auto vendor. That vendor calls you up and tells you that they will not be paying you for any parts this year, and you can figure out what to do with them. You can't afford to run your production at half the revenue, so you're screwed. You call up your fab and ask if you can get out of that contract and pay a penalty for doing so, and you reduce your fab order to 100 wafers per month, so you can at least serve your other customers. The fab is annoyed but they put out an announcement that a slot is free, and another vendor making a PMIC for computer motherboards buys it, because they can use the extra capacity and expect increased demand for computers. So far so normal. One vendor screwed, but they'll manage, one fab slightly annoyed that they had to reduce throughput a tiny bit while they find a new buyer.
Then a few months later the car manufacturer calls you again and asks for their orders back, and more on top. You tell them to fuck off, because you can no longer manufacture it this year. They tell you they will pay literally anything because their production lines can't run without it because (for religious reasons) they have zero inventory buffers. So what do you do? You call up your fab and they say they can't help you, that slot is already gone. So you ask them to change which mask they use for the wafers you already have reserved, and instead of making your usual non-automotive products, you only make the customized chip for the automotive market. And then, because they screwed you over so badly, and you already lost lots of money and had to lay off staff due to the carmaker, you charge them 6x to 8x the price. All your other customers are now screwed, but you still come out barely ahead. Now, of course the customer not only asked for their old orders back, but more. So you call up all the other customers of the fab you use and ask them if they're willing to trade their fab slots for money. Some do, causing a shortage of whatever they make as well. Repeat this same story for literally every chipmaker that makes anything used by a car. This was the situation in January 2021. Then, several major fabs were destroyed (several in Texas, when the big freeze killed the air pumps keeping the cleanrooms sterile, and the water pipes in the walls of the buildings burst and contaminated other facilities, and one in Japan due to a fire) making the already bad problem worse. So there are several mechanisms that make part availability poor here:
1. The part you want is used in cars. Car manufacturers have locked in the following year or so of production, and "any amount extra you can make in that time" for a multiple of the normal price. Either you can't get the parts at all or you'll be paying a massive premium.
2. The part you want is not used in cars, but is made by someone who makes other parts on the same process that are used in cars. Your part has been deprioritized and will not be manufactured for months. Meanwhile stock runs out and those who hold any stock massively raise prices.
3. The part you want is not used in cars, and the manufacturer doesn't supply the car industry, but uses a process used by someone who does. Car IC suppliers have bought out their fab slots, so the part will not be manufactured for months.
4. The part you want is not used in cars, and doesn't share a process with parts that are. However, it's on the BOM of a popular product that uses such parts, and the manufacturer has seen what the market looks like and is stocking up for months ahead. Distributor inventory is therefore zero and new stock gets snapped up as soon as it shows up because a single missing part means you can't produce your product.
So here we are. Shameless plug - email me if you are screwed by this and need help getting your product re-engineered to the new reality. There's a handful of manufacturers, usually obscure companies in mainland China that only really sell to the internal market, that are much less affected. Some have drop-in replacement parts for things that are out of stock, others have functionally similar parts that can be used with minor design adaptation. I've been doing that kind of redesign work for customers this whole year. Don't email me if you work in/for the car industry. You guys poisoned the well for all of us so deal with it yourselves.