Save that for your local mom-and-pop store: Microsoft is a multi-billion dollar corporation with enough resources to, at the very least, provide a reliable service for enterprise clients.
The "AI is using a lot of resources" excuse was maybe acceptable last year but not in Q3 2026.
Doubly so given this is hardly a surprise. We've been on this trajectory for at least a couple of years now. They don't get to shrug, point at 10x volume, and act like they've been blindsided.
They had large increase in volume, and trying to move to Azure at the same time. I don't envy them for either work they need to do. But also don't feel pithy because it's Microslop, at the end of the day.
All being said, I have most sympathy towards engineers of all people. They don't get to call the shots, they do as they are told. 99% of the time by an extremely out-of-touch managers.
Managers and executives though? Those I do blame. Surely at this point it should be blindingly obvious their current strategy is not working.
Those engineers also chose to work in US big tech for the ridiculously high salaries that come with it. Having to deal with the problems resulting from that scale is just part of the deal.
Ok but if the same problem will happen with the mom-and-pop store, switching there doesn't help. This isn't a moral dilemma, people just want their stuff to be up.
Yeah let's not forget that in those brief few days when Sam Altman was fired, Satya Nadella swooped in and hired him. Despite the failure of that move, Microsoft has invested $13 billion directly into OpenAI, and has spent $100 billion on the partnership. No sympathy from me watching their software crumble as a result.
> scaling problems due to the influx of LLM assisted coding
You mean people pushing LLM assisted coding, or github's own coders pushing LLM assisted code to production ? :-).
I am more inclined to think that it is the second, but if the reason is the first, what is particular hard to scale ? It's not like everybody is vide-coding into a single repository, so I would expect that most of the load would naturally be scalable by increasing the resources. If for some reason that is not possible, I would expect a corporation as big as Github to keep the paying customers working by gracefully denying service to the free tier, not to collapse like an house of cards.
The quality and performance of GitHub enterprise has degraded horribly this year. Their problems are self-inflicted and influx is just an excuse.
Despite minimal new features, and drastically increasing hardware resources to keep up, self-hosted GitHub enterprise went from things being instant to most requests taking many seconds. And activity on issues in one repository tends to mysteriously re-open issues with the same number in other repositories.
I think they are vibe coding and facing the consequences.
Please let this not be true :( I mean nothing against vibe coding for hobby and assisting you when you get stuck, but at something important like Github they should have hooks to stop low-quality commits and repeated CI-breaking or issue causing should automatically restrict accounts at such companies I think.
Do you think they have such measures in place? I remember when my indention wasn't spotless our university rejected my commits or if my function complexity was too high.. Tears in my eyes.. I think it was called Gerrit Code Review.
It’s become clear to me that any company acquired by Microsoft should be avoided. All software dev is hard. Software dev within a huge organization is damned hard. And Microsoft is SO HUGE and has so much infighting and finger pointing going on that they’re now, as far as I’m concerned, incapable (let alone unwilling) to ship good software.
No faith left. None. They turn everything into a quagmire.
Come on now let’s be fair. They couldn’t possibly fix all of GitHub‘s bugs and deployment problems while maintaining their breakneck pace of mission critical feature deployments. You know, probably something sexy like replacing Win11’s maximize, minimize and close buttons with a chat prompt where you can just tell copilot what you want the window to do.
How many weeks/months can they use this excuse? They are literally are at the forefront of this emerging industry and are capturing untold value. To let their product suffer and potentially lose market share because of it is extremely foolish
Scaling can take time but they've had at least a year to prepare. It's not like any demand increase they've seen was overnight...
At most normal companies you monitor your systems and address potential bottlenecks before they reach a tipping point. And generally you want enough headroom that a sudden 2-3x increase in demand wouldn't take out the service.
Either Github's technical leadership/talent is completely out of their depth (seems unlikely), or they're not hiring enough engineers to work on these capacity issues.
> Either Github's technical leadership/talent is completely out of their depth (seems unlikely), or they're not hiring enough engineers to work on these capacity issues.
Regarding their leadership, I'd argue the latter implies the former.
More likely, as seen in a lot of companies, perhaps infra teams got whittled down (or frozen HC, less than BAU, etc) with resources reallocated to AI org units.
I got the impression from a previous thread that the continual issues were largely driven by the required move to Azure, and underlying Azure stability
Tracks with some of my internal metrics. There are certain projects I'm tracking (solo dev or a small team of a few people) doing 500k LoC changes _per week_. There's one project that caught my attention in particular because they had something like 800k commits and around 7 million lines of code... and they created the repo in _July_. Also, yes I checked, the repo gradually grew over time they didn't just splat it in the first commit.
Isn't all the LLMs and AI supposed to be "solving these problems" because all the engineers are now 10x ? Not being sarcastic but I mean, this is what we were told and sold...by the likes of...Microsoft.
HTTP traffic is different from run this CPU intensive pipeline service for 10 minutes and now increase the volume by 100x in a few months.
I used to manage a popular app/website that had tens of billions of real user page views per month. It wasn't easy but it also wasn't hard. It was surprisingly easy with a single large Postgres DB, a lot of caching, optimized tables and queries.
Yes, but their shapes are very different. Most users interact with a small, geographically constrained bubble. It is (comparably) an easier problem for the usage patterns of Facebook.
The app is also designed so when you open Insta, only a few images are shown, infinite scroll is there for a reason. The search page suggestions? Probably from a cache and then re-ranked quickly.
It is though. Maybe you think they should be doing a better job but 12-24xing scale when you were already handling billions of commits is definitely understandable to me. It just does feel like they should have some way of ensuring customers, paying ones, don’t have to suffer so much.
Yes. It seems that at the very least repositories owned by paying customers should be processed on priority threads. And perhaps throttle commits/pushes to repos where there are a lot of pushes from the same person all the time. This can be effectively throttled at the push level.
So in other words, a 12x increase - which is exactly that I already responded to.
A 12x increase in volume should not be world ending for Microsoft given that the number of developers in the world is a pretty small baseline number to start from.
That’s the reason they gave, and I believe them. But I also think there has to be more to it. If your paying users are seeing degradation caused by (likely) free tier users (for years by the way), restrict the free users.
They've had an insane influx. It seems that they should more aggressively limit free tier activity or isolate paid users, this has become unbearable...
Github has had a pretty spotty track record for uptime for a very, very long time. It didn't get any better when MS bought it out.
It has really just coasted on GH's network effect because everybody uses GH for git by default. It doesn't really have much direct competition and anything that does offer something like GH just doesn't have the same mindshare.
I'm really curious how other codeforges are faring. I only use codeberg and github - codeberg seems to have less outages, especially for a much smaller organisation, but they've recently removed support for LLM projects which could be linked.
Anyone here have data on whether the likes of gitlab have been impacted in the same was as github?
I love codeberg but it also have outages and is sometimes quite slow.
But I’m more lenient with codeberg because they are a non-profit and as you say, they are banning vibe-coded project because of the cost to infrastructure. In the meantime, GitHub is encouraging vibe-coding, multiplication of the source code, and genAI discussions.
Yes. And the same for the database and then you get it into “professional” territories with more moving parts that can also fail and take 3 hours to figure it out and recover.
Yup. It's kinda binary: either almost no downtime for years with a crazy number of nines, or a total mess for several days while people figure out the issues.
That being said, a Git forge for a small/mid-sized company is not something that requires rocket surgery. Forgejo with an SQLite database and a daily backup to S3 would be more than enough for the majority of users.
No, that's almost exactly right, except that scaling problem is inverse!
MSFT is giving away most of their Azure Datacenters resources to AI, to make money and doesn't care if Github suffers along the way and drains Github's available resources to bare minimum. Leading to outages like this, but you're 100% spot on right too. All those Claude/OpenClaw instances produce so much Code, people setup more and more repositories.
Well, it has been 8 years since the acquisition and they have had access to Azure infrastructure to keep up with the scale and yet, it is still struggling due to free compute thanks to Actions.
They need to start getting rid of the free tiers for GitHub Actions and actually charge appropriately or increase prices for it.
If they did, they would’ve paused new feature development a long time ago and redirected that resource towards scaling the platform. And yet they’ve been releasing tonedeaf features more than ever.
Many on the outside might say the US is very pro-self and anti-everyone-else right now! America first, right?
Well, take a random country. We'll make a made-up one called Tristonia. Should Tristonia do things in Tristonia's best interests? Absolutely! If Tristonia had some kind of military or economic threat from another country, should they take steps to protect their country, resources, and people from the threat? Absolutely! The same thing the US would do.
Most of these people aren't necessarily anti-US, they're against being beholden to another country, especially one that has been incredibly... inconsistent, as of late.
I'm a US citizen and I recommend people avoid most US corporations all the time. Helping US corporations does not help the blight of everyday Americans, all you do is make like a few thousand billionaires more wealthy.
I also find the excuses for these trillion dollar companies pathetic. These are entirely human constructed systems, if they can't understand what they made and how to improve it then maybe we need to question the way they write software in general if it's this unsustainable.
I hardly doubt some other team with better leadership qualities couldn't improve things to be better than they are now.
They've done the typical sillicon valley sillyness of having 10 000 engineers over-engineer a product so that there's way too many features built on top of what should be a simple tool and are now paying the price. Literally every larger US company makes this mistake.
But maybe I’m wrong?