One thing people often don't realize or ignore: these LLMs are trained on the internet, the entire internet.
There's a shit-ton of bad and inefficient code on the internet. Lots of it. And it was used to train these LLMs as much as the good code.
In other words, the LLMs are great if you're OK with mediocrity at best. Mediocrity is occasionally good enough, but it can spell death for a company when key parts of it are mediocre.
I'm afraid a lot of the executives who fantasize about replacing humans with AI are going to have to learn this the hard way.
Maybe it's things like 4-year tenure, or shorter tenure, or something else.
But I think it's a matter of motivation, Bob.
> The thing is, Bob, it's not that I'm lazy, it's that I just don't care. It's a problem of motivation, all right? Now if I work my ass off and Initech ships a few extra units, I don't see another dime, so where's the motivation? ... my only real motivation is not to be hassled. That, and the fear of losing my job. But you know, Bob, that will only make someone work just hard enough not to get fired.
No, big tech engineers are highly motivated. There's lots of money, good management, and plenty of incentive. (I'm a Google engineer myself).
The problem I observe is a fairly universal one: management doesn't care about good code, it cares about results.
It's generally hard for anyone without specific experience with a codebase to tell what you're doing with it. Management can't evaluate the value of maintenance work, so it doesn't value it at all.
> The problem I observe is the universal one: management doesn't care about good code, it cares about results. It's generally too hard for ANYONE to tell what's going on in a codebase unless you're experienced with it. Management can't evaluate the value of maintenance work, so it doesn't value it.
I think this is a very telling statement, but perhaps not in the way you intended. I would agree that management only cares about results, but I would posit that maybe that's a good thing. If you don't have ground-truth knowledge of a problem, you must rely on either the word of someone who does, or metrics that can be used as a yardstick.
When all a manager has to go on is someone's word, it can be really hard for them to gauge the depth, severity, and impact of the problem being expressed to them— and without any metrics, they have no way of tracking progress on resolution. In a modern codebase, you could spend YEARS on improving maintainability and still not "finish". The key (that I've found, personally) in this situation is to give the manager some form of metric to describe the problem. If you can establish a number to measure what you're advocating for, and quantify the consequences of not doing it into actual business impact, I've talked managers into taking my suggestion more often than not.
> in this situation is to give the manager some form of metric to describe the problem
how do you have a metric to measure a future issue that got prevented by having good maintenance?
Either you cannot actually imagine nor describe the problem that was prevented, or you could just make something up which cannot be disproved nor falsifiable. So if you asked for time/resources/budget to do maintenance, you cannot then give proof that this maintenance was useful!
The only way to get a metric is to have an incident or have issues crop up, and then in the retrospective, claim that certain maintenance work could've prevented it.
It's because the only people who can understand the code are people who write the code every day. And sometimes the devs don't even really understand it.
Occasionally you'll have a manager who manages in the day and codes at night, which is fine if not bad for his personal life. But HIS manager almost never does that. And his manager's manager? Forget it.
To be fair to management, it is the results that matter.
Management cares about what their management cares about. So this boils down to what the CEO cares about. The CEO cares about what the board cares about. The board cares about share prices going up.
I do believe that e.g. retaining engineers is something that helps the business. It's stupid that someone ramps up for 2 years and then goes to a different job just as they start becoming really effective. It costs companies a ton of money which they could instead just use to get people to stay. But I'm not on Google's board. It's only when Google board decides that fostering the right engineering culture is important enough for the business that something is going to change. And so far- they don't (s/Google/BigTech).
Re: Incentives- Obviously(?) the incentives are not right. So when you say "plenty of incentive" what it really means is incentive to ship sloppy code and get promoted. Or the incentive to go from Google to OpenAI and get a pay increase or whatnot.
> To be fair to management, it is the results that matter.
To be fair to me, I don't recall ever signing a contract through which I am directly responsible for the company's financial and customer-acquisition / retention efforts. I sign up as an individual contributor who helps advance the customer & product mission forward. I am NOT a cofounder.
So that shows, yet again, how myopic and egocentric managers are. Wise ones -- the all 2-3 I have met throughout a 20+ years of experience -- understand that they must enable you to produce the outcomes they care about.
Unsurprisingly, I worked fantastically well with those managers and we achieved near-miracles in some measly 4-5 months.
But all others? "I never gave you time to optimise cloud spend but now I am angry at you for not doing it in your sleep", more or less. Or "I pushed you to the brink of 12-hour workday regularly and started reaching into your weekends and you rushed that feature I pressured you for and it has one small performance regressions? You are fired!". Deal with it.
/rant.
Not directed at you, obviously. Got triggered a little.
We need to disconnect the question of bad managers from the structural issues though. I try to be a good manager but I still have people jump ship for better comp and where we won't match it. I still need to deal with an incentive structure that doesn't match what I am trying to do with my team.
This same structure is also what helps bad managers. Who is going to get promoted to a director role? The person who stands up for their team and argues with the VP or the person who toes the line? The things that you think are near-miracles are not visible and the people that play politics will make their stuff look more valuable to the company.
Yours and my experience are not mutually exclusive, I think we both see it. I dream of managers like you but I never get hired under them for some reason. (Likely regional culture, experience shows.)
I'm at a stage of life and career where I'd happily take a small pay cut for a year just to establish myself in a place and have stability. Then we'll talk about competitive compensation.
My chief issues are with people that are best described with the proverb "give them an inch and they will take a mile".
Yours seems to be that you deal with people that constantly think that they can do better in terms of how much they take home (let's not sugar-coat it, they're spoiled -- I was too).
Heroics being invisible and people who have their coffee with leadership getting the money and the influence is the wrong system. Always was and apparently always will be.
It's interesting that you talk that there's a lot of good management, followed by a list of things I'd call bad management: Forget about good vs bad code, of course that's unimportant. But insufficient maintenance that causes lower velocity on the next set of changes matters. Good engineering will tell you when the issue isn't just about code that is ugly, but code that slows changes down so much you'd be better off improving it.
If you set incentives that say that being sloppy and leaving landmines for the next group of people is the way to get promoted, guess what? the management is bad. Often because they are also looking at their own self interest, and expect to leave the consequences to whoever comes after them. This isn't new to big tech: You'll find this all described in books about corporate dysfunction written in the 90s.
It's all traditional principal agent problems, which just get worse and worse as you add layers of management, as the principal has agents upon agents underneath, all using the misaligned incentives. One either wants t avoid getting fired while doing the minimum, or sacrifice the health of what is around them for a good enough promotion packet/review. And since there's no reasonable way for individual objectives to align well with long term objectives, people leave landmines. When there's enough landmines everywhere, you are always better off in greenfield development. And at that point, doing any maintenance, or being stuck in a project that isn't getting fed a bunch of capital to grow it is career suicide. All about bad incentives, set by bad management.
> The problem I observe is a fairly universal one: management doesn't care about good code, it cares about results.
The thing is, good code is a form of a good result. You need to solve the underlying problems (which manifest as impact) but if you used code to get there, that code if well designed is extensible, reusable, then you pay low maintenance on it and that same code can be used to solve other problems (ideally).
It's a difficult judgement call to make though. If your org doesn't have the right technical leadership and the performance management structure doesn't reward it then you get what you see.
It's a lesson that's always learned far too late when it becomes slow and costly to deliver something new because you've amassed so much tech debt so it's often cheaper to start from scratch (which is saying something).
I see this all fwiw in big tech myself and with my peers at peer companies.
> It's a lesson that's always learned far too late when it becomes slow and costly to deliver something new because you've amassed so much tech debt
No, it is just standard operating procedure: deprecate a working system and write a new system from scratch, with 50% of features not supported. This side-steps the tech debt and gives everybody artifacts for promotion. It screws all users of the system but who cares about them!
The thing is shipping sloppy code is orders of magnitudes easier, because that's the default result. Any sufficiently determined hack-job can do it. On the other hand, steering a team of 5-10 engineers to deliver quality on (or before!) a deadline requires excessive amounts of coordination and skill. Now, is this trade-off "worth" the effort? I guess that's a matter of opinion, though I would argue in the long term quality wins out by a large margin.
Sometimes teams will do the sloppy thing because it's the quickest path to resolution of an issue deemed to be critical, which can make sense in the moment, but if you do this too many times without spending the effort to improve the solution after the immediate pressure is gone, issues like this will continue to get more common. I like to use an analogy of trying to put out a fire on the other side of a building by filling a cup in the sink and then running across to toss the water onto the fire; in the short term, it might be the best fix, but if you keep finding yourself having to do this, you're probably better off installing a fire hose even if it requires a lot more effort up front.
"management doesn't care about good code, it cares about results."
And management is probably right: code is a mean, not an end
It is us, as software engineers, to make that better code brings better results
Some benefits of better code can be:
- less infrastructure cost
- more speed / efficiency / whatever metrics available from the users
- easier to integrate new features
- less issues, less time spent in maintenance etc
At my current job, we have some instance of people who animate around bad practices: they scan everything, identifies some "bad practice" (which can be whatever) and then raise the issue to all teams and give them a month to fix the issues
Very political stuff
I try to do good code. My team is rarely cited in that list of "bad players". Management does notice.
Corollary: our roadmap is not messed up. Product managements notices.
Also, good code must mean robust infrastructure. We have very few incident. Almost all our changes are successful. Incident and change managers notice.
Anyway, my point is : if good code brings no benefits to the outside world .. how is it good, exactly ?
Yup, tolerating inappropriate tech debt for too long leads to bad results. I see this going wrong all the time and I thought it was mostly due to being surrounded by bootcamp grads given nothing but year after year of pressure and no examples of what good code can look like. Surprised to hear the same thing happens in big tech.
There is also a lot of money, there is also good management, and there are also lots of incentives.
But management depends on your manager; at scale it becomes likely there are bad apples in every management tree. Incentives may not align with what you want or need, with work From Home policies getting shrunk. Even money sometimes is a point of contention.
That's just the thing though: I want to know what those other incentives are and I was never told anything else than a hand-wavy "money" blurb with zero elaboration.
I mean OK, technically the ad viewing experience in f.ex. iOS is terrible; you sit through 30 seconds, then a white button on an almost-white background appears (dark pattern, they want you to sit looking at the ad longer), then when you "dismiss" is, an AppStore pop-under shows up and you have to dismiss that as well, and ONLY THEN you get another screen where you have to wait 5-10 seconds until the blessed micro X button finally appears.
This can be made much better: the ad platform might enforce the top-left corner be always black and the X button to appear only once and be effective immediately, for example. No shenanigans with bluffing that you are now leaving the ad but haha, you have fallen into our trap! Here's our AppStore page!
But why should Apple care? The money is literally pouring in! And humans operate on fight-or-flight responses much more than what 99% of them would be willing to admit; an Apple executive can drown you in executive jargon but the naked truth would still be "We don't want to change anything that might slow down our income". Or even more bare: "Don't touch it if it works and makes money".
So yeah, that's one of the examples where the hand-wavy "money" blurb would make total sense; I get it.
But in all my career I was never told in clear certain terms -- and they must also make sense -- about why not investing just a measly two hours more on technical excellence is so extremely unwelcome. Of course they always cite velocity and customer retention and how we must make sure we don't miss a potential client but I've never seen even one little piece of evidence of customers churning because a feature was delivered on Wednesday and not Tuesday. I am sure it happens, mind you, but I could never shake the feeling that those dangers are always hugely exaggerated.
Was McNamara really thinking that's how you win a war and people just went along with it or was it just another excuse in a long line of excuses to keep the gravy train going?
I used to be an extremely motivated engineer. I cared about the code that I wrote, the other people on my team, making sure things were documented and understandable. I tried to write good code where I could, and detailed PRs and issue writeups where I couldn't.
Despite that, I was always paranoid I wasn't doing enough, because it always felt like there was someone else that was shipping more code than I was. Some of this was almost certainly social comparison bias and impostor syndrome-like feelings at work; but I also had a string of managers that pointed out all the work I was doing, and how I was helping the team as a whole.
Eventually, the company got acquired by exactly the sort of company this article is about, my manager got a new director from outside the company, and my manager had to go on extended medical leave after a cancer diagnosis, leaving the director with ~7 new reports. I started hearing about how the number of PRs I was opening weren't as numerous as some other people's, and the code didn't look "hard" enough to their glance. Never mind if the easy code was hard to come to, or if talking through it after the fact they agreed with my assessment, or if I had performed a detailed investigation and writeup, or if my peers left reviews or public plaudits about work I had done. Those weren't PRs, which is ultimately were what they wanted, since that was the metric they could easily see, and justify to their boss.
I did _try_ to do better by their metric, though I never had a definition of what "better" would actually be. Funnily enough, that person was fired a few months after I was.
Also kind of funny to me is that, if I weren't motivated and didn't care, none of this would've affected me all that much.
Depression isn't just feeling sad. It's not necessarily caused by anything external. You cannot necessarily just "figure out why" you feel bad; that's really not how it usually works.
>It's not necessarily caused by anything external. Y
Then how could a drug fix it? We're positing that there is not only a mechanism causing it, but that this mechanism can be manipulated external to their own self/agency/whatever.
I think that it is at least as absurd to posit that you can come up with one chemical substance or another that will alleviate their depression when you dismiss the idea of coming up with a sequence of words spoken to them that might alleviate their depression. It's the conceit that we have a better idea of how their brains work chemically than we do of how their brains work cognitively.
Many things that are not necessarily caused by anything external can be fixed by drugs. I don't understand your point.
If the problem is brain not working correctly (because some organ is not doing it's job properly) then no sequence of words will make the brain physically fix itself, just like no sequence of words will cure a heart attack.
Of course it depends, and many people just need a correct therapy. I'm not dismissing talking and figuring out the root cause.
> You cannot necessarily just "figure out why" you feel bad
Well, of course, if you anesthetize someone they can't feel anything. If you cut off the physical pathways of ""feeling sad"" then they can't feel sad, but is that really the same as "fixing" the reason for why they were feeling sad in the first place?
Unless the reason was that the physical causes are running haywire and making someone feel sad when they otherwise wouldn't, but how often is that just uhh a lazy scapegoat? "Oh this person has no reason to feel sad, something must be wrong in their brain"
> Well, of course, if you anesthetize someone they can't feel anything.
Not what the post you replied to was talking about.
> Unless the reason was that the physical causes are running haywire and making someone feel sad when they otherwise wouldn't, but how often is that just uhh a lazy scapegoat? "Oh this person has no reason to feel sad, something must be wrong in their brain"
That's basically the definition of clinical depression. Doctors try quite hard to make sure it's not a scapegoat.
I wouldn't go that far, but there was a now-famous study (Princeton?) that showed that doing aerobic exercise for maybe 30 mins every day, about five days per week, was equally effective at alleviating depression symptoms.
There's one big problem with that - getting seriously depressed people to do 30 minutes of exercise (or anything else) five days a week. "Get more exercise" is excellent advice for someone who feels a bit down, but it's absolutely useless for someone who can barely summon up the strength to eat or brush their teeth.
It gets even harder if you offer them the alternative of just taking a pill. For widespread health policy, we should want the proportion of depressives who will never learn to manage it themselves because a pill is offered to be smaller than the proportion for whom the pill is effective. I had always assumed these pills were effective enough but studies like these make me wonder.
They are by no means mutually exclusive. If you want depressed people to get more exercise, then a really useful starting point is to give them a pill that could rapidly increase their energy and motivation. The idea that people will be stuck on those pills forever is just lazy psychiatry; ongoing maintenance treatment is often the best option for patients with a history of severe depression and a high risk of relapse, but antidepressants are equally useful as a stepping-stone towards self-management.
Bluntly, if someone is capable of actually starting and sticking with an exercise routine, then they aren't very depressed and should not be offered medication as a first-line treatment. Antidepressants are markedly less effective in patients with milder illnesses, so psychotherapy and lifestyle interventions are a far better initial treatment option. It's only when these treatments fail - or when engaging with them is severely impaired by the severity of the illness - that medication becomes the favoured option.
It’s always frustrating to see the implication that people just need to exercise to solve their mental health struggles. It might not be your intention, but it's a take I see a lot online from influencers.
I say this as someone who is extremely fit. I've worked out religiously since high school. While exercise is integral to me feeling somewhat normal and provides a short-term boost, that is just not how it works for everyone. Some of us have 'broken brains' that cardio can't fix.
Exercise manages my baseline, but sertraline is what helped me finally bridge the gap. It allowed me to regulate my emotions and anxiety in a way that no amount of exercise ever did. And the introspection from being on it helped me make lifelong changes.
To be honest, fearmongering from folks online is what stopped me from taking it sooner, but I wish I had. It was fairly life-changing.
Yeah, it does matter, though the issue is not exactly just monetary profit. The fundamental problem is OpenAI has made the GPT model weights artificially scarce. But at the same time they claim that other artificially scarce information such as books should not be scarce and instead belong to the intellectual commons. The latter part which I agree with, but they took from the commons and are claiming what they took as exclusively their own. That is just evil.
There would be no problem if they open-sourced everything including the model weights. That was their original mission which they have abandoned.
Another fundamental difference: OpenAI explicitly markets their tool as a replacement for the copyrighted material it was trained on. This is most explicit for image generation, but applies to text as well.
As a reminder, the 4 factors of "fair use" in the United States:
1. the purpose and character of the use, including whether such use is of a commercial nature or is for nonprofit educational purposes;
2. the nature of the copyrighted work;
3. the amount and substantiality of the portion used in relation to the copyrighted work as a whole; and
4. the effect of the use upon the potential market for or value of the copyrighted work.
What would you propose? Donating money to blackhole political organizations? Taking to the streets? None of that, or much of anything else, will "do something about this". There is no doing anything about this. It's also just meaningless posturing, just another tantrum by a man-child.
If only there was a clause in your constitution that you hold particularly dear and deals with tyrannical governments. Either that or wait 3 years, show's just as entertaining for the rest of us.
There's a shit-ton of bad and inefficient code on the internet. Lots of it. And it was used to train these LLMs as much as the good code.
In other words, the LLMs are great if you're OK with mediocrity at best. Mediocrity is occasionally good enough, but it can spell death for a company when key parts of it are mediocre.
I'm afraid a lot of the executives who fantasize about replacing humans with AI are going to have to learn this the hard way.