I think pretty much everybody is missing an important point of deadlines: money. If you are not done by the deadline you are dead. Management (with marketing) has a good idea of what will sell, fail to make the deadline and the product won't make money in the market and should be canceled early. This is the point of estimates: to decide early if the project is worth paying anyone to work on. (or course once you are part way in the project sunk costs become an issue)
Note a result of this is deadlines are not something you should come close to: a break-even project isn't worth your time to develop, so there should be plenty of slack in the schedule to cut into before hitting the point of no profit.
> to decide early if the project is worth paying anyone to work on.
Yes, we know that’s what you think. We keep trying to explain to you that you’re living in a fantasy world where software development can be predicted accurately in significantly less time than it takes to just do the work. I understand why you WANT that to be true, but wanting something to be true doesn’t make it true, to the frustration of MBAs all over the world.
I do think well-functioning development processes & teams, with good integration with business is something every company wants, but very few companies are willing to pay for, and most just cannot accept that they're opting out of what they claim they want by not paying enough, so they try to get it anyway.
Why's it expensive? I think people who are capable of building and managing such teams, and dragging organizations into a set of processes that truly work, exist at a rate of about 1/1000 the demand for them. Meanwhile the people who can discern useful practices from useless ones or good implementation from bad are similarly rare and overlap largely with that same group, leading to an ecosystem of development practices that's almost entirely snake oil and wishful thinking, and where almost no-one can tell when they're looking at one of those 1/1000 folks who've got the knowledge and skills to make it work if you give them the authority, or some delusional schmuck or huckster.
The best most companies can do is to learn to live with a ton of uncertainty and work humanely and effectively with that, and few achieve even this lesser outcome, which tends to come with its own (lower) costs in hours and less-than-ideal predictability that many orgs just can't accept they have to pay to keep things running even sort-of smoothly.
It's not really faith, it an educated guess:
CEO (or Director of Department) - Makes an educated guess, that if they can release Product A in next 18 months, they will earn $1,000,000.
That means that they need to know that A:
1) Product A will be released in next 12-14 months.
2) They will spend less than $700,000. Otherwise they might be better off buying bonds or something.
It's effectively a job of an exec / director to make bets. Hopefully they are educated guesses, but there are times when somebody just have to flip a coin too.
There is a lot of education in those guesses. We know what previous sales are, we know what industry sales are. We know population growth rates. We know many other things. Everything we know puts bounds on what we can sell.
this is literally why a significant portion of high-end intellectuals refuse to participate in this charade, and instead build quality technology in the open. Open source technology, built largely without artificial deadlines and fake drama, define the execution environment that this exchange is taking place upon (in many cases).
The old industrial plant planning algebra in 1950s textbooks, say something like "if we make widget N in six months with a team of 6 and $100,000 budget, we have a 30% chance of breaking even in two years, but if we make widget P in 12 months with a team of 4 and $20,000, we have a 60% chance of breaking even in 24 months" etc.. this is slightly useful in certain environments, but anyone here is going to argue that the "future" in 12 or 24 months is stable, technically?
The capacity of certain mental models to "Reductio ad Absurdum" is amazing to me, in an age of moving literally one billion+ conversations at once across wires and through space.
In finance, it's called risk appetite. Different people and different entities have a different appetite for risk, and they will act accordingly. There is nothing intrinsically wrong guesswork if parties understand the context, and don't lie to themselves.
>12 or 24 months is stable, technically?
Hm. Faster iterations are better, usually, again if we understand the context, but I see a lot of companies that are too quick to go into full panic mode. An example would be recession – If you plan to invest for the next 60 month, and a recession hits you on month 20, the right choice would be to ignore the inputs, and keep investing. Those who pull their assets when prices are low usually lose.
Quick or long iterations are not be all solutions, once always have to keep the context in mind and use first principles to understand what is that we are doing, and trying to achieve.
GP's comment is underrated. Engineering is about effectively managing to make something within constraints. Memory, CPU, I/O bandwidth, network outages, screen size, all constraints that engineers work within. Development time should be treated as a first class citizen among those constraints.
For example, if you're working on embedded software that gets flashed onto ROM as part of a device's trip down the assembly line onto a boat, and the factory is ramped, you can't just say, oh we just need another month because our estimate was way off. Something is getting flashed onto those ROMs starting Day 0. If you're an engineer working on this software, this constraint needs to be taken as seriously as the OS API or memory footprint.
There are various types of reasons for deadlines. I have seen everything from very good reasons (an external API has to be replaced before its sunset date; regulatory changes that must be met when they come in place) to very bad ones ("$manager said it should be done by $date"; when asked $manager why, the answer was "I thought it should be doable by then, but it's not actually that important").
A deadline without the information of its importance is pretty useless, unless you happen to be in an organization where all deadlines are there for good reasons.
I work in consulting and this is the constant knife-edge we walk on. We scope and plan a project and then have to attach a budget to it in a contract knowing full well that our estimates are going to be wrong. It's such a struggle to get clients with fixed budgets to understand how they have to flex on scope and timing and it's not just because we're slow and lazy.
I once managed a handful of products for manufacturing. Our customers planned their down time, upgrades, etc around our releases. Defects, delays cost our customers real money.
Note a result of this is deadlines are not something you should come close to: a break-even project isn't worth your time to develop, so there should be plenty of slack in the schedule to cut into before hitting the point of no profit.