I have no background in statistics, but that may be to my advantage here. Doesn't keeping confidence intervals vague in the same way keeping estimates vague help?

"Yes, I think it's a minor change. Almost certainly done by the end of the week, or else we have bigger problems."

"I don't know what this entails. Looks like it could be done in a day, but I wouldn't bank on that unless I can spend an hour investigating."

"This is a large feature, but nothing seems terribly complex. I think it's one or two weeks, but we should be ready for it to get snagged on small defects. We should leave room after the release for a rapid followup patch that may be needed."

> If your decide your confidence intervals are described by classical statistics, then adding them won't change anything on the outcome.

I'm trying to understand what you mean by this. Are you referring to the fact that for thin tails, the sum of expectations will grow significantly faster than the standard deviation, and thus the 90th percentile will, relatively speaking, tend toward the expectation with more tasks? (I.e. an appeal to the LLN.)

I believe what the parent is referring to is that the sum of two normally distributed variables can be modeled as normally distributed[1]. Ie: normal distributions are closed under addition. This fact leads to one's naive idea about adding variances working as expected.

However, in task estimation[duration] aren't normally distributed - they're much more log-normally distributed, and you can't simply add log-normal distribution parameters in the same way[2]. Instead he log-normal distribution is (largely) closed under multiplication which is fairly useless when we want to add tasks to determine total time.

Moreover, the reductive conceptualization of normally distributed estimations leads people to the erroneous communication of "X plus or minus Y". The assumption is that the bounds are symmetric(they aren't). The chance that the estimate is higher than the mode is much greater than it is lower than the mode.

As a data scientist, I want to say that the model for task estimation is over-reduced. Even so, I feel like the push-back I'd get from suggesting that folks estimate in log-space would be that I'm being ridiculous complicated.

1. https://en.wikipedia.org/wiki/Sum_of_normally_distributed_ra...

2. See the Fenton-Wilkinson approximation.

> Even so, I feel like the push-back I'd get from suggesting that folks estimate in log-space would be that I'm being ridiculous complicated

Its a more theoretically grounded relative of the common agile “T-shirt sizes quantified by Fibonacci sequence, without the repeated ‘1’” estimation recommendation (which is pretty close to just doing “each complexity step is ×1.5”.)

Adding variances is not a naive idea but a fundamental property of variances! And I don't think that's what GP means because they bring up heavy tails as a point separate from "classical" methods -- and the lognormal distribution is heavy-tailed under appropriate parameters.

Yes, it's about the LLN. At the end you will get an error proportionally smaller than a typical confidence interval that everybody already has an intuition for.

You won't add meaningful information to the analysis.

For software estimation, you can get a long way by just assuming that the distribution is lognormal.

https://jbconsulting.substack.com/p/task-estimation-conqueri...

The log-normal is one of the distributions that can lead to a perfectly tame project or to everything blowing out and no expectation being met with the same numbers, just by changing the estimation methodology.

That said, I don't think the findings on that article generalize well. There seem to exist some properties that makes many software projects more erratic than a log-normal would imply.

I appreciate the feedback but I'm not sure I follow. What changes to the estimation methodology do you mean? And what properties make software projects more erratic than a log-normal (which is already very erratic)?

For example, the results are absurdly sensitive to the choice of estimating your uncertainty as a defined deviation, or a defined low percentile within it, or a defined high percentile within it. They are also absurdly sensitive to the choice of estimating a mode or a mean value.

On more erratic than log-normal, well, I mean it. There are projects that can't be fit to a log-normal, they have fatter tails. I have no idea why. A log-normal is something tractable, many projects aren't even that.

I'm still not sure I follow. In the article, the uncertainty is not intended to be independently estimated. The distribution described has a single parameter, which is the median completion estimate, and that is the only parameter needed; the uncertainty is fully derived from that. The distribution is not absurdly sensitive to this; it's just the scale parameter, so it scales the entire distribution linearly, which is a property that you would expect from fundamental symmetry.

(Unless you mean that the summary of the estimate, rather than the shape of the distribution, varies a lot depending on whether you're quoting a 95% or a 99% confidence interval. But that is just the nature of long-tailed distributions).

I'm also not sure what it means to "fit" a project to a log-normal. A single project doesn't have a distribution, unless you're measuring completion time of individual tickets within that project. The entire project's completion time might be very far off of the median estimate, but as long as the distribution assigns a nonzero probability to it, it's hard to say from just one sample whether it was right or wrong.

If you are measuring the distribution of the individual tickets and they aren't distributed along a log-normal, I'd be very surprised; it would be worth seeing what distribution they do fall into to learn what state of knowledge is being captured by those estimates.