> The paper was published in 2014, so the period is 2010-2014, not 2010-2025.
Oh indeed, my bad.
> The bugs were randomly chosen; "hand picked" would imply that the authors investigated the contents of the bug reports deeply before deciding whether to include them
No, no, they were not randomly chosen, they even have a whole paragraph explaining how they randomly picked _from a pool of manually selected bugs_. Their criterion of selection varied from "being serious", to "having a lot of comments" and "they can understand the patch", or "the patch is not from the reporter".
> The authors also provide a measure of statistical confidence in the 'Limitations section'.
This is a measure of confidence of their random sampling being representative of the hand picked bugs...
> under standard assumptions, the Central Limit Theorem predicts a 6.9% margin of error at the 95% confidence level
I would love to see them prove the normality assumption of bug root cause distribution.
Also, the whole categorization they do seem purely qualitative.
I have been guilty of such kind of comments and I have come to realize they do nothing to further my point. I would suggest to read the guidelines because they are very well written about this: https://news.ycombinator.com/newsguidelines.html
Oh indeed, my bad.
> The bugs were randomly chosen; "hand picked" would imply that the authors investigated the contents of the bug reports deeply before deciding whether to include them
No, no, they were not randomly chosen, they even have a whole paragraph explaining how they randomly picked _from a pool of manually selected bugs_. Their criterion of selection varied from "being serious", to "having a lot of comments" and "they can understand the patch", or "the patch is not from the reporter".
> The authors also provide a measure of statistical confidence in the 'Limitations section'.
This is a measure of confidence of their random sampling being representative of the hand picked bugs...
> under standard assumptions, the Central Limit Theorem predicts a 6.9% margin of error at the 95% confidence level
I would love to see them prove the normality assumption of bug root cause distribution.
Also, the whole categorization they do seem purely qualitative.
This paper seems to lack rigor to me.