Random sampling, basically. The value of random sampling is hard to overstate - it gives you a real picture of what's going on. A non-random sampling gives you a picture, but you have no way of confirming that the picture is a reflection of reality.
Random sampling and privacy run into conflicts not just in the browser space, but everywhere else. For example, recently the Canadian government went through a period where it allowed census respondents to optionally answer some questions that were previously mandatory (using privacy arguments). The result was several years of poor census information. The recent government reinstated the mandatory census questions.
The browser is just one arena where this everpresent conflict between knowledge and privacy plays out.
I've used Netscape then switched to Firefox when Netscape became way too bloated, then enjoyed years and years of Firefox getting better, supporting new JS and HTML5 features all WITHOUT telemetry and with the Crash Reporting window where I can see the data that is being submitted and submit it if I want to submit it.
What have changed so much in last 5 years or so that now you have to get all this data? What is wrong with just building a standarts compliant browser that runs JS fast and has easy to understand settings (where I don't have to go to about:config to disable the WebRTC/telemetry/Pocket etc) ?
> What have changed so much in last 5 years or so that now you have to get all this data?
To be honest, a lot. Once again, this is my personal take on the matter, not Mozilla's view.
First off, browsers were a LOT simpler back then. The sophistication and complexity in a browser has grown significantly in the last decade or so.
Secondly, browsers have matured. Remember that this software category has only been around for 20 years or so. Compared to the code quality in browsers today, browsers of 10 years ago were crude and simple. As a software category matures, the low-hanging fruit dry up, so it's harder and harder to improve your product.
Lastly, competition. Firefox has the luxury of being released when the biggest competitor (Microsoft) wasn't putting real effort into its browser product. Google will not make that same mistake with Chrome.
Basically, the information we needed back then was less, because the problems were much more obvious, because the whole industry was still pretty young. Now browsers are much more mature, the ecosystem is much more complex, has a much wider user base, and the problems are becoming harder and harder to pin down.
I think it's good to annotate this with your other comment as to why Firefox has to join this competition, rather than just do it's think and disregard its marketshare:
> A perfectly private browser with no marketshare doesn't help users. A completely compromised browser with 100% marketshare doesn't help users either.
You might miss out on issues if users don't submit, but each submission is an indication of problem (because it's Firefox that decides a problem is bad enough). And you can still prioritize based on how common that problem is.
Under the "collect and prompt" scheme, you are still sampling randomly.
A random sample of users experiences perf issues, a random sample of users opts-in to the collection, you get a random-sample of data. (If you suggest they opt-in to continued collection, you might even get a continuous stream of samples from the same user.)
Yes, that data won't cover the people who don't have issues, but do you need to optimize for them? It also won't cover people who have issues but still don't opt-in, but do you think that is somehow correlated to the severity of the issue? Otherwise the data will be mostly unbiased. The variance will be higher than if you made it opt-out, but if you are doing sound statistics, you will have to handle that anyway.
The thing is, I think, that the users that opt-in to the collection aren't "a random sample", but rather "a sample of users biased towards certain profiles".
In part, AFAIA (but I've only limited statistical training) this is a well known feature of voluntary datasets. But IIRC they've also done user studies and compared those.
Random sampling and privacy run into conflicts not just in the browser space, but everywhere else. For example, recently the Canadian government went through a period where it allowed census respondents to optionally answer some questions that were previously mandatory (using privacy arguments). The result was several years of poor census information. The recent government reinstated the mandatory census questions.
The browser is just one arena where this everpresent conflict between knowledge and privacy plays out.