> And rejecting the data driven answer and then filling in your own interpretation retroactively is not a good way to arrive at conclusions that turn out to be correct.

That sounds like the scientific process to me. You look at a bunch of data points and say, "I'm going to reject the notion that this is an arbitrary correlation, and hypothesize that in fact there is some underlying cause X that drives the correlation." Then you control for everything else and do an experiment.

In tech startups, this might mean saying, "I think age doesn't actually matter, what matters is that someone in the founder's immediate circle (perhaps oneself) has a wealth of domain information." That's a perfectly testable hypothesis, even though the experiment would be expensive and difficult to do cleanly. You just take a bunch of older and younger founders, and have them found companies outside their domains. Some of them you provide with VCs and advisors with plenty of domain knowledge, others you just give money. If the domain knowledge proves more valuable than the age, you have confirmed your hypothesis. If the age matters more than the domain knowledge, you have rejected it.

Now of course you wouldn't find older entrepreneurs who would willingly forgo VCs and advisors with domain knowledge, and most founders choose industries they are familiar with. But maybe you could find some markets where this happens naturally. Maybe you could do an analysis of some industry like online education or fashion or something where you have a mix of types of founders, and see who has success. In any case, rejecting correlational data in favor of testable underlying causes is fundamental to scientific induction (it is only the difficulty of actually controlling ancillary variables that makes this practice hard to apply to sociological hypotheses).

> That sounds like the scientific process to me.

> Then you control for everything else and do an experiment.

What grellas was doing was coming up with alternate explanations based on intuition and rejecting the conclusions of those that did do an experiment. And he was doing so retroactively. That makes it not science, but what is often called a "just so story" and is technically called the ad hoc fallacy.

Speculation about how to do a better experiment is a great discussion to have, but both yourself and grellas are moving towards a perfect solution fallacy, where you are ignoring the data you have now, that tells you something interesting in favour of waiting for some ideal study in the future (that can somehow test underlying causes in a sociological problem, which is, as you say, extraordinarily difficult).

This is a normal reaction to cognitive dissonance, find a flaw in the source that contradicts what you believe (it's correlational) and then you don't have to change your mind while you wait for a perfect solution. Seems to make sense but what you are actually doing is believing the narrative that has less evidence and then rationalizing that belief.

The idea of an accelerator actually running a controlled study is an interesting one, but the structure of such a study would mean that you would be very confident in getting lower returns than normal for that cohort. Maybe you would discover something that made it worthwhile from the data, but likely you would not. It would be great for everyone if someone did it, but no one wants to literally sabotage young entrepreneurs in order to maintain valid controls. Maybe you could use the same ethical arguments that medicine uses ... but it would be a hard sell.