> I feel like the hardest parts are getting participants to adhere to their assigned intervention, and preventing participants with a pre-existing agenda from joining.
This combined with perverse incentives are why it's not possible. Non-experts with the resources to solve these problems are generally trying to sell you something, and aren't particularly interested in the truth.
Perhaps so. Saying it's "not possible" seems like an overly strong conclusion.
For specific interventions, I think it might be possible to have open, blinded participation. E.g. for supplements which may affect memory:
- Participants sign up to buy a supplement which they know will be one of k generally safe supplements or a placebo
- The specific supplement or placebo they receive it thompson sampled (or similar) from our bandit given data so far, and sent to them unlabeled
- They're only able to receive their next bottle if they complete weekly tests or measurements
The value prop is that you're probably buying a pretty good intervention, even if you don't know which one it is (and perhaps you're getting a discount subsidized by the minority of people who are paying for a placebo at any given time).
For a number of questions related to exercise, people may fall into camps about their prior beliefs, and may self-sabotage if they're in treatment group that goes against their preferences but there isn't a clear thing to sell. E.g. which of a family of training schedules helps people reduce their 10k running time the most?
This combined with perverse incentives are why it's not possible. Non-experts with the resources to solve these problems are generally trying to sell you something, and aren't particularly interested in the truth.