The tricky requirement that ends up existing and torpedoing attempts to clean up feature flags is a requirement for long-term holdback.
e.g. "Test was successful so it's rolling out to all users, minus a 0.5% holdback population for the next 2 years"
This then forces the team to maintain the two paths for the long-term, ensuring the team might get re-orged / re-prioritize their projects sometime a year later making the cleanup really hard to eventually enforce.
> "Test was successful so it's rolling out to all users, minus a 0.5% holdback population for the next 2 years"
Man, I couldn't imagine being a user in such a situation. "Oh, I guess I'm just not getting the better functionality?" Even worse if I were a paying customer.
It’s actually usually the paying customers asking via support to be added to the holdback, improved experience or no.
This is more true for larger flags that substantially change the experience and may not implement niche or edge-case functionality. Obviously you want to avoid these kinds of tests if possible but it’s not always possible.
e.g. "Test was successful so it's rolling out to all users, minus a 0.5% holdback population for the next 2 years"
This then forces the team to maintain the two paths for the long-term, ensuring the team might get re-orged / re-prioritize their projects sometime a year later making the cleanup really hard to eventually enforce.