I think you assume that encryption keys are held by people like a house key in their pocket. That's not the case for organizations who are security obsessed. They put their keys in HSMs. They practice defense in depth. They build least-privilege access controls.
Or how much it is cost to kidnap significant one of key bearer(s)?
I think, it is very reasonable sums for governments of almost any country.