You also have to trust your VM host not to provide an emulated SGX (which is what https://github.com/sslab-gatech/opensgx is, unless I'm very much mistaken).
I think you're mistaken. Intel provides infrastructure to ensure you're talking to an enclave running on an actual Intel machine, and you can then do a remote attestation to verify the contents of that enclave.
