I believed Hyper V is more suited if the machine is a server hosting the VM, where client remote into this server from another machine (correct me if I am wrong; perhaps Xen works the same way as well).

It would be nice to get some suggestions on Minimalist/Lightweight Host OS to run VirtualBox. Lubuntu?