Of course, such a system already exists, although it makes more and better use of mice than Emacs. See http://plan9.bell-labs.com/sys/doc/acme/acme.pdf for details.