There's been a number of augmented terminal programs, such as Mozilla's ancient XMLterm: http://www.xml.com/pub/a/2000/06/07/xmlterm/index.html

I think the big problem is none of them have got the balance right between offering more capabilities that are actually useful vs. how intrusive they are.