Lisp and Haskell to my mind form the two leading languages of two fundamentally contradictory families. Lisp works by empowering programmers and building on that power, and Haskell works by limiting the programmer and building on those limitations. Despite what the leading word "limitation" may lead you to believe, both philosophies can lead to great results. For example, for the same basic reasons you can't even build an STM in C#, it's probably even more impossible to build an STM in Lisp, whereas in Haskell it's pretty easy, because of the limitations.
You can not just macro your way in a (conventional) Lisp over to Haskell, because Haskell is fundamentally built on limitations being not merely suggestions, but things you can then further build on. If you do, with something like Clojure and its data model, you've got something that isn't the same language any more, and you need all-new libraries to make it work.
You can build an STM in pretty much any language. There have been STMs in at least C++, Java, C#, Common Lisp, Clojure and OCaml.
Haskell may be one of the rare exceptions. Haskell's STM was not implemented in Haskell, and AFAIK cannot be implemented in type safe Haskell due to limitations of its type system unless you add specific extensions required for things like STM.
So to say that STM is enabled by the limitations in Haskell is exactly backwards.
"You can build an STM in pretty much any language."
Sure, but they don't work. STMs don't work because whenever a transaction is retried, anything that operated on "real state" gets redone, and attempts to deal with that with unrestricted side effects are generally agreed to have been a failure.
(Except Clojure, which built it in either from day 1 or at least very early, and in fact does manage effects.)
"Haskell's STM was not implemented in Haskell, and AFAIK cannot be implemented in type safe Haskell"
I just skimmed the entire STM library in Haskell and have no idea what you are talking about. I don't see anything terribly exotic in the LANGUAGE declarations: CPP, DeriveDataTypeable, FlexibleInstances, MultiParamTypeClasses, MagicHash. That may not be Haskell 98 but nobody cares, those are all either very standard at this point or completely boring (CPP ("use the C preprocessor"), MagicHash ("let me end variables with #")). unsafePerformIO doesn't even show up.
By the way, "skimming the entire library" is about a minute of your time. The whole thing's just shy of 16KB of source.
Where do I find this library? The only thing I've been able to find is a bunch of files doing no real stuff except building a nicer interface on top of more primitive STM operations.
As far as I know you cannot implement typed variables (monadic, of course) in Haskell in a type safe way. Note that I'm not even talking about efficient references, but any implementation of references. I could be wrong though.
A natural way would be to start with the State monad and make a State of Heap, where Heap is a data type that will hold all the variables and passing around this Heap will be hidden by the State monad. Unfortunately you're now stuck when trying to implement operations for allocating a new variable and for setting/getting the value of a variable. If Haskell were dynamically typed this would be easy: just make Heap an array or list.
I'd disagree with you that those STMs are broken. Yeah, you have to do all your state in transactional variables, but then the same is true for Haskell. True, Haskell enforces this in its type system, but that's hardly a requirement for a library. By the same logic Clojure's string library is broken, because it does not statically enforce that you actually pass strings to the library. Neither do all Haskell libraries enforce all their preconditions/invariants statically, or at all. And actually Haskell doesn't really enforce that you don't use external effects in STM, since you can always do unsafePerformIO. When working with other languages you should regard external effects as you would regard unsafePerformIO in Haskell.
You can not just macro your way in a (conventional) Lisp over to Haskell, because Haskell is fundamentally built on limitations being not merely suggestions, but things you can then further build on. If you do, with something like Clojure and its data model, you've got something that isn't the same language any more, and you need all-new libraries to make it work.