Exactly, this simple example demonstrates how hard is to learn even a simple function. Playing with your code, 10 hidden units give error 0.6 in 1000 iterations, 1000 hidden units give error 0.02 in 10000 iterations so it takes way longer to train. This is not an easy technology.
https://github.com/ghostFaceKillah/tensorflow-experiments/bl...