The way I remember it was everyone was anticipating the Apple phone, but nobody knew what it would be. It wasn't like only visionaries had the concept.
I remember (back when Cingular was a thing) seeing the iTunes phone in a store, and thinking "really? ok, I guess.."
Now in restrospect, apparently the party line was it was a massive failure, because that's what everything is that isn't the largest success in history. But living through it, it seemed like just a headfake before the inevitable "real" Apple phone, and an early attempt at extending the iPod to a multifunction device.