This is _so_ on point considering a lot of my customer interactions. Until I go in and go over establishing a process framework that goes through the models in use and sets criteria for their evaluation (even something as simple as ROC), a lot of ML work in companies is mostly tinkering with things (sometimes with wrong theoretical underpinnings) until it performs adequately.
The hype is only real if you systematically work it into measurable process, not virtuoso jamming.
Stone age tools as Chomsky calls them. The problem with simple to use tools, is the simplicity masks two key facts from the user - whether aptitude for the tool exists & amount of effort and time to mastery.
The hype is only real if you systematically work it into measurable process, not virtuoso jamming.