Indeed. The whole AI game is predicated on the fact that they can deliver work equivalent to humans in some cases. If that is never going to be the case, then this whole agentic stuff goes belly-up.
The alternative scenario is they get better and do some work really well. That is an interesting territory to focus on.
I think you'd want it to correctly compute your taxes. Especially if you get a letter a year or two after the fact saying you owe the government money