HSM have other security properties that you cannot replicate with software. They are tamper resistant in a way a regular server is never going to be, and they have been engineered to prevent sidechannel attacks. The latter is something very hard to prevent with a regular server.

I agree that for the majority of usecases, a HSM is not necessary, but they do bring security to the table that a simple auth server cannot, at least not without significant engineering effort.