The number itself is no longer permanently their network, sure - however, they have instant access to look up* any number and find who presently owns it - which they must use to be able to route calls successfully. It's completely true that say, T-Mobile could today refuse to connect a call when the caller pretends to be calling from a T-Mobile owned number and yet is calling from outside their network.

Sure, it would only kill a subset of the spam if they could only do this when they currently control the spoofed number themselves, however it would still do something!

* Source: https://teraquant.com/local-number-portability-and-how-to-tr...