That's the nature of these add-ins. Modern Add-ins are all little XML frames with some JS or whatever. All processing occurs server-side, hosted by the add-in publisher.
This is counter to the old (security nightmare) COM model where processing could be local.
To clarify: add-ins are essentially web pages. They can do some processing client side if they want, but yeah in the case of a ChatGPT add-in it's not like they're running the model in a web frame.
This is counter to the old (security nightmare) COM model where processing could be local.