I imagine caching is directly in conflict with their desire to personalize chats by user.

See: ChatGPT's memory features. Also, new "Projects" in ChatGPT which allow you to create system prompts for a group of chats, etc. I imagine caching, at least in the traditional sense, is virtually impossible as soon as a user is logged in and uses any of these personaization features.

Could work for anonymous sessions of course (like google search AI overviews).