3 ms·

Anyone noticed that chat history is gone? I was wondering how can they afford to keep all that data, I thought that maybe its peanuts compared to the cost of r

Anyone noticed that chat history is gone? I was wondering how can they afford to keep all that data, I thought that maybe its peanuts compared to the cost of running the GPT, but well, maybe it's not after all. If the free version will be around for some time, I guess they could create a browser extension to store the responses. Or something as simple as allowing to download the conversation data and load it. Anything would be better than no chat history at all.

mistercow 4y ago
Looks like it’s a temporary situation: > Due to high demand on our systems, previous conversations are temporarily unavailable I find it hard to believe that that’s a data volume problem, though. It’s just relatively small snippets of text, and they’re almost certainly keeping the data because keeping the data is one of the main points of the research preview. My guess is that disabling history is a way to mitigate certain user patterns that increase load.
- replygirl 4y ago
  yes, i do believe "high demand" implies increased load
- londons_explore 4y ago
  Each of those history items was selectable, and you could continue any of those conversations. To do that, they need to store the model state (to allow continuation). But when they switch out the model, the old model states wouldn't be valid anymore - they need regenerating. Regenerating the model state for a new model uses as much computation as the original conversation did. So, if they want to restore history, they have to do all the compute work they have done in the past, again. Thats gonna be a lot of GPU-hours - Expensive! Another option is they set up another smaller farm of the old model version, and use that for continuing old conversations - but obviously thats some effort to set up, and they'd need to dedicate a set of a few hundred GPU's to running each old version forever. - Expensive! Or yet another option is they show old conversations, but don't allow continuation. That probably just requires a little dev work - and in their position, thats what I'd do. - Worse user experience.

mistercow 4y ago

Looks like it’s a temporary situation: > Due to high demand on our systems, previous conversations are temporarily unavailable I find it hard to believe that that’s a data volume problem, though. It’s just relatively small snippets of text, and they’re almost certainly keeping the data because keeping the data is one of the main points of the research preview. My guess is that disabling history is a way to mitigate certain user patterns that increase load.

replygirl 4y ago
yes, i do believe "high demand" implies increased load
londons_explore 4y ago
Each of those history items was selectable, and you could continue any of those conversations. To do that, they need to store the model state (to allow continuation). But when they switch out the model, the old model states wouldn't be valid anymore - they need regenerating. Regenerating the model state for a new model uses as much computation as the original conversation did. So, if they want to restore history, they have to do all the compute work they have done in the past, again. Thats gonna be a lot of GPU-hours - Expensive! Another option is they set up another smaller farm of the old model version, and use that for continuing old conversations - but obviously thats some effort to set up, and they'd need to dedicate a set of a few hundred GPU's to running each old version forever. - Expensive! Or yet another option is they show old conversations, but don't allow continuation. That probably just requires a little dev work - and in their position, thats what I'd do. - Worse user experience.