3 ms·
I did get an answer from a senior executive at one AI lab who called this the "regurgitation problem" and said that they pay very close attention to it, to the
by simonw 10mo ago
I did get an answer from a senior executive at one AI lab who called this the "regurgitation problem" and said that they pay very close attention to it, to the point that they won't ship model improvements if they are demonstrated to cause this.
- nprateem 10mo agoLol and that was enough for you? You really think they can test every single prompt before release to see if it regurgitates stuff? Did this exec work in sales too :-D
- simonw 10mo agoI think they can run benchmarks to see how likely it is for prompts to return exact copies of their training data and use those benchmarks to help tune their training procedures.
- TeMPOraL 10mo agoThey have a clear incentive to do exactly as said - regurgitation is a problem, because it indicates the model failed to learn from the data, and merely memorized it.