3 ms·
I'm seeing the same behaviour. It's as if they have a post-processor that evaluates the quality of the response after a certain number of tokens have been gener
by selcuka 3y ago
I'm seeing the same behaviour. It's as if they have a post-processor that evaluates the quality of the response after a certain number of tokens have been generated, and reverts the response if it's below a threshold.
- dhon_ 3y agoI've noticed Gemini exhibiting similar behaviour. It will start to answer, for example, a programming question - only to delete the answer and replace it with something along the lines of "I'm only a language model, I don't know how to do that"
- flakiness 3y agoThe linked article mentions these safeguards as the post-processing step.
- visarga 3y agoThey have both pre and post-LLM filters.
- baby 3y agoAlways very frustrating when it happens.
- extraduder_ire 3y agoThis seems like a bizarre way to handle this. Unless there's some level of malicious compliance, I don't see why they wouldn't just hide the output until the filtering step is completed. Maybe they're incredibly concerned about it appearing responsive in the average case. Would not be surprised if there were browser extensions/userscripts to keep a copy of the text when it gets deleted and mark it as such.
- Breza 3y agoI've seen the exact same thing! Gemini put together an impressive bash one liner then deleted it.
- chupchap 3y agoIt might be copyright related and not quality related. What if X% of it is a direct ripoff an existing song?