4 ms·
I’ve used it. The API is incredibly buggy and flakey. A particular pain point is the “recitation error” fiasco. If you’re developing a real world app this basic
by arresin 2y ago
I’ve used it. The API is incredibly buggy and flakey. A particular pain point is the “recitation error” fiasco. If you’re developing a real world app this basically makes the Gemini api unusable. It strikes me as a kind of “Potemkin” service.
Google is aware of the issue and it has been open on google's bug tracker since March 2024: https://issuetracker.google.com/issues/331677495 https://issuetracker.google.com/issues/331677495
There is also discussion on GitHub: https://github.com/google-gemini/generative-ai-js/issues/138 https://github.com/google-gemini/generative-ai-js/issues/138
It stems from something google added intentionally to prevent copyright material being returned verbatim (ala the NYT openai fiasco), so they dialled up the "recitation" control (the act of repeating training data—and maybe data they should not have legally trained on).
Here are some quotes from the bug tracker page:
> I got this error by just asking "Who is Google?"
> We're encountering recitation errors even with basic tutorials on application development. When bootstrapping a Spring Boot app, we're flagged for the pom.xml being too similar to some blog posts.
> This error is a deal breaker... It occurs hundreds of times a day for our users and massively degrades their UX.
- deleted 2y ago[deleted]
- screye 2y agoThe recitation error is a big deal. I was ready to champion gemini use across my organization, and the recitation issue curbed any enthusiasm I had. It's opaque and Google has yet to suggest a mitigation. Your comment is not hyperbole. It's a genuine expression of how angry many customers are.
- 3choff 2y agoIt seems the recitation problem has been fixed with the latest models. In my tests, the answer generation no longer stops prematurely. Before I resume a project that was on hold due to this issue, I'm gathering feedback from other users about their experiences with this and the new models. Are you still having this problem?