6 ms·

I tried building with LLMs but it has the basic problem that it’s totally wrong 20-50% of the time. That’s very meaningful for most business cases. It does fine

I tried building with LLMs but it has the basic problem that it’s totally wrong 20-50% of the time. That’s very meaningful for most business cases. It does fine when accuracy isn’t important but that’s fairly rare other than writing throwaway content

jiggawatts 3y ago
But you probably didn’t use GPT 4 because its API is available only to a select few. “I tried a known-bad early iteration of a technology that has already been superseded — hence it is bad and will always be bad.” is not a convincing argument. The first transistor was an ugly and impractical thing too.
- thaanpaa 3y ago
  I have access to OpenAI's GPT4, and there appears to be no significant difference from GPT3 (or GPT3.5). It looks like we are pretty far into the curve of diminishing returns already.
  - laratied 3y ago
    This is an absurd statement. I really don't understand what you can possibly be asking it. It doesn't matter though. I would prefer you don't use it.
    - thaanpaa 3y ago
      What a strange thing to be upset about.
toolslive 3y ago
ideally for marketing purposes. It can generate a several slogans, or posters and you can pick the one you like most. However if it needs to generate a graph, and the graph needs to be accurate, you're out of luck.
phillipcarter 3y ago
> totally wrong 20-50% of the time That to me feels like there's some prompting improvements that you could do. It could be that your problem is just harder for LLMs than ours, but 20-50% of the time isn't what we observed after several prompting changes. The other thing is that we do regularly get outputs that are "mostly correct" and we can actually correct them manually, so while the model may have a higher fault rate, the actual end-user experience has a lower one.