4 ms·

With all the money, research, and hype going into these LLM systems over the past few years, I can't help but ask: if I still can't rely on them for simple easy

With all the money, research, and hype going into these LLM systems over the past few years, I can't help but ask: if I still can't rely on them for simple easy-to-check use cases for which there's a lot of good training data out there (e.g., sports trivia [1]), isn't it deeply irresponsible to use them for any non-toy application? [1] https://news.ycombinator.com/item?id=43669364 https://news.ycombinator.com/item?id=43669364

orangevelcro 2y ago
Absolutely. I always thought LLMs were interesting, but in no way 'ready for prime time' and have been absolutely blown away that it seems like it's been shoehorned into every possible real life use case in such a reckless way. I've even heard of lawyers being forced to use it. It is breathtakingly irresponsible.
doug_durham 2y ago
Nope. There are many applications where the current behavior is fine. Thousands of developers use them to be more productive every day.
- rad_gruchalski 2y ago
  [flagged]
- namaria 2y ago
  A feedback with delayed signalling is a recipe for system instability and runway behaviors. We will have a better idea of whether LLMs are fine for code generation a bit further down the line.
bcoates 2y ago
It’s honestly a little surprising that LLMs are good at information retrieval from their training set at all. (Specifically, it's not a surprise that they hallucinate-it's a surprise that sometimes they don't at rates better than chance) It’s not necessary for them to ever be reliable at it for them to be useful... just stop asking them to do things they aren't good at, and may never be.
- jaggederest 2y ago
  > just stop asking them to do things they aren't good at, and may never be. So what are they good at then? Is there a list I can refer to? Maybe I should ask an AI to make a list of things it's good at?
  - bcoates 2y ago
    They convert text to embedding vectors and vice versa. So text->text transformations, autocomplete, adding a text input or output to another model (image generation/scene description), vector similarity search, things of this nature. The pseudo-agi AI girlfriend is a parlor trick
  - southernplaces7 2y ago
    >Maybe I should ask an AI to make a list of things it's good at? But then maybe it's bad at this too, and simply lies to you....
emtel 2y ago
The strength of LLMs is areas where verification is significantly easier than production. This is why they have had so much success as coding assistants.
- bigstrat2003 2y ago
  The problem is that doesn't hold true for coding, which is why it's deeply irresponsible to rely on them for that. If it was so easy to verify code is correct, we never would have bugs in the first place.
  - emtel 2y ago
    Of course it’s true for coding. Most people reading this probably got into programming because of how fun it is to just try things and see if they work. Of course you need a way to verify the code (types, tests, etc), but you already needed that anyway!
    - roxolotl 2y ago
      It’s true for algorithms, it’s not true for software systems. There is a ton of undefined behavior and unverifiable in software that’s widely distributed. A lot of the time this is even because those working on the systems don’t know the edge cases exist.
ksec 2y ago
My exact thinking. It is the wrong end to measure it. As with nearly all tech benchmarks. They measure success rate, if LLM is 95% accurate it is good enough as human on average only get about 97%. What we dont measure is the difference between the 5% and 3% of failure rate. Some of the error in 5% error couldn't have happened to any human or are basically non-human error. Vs say 3% of human error that are different type. I still haven't found a term for this. And I have been saying it since Apple's Keyboard butterfly fiasco. When Apple supporters keep saying the key press has 99.99% success rate so the problem with the keyboard were amplified. What they dont realise was the old scissors keyboard has infinitely close to 100%, or 99.99999999999999999% success rate so to speak. A normal consumer comparing intuitively will felt the butterfly keyboard is 0.01 / 0.000000000000001 many times worst because the error doesn't happen before with previous keyboard. Since I dont see anyone online doing this I am going to call it Ksec's Law.

orangevelcro 2y ago

Absolutely. I always thought LLMs were interesting, but in no way 'ready for prime time' and have been absolutely blown away that it seems like it's been shoehorned into every possible real life use case in such a reckless way. I've even heard of lawyers being forced to use it. It is breathtakingly irresponsible.

doug_durham 2y ago

Nope. There are many applications where the current behavior is fine. Thousands of developers use them to be more productive every day.

rad_gruchalski 2y ago
[flagged]
namaria 2y ago
A feedback with delayed signalling is a recipe for system instability and runway behaviors. We will have a better idea of whether LLMs are fine for code generation a bit further down the line.

bcoates 2y ago

It’s honestly a little surprising that LLMs are good at information retrieval from their training set at all. (Specifically, it's not a surprise that they hallucinate-it's a surprise that sometimes they don't at rates better than chance) It’s not necessary for them to ever be reliable at it for them to be useful... just stop asking them to do things they aren't good at, and may never be.

jaggederest 2y ago
> just stop asking them to do things they aren't good at, and may never be. So what are they good at then? Is there a list I can refer to? Maybe I should ask an AI to make a list of things it's good at?
- bcoates 2y ago
  They convert text to embedding vectors and vice versa. So text->text transformations, autocomplete, adding a text input or output to another model (image generation/scene description), vector similarity search, things of this nature. The pseudo-agi AI girlfriend is a parlor trick
- southernplaces7 2y ago
  >Maybe I should ask an AI to make a list of things it's good at? But then maybe it's bad at this too, and simply lies to you....

emtel 2y ago

The strength of LLMs is areas where verification is significantly easier than production. This is why they have had so much success as coding assistants.

bigstrat2003 2y ago
The problem is that doesn't hold true for coding, which is why it's deeply irresponsible to rely on them for that. If it was so easy to verify code is correct, we never would have bugs in the first place.
- emtel 2y ago
  Of course it’s true for coding. Most people reading this probably got into programming because of how fun it is to just try things and see if they work. Of course you need a way to verify the code (types, tests, etc), but you already needed that anyway!
  - roxolotl 2y ago
    It’s true for algorithms, it’s not true for software systems. There is a ton of undefined behavior and unverifiable in software that’s widely distributed. A lot of the time this is even because those working on the systems don’t know the edge cases exist.

ksec 2y ago

My exact thinking. It is the wrong end to measure it. As with nearly all tech benchmarks. They measure success rate, if LLM is 95% accurate it is good enough as human on average only get about 97%. What we dont measure is the difference between the 5% and 3% of failure rate. Some of the error in 5% error couldn't have happened to any human or are basically non-human error. Vs say 3% of human error that are different type. I still haven't found a term for this. And I have been saying it since Apple's Keyboard butterfly fiasco. When Apple supporters keep saying the key press has 99.99% success rate so the problem with the keyboard were amplified. What they dont realise was the old scissors keyboard has infinitely close to 100%, or 99.99999999999999999% success rate so to speak. A normal consumer comparing intuitively will felt the butterfly keyboard is 0.01 / 0.000000000000001 many times worst because the error doesn't happen before with previous keyboard. Since I dont see anyone online doing this I am going to call it Ksec's Law.