9 ms·

I was six years old when I learned that garbage in equals garbage out. Nothing has changed. It's the same for LLMs as it was for 8-bit computers as it was nearl

I was six years old when I learned that garbage in equals garbage out. Nothing has changed. It's the same for LLMs as it was for 8-bit computers as it was nearly 200 years ago when Babbage remarked, "I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."

toyg 3y ago
LLMs take "garbage out" a bit further, though. You can have perfect inputs and, technically speaking, there will still be a combination of statistical tricks in the model that results in garbage.
- rchaud 3y ago
  Except a calculator will just show you an error if the inputs can't be calculated (dividing by zero). An AI will give you the wrong answer and act like it's right and that you're wrong.
  - marcosdumay 3y ago
    > and act like it's right and that you're wrong Nah, chatgpt will profoundly apologize and claim your answer is the correct one. It doesn't even matter if it's wrong or not.
    - mrguyorama 3y ago
      Isn't that only after it was caught gaslighting people?
Terr_ 3y ago
For a bit more context, since it's a worthy quote: > On two occasions, I have been asked [by members of Parliament], 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able to rightly apprehend the kind of confusion of ideas that could provoke such a question.
- pmontra 3y ago
  I have no clues about the goal of that question and whom asked it but people sometimes are able to detect mistakes in questions. A simple example: "I think that actually you wanted to ask me 6 times 6, not 6 times 5,because we just measured this room and it's 6 by 6 meters, and the answer is 36." Maybe that member of the Parliament wanted to know if the machine is intelligent enough to reason about their inputs. Nobody born in the last 50 years would ask that question about computers, because we know that they are dumb electrical circuits. Basically nobody had that kind of insight about the mechanical computing machines of the time, when that question was asked. However somebody born in the last few years might start to ask that question again now, because today's machines seem to reason.
  - bonoboTP 3y ago
    Right, I hate when people smugly repeat this anecdote as if it was totally unreasonable to expect something more. The ideas of AI and cybernetics were of course quite far away in Babbage's time (notwithstanding the vision of Kempelen's Mechanical Turk) and learned men were enamored with the cool new steam-powered mechanical passive valve-logic worldview, but normal people's intuitions don't have to follow whatever is the new fad. If 30 years ago I asked a truck enthusiast: "If there's a car in front of me on the highway and I slam on the accelerator and don't touch the brakes, will the truck slow down before collision or am I going to crash into the car?" - Trucker in 1993: "I can't even." Truck today: *brakes*.
    - boredumb 3y ago
      Smug or not asking an LLM to repeat a bleach based recipe and then receiving one is exactly what this quote is alluding to. ""If I ask for something I don't want, will it give me something I want??" - no
      - bonoboTP 3y ago
        It's a design decision and depends on the maker's philosophy. It's not absurd to have it either one way or the other. Is the user assumed to be an idealized empowered rational adult who knows what they want, or more like a toddler babbling around who has no idea what they want and can just approximately gesture in the direction? Or where exactly inbetween. Again, depends on the philosophy.
killerstorm 3y ago
Not quite. If you're using a capable LLM (e.g. GPT-4), you can easily fix this with a prompt, e.g. "remove non-edible items from the ingredients". It's perfectly capable of returning an error when the input is garbage, but it needs a context for it. Without a context it might interpret it as a creative writing task.
- dylan604 3y ago
  Doesn’t this assume that an LLM knows what you mean by non-edible? So it leaves out the plastic and cardboard and not the toxic substances. Or maybe going the other way, it leaves out processed food
  - consp 3y ago
    Cardboard is edible. If it is healthy is another question altogether.
    - hypertele-Xii 3y ago
      Michel Lotito is in the Guinness World Records for having eaten 18 bicycles, 15 supermarket trolleys, seven TV sets, six chandeliers, two beds, a pair of skis, a Cessna light aircraft and a computer.
    - dylan604 3y ago
      That’s my point. Chlorine is edible too. So is arsenic, bug spray, and a plethora of other toxic substances. So your pedantic response was already covered.
      - digging 3y ago
        Well, no. Chlorine is unsafe. Cardboard is just not nutritious. They're different categories.
        smileysteve 3y ago
moss2 3y ago
For AI, nothing in can mean garbage out
- mdp2021 3y ago
  > AI Careful with those words. People are increasingly confusing LLMs with AI, and even further, you seem to be identifying AI as if it were made of LLMs. I had to spell it out even yesterday: AI is the automation of problem solving, focused on reliably giving good solutions to definite problems. If LLMs use technologies developed for AI, this does not make them AI.
  - kortex 3y ago
    > further, you seem to be identifying AI as if it were made of LLMs. GP is doing nothing of the sort. LLMs are a form of AI; artificial intelligence has been around since the 60s. They are not AGI (artificial general intelligence), and no one (credible) is claiming LLMs are AGI.
    - mdp2021 3y ago
      > GP is doing nothing of the sort The poster wrote that «For AI, nothing in can mean garbage out». The poster seems to mean LLMs with «nothing in can mean garbage out». The poster seems to be calling LLMs AI. The poster seems to be attributing to AI properties of LLMs. Hence, the poster seems «to be identifying AI as if it were made of LLMs». > LLMs are a form of AI Prove it (or, defend it). I would say that it is arguable that they are not, unless one describes LLMs as "engines that reliably solve the problem of generating convincing text". The issue with that perspective is that «generating convincing text» is hardly per se a problem: it does not define a complete problem - text does not "stand alone" (content remains crucial). > artificial intelligence has been around since the 60s I know (and I should know decently well. Pedantically, a few years earlier - specifying just in case): what are you trying to say with that? Which application are you proposing to defend your perspective?
      - criddell 3y ago
visarga 3y ago
"Garbage in-garbage out", meet "Quantity has a quality of its own". LLMs are trained on a huge amount of text, this tends to compensate for garbage in. Of course if the model was not properly tested and fine-tuned, a LLM would happily execute any command.
- dylan604 3y ago
  Does it? Or does it just show that even in those quantities, there’s a preponderance of garbage? If it is sourcing places like Twitter, Reddit, and what not, then there’s most definitely a lot of garbage present.
FrustratedMonky 3y ago
>"garbage in equals garbage out" as is for humans
II2II 3y ago
How do we know it is GIGO? Plenty of people like to test edge cases. It is quite possible that someone fed the AI ingredients known to produce chlorine gas to see what would result. That is not to say that the company's excuse is valid. There should definiely be tests of the inputs to ensure they are safe (i.e. test for garbage in). The garbage out bit though would be more difficult seeming as that requires a knowledge of chemistry. Seeming as most of our knowledge of chemistry was founded.on experimentation, then later theory backed by experimentation, it is something that I wouldn't trust in the hands of an LLM.
- michaelmior 3y ago
  Testing the "garbage in" case seems as though it would be sufficient for the vast majority of use cases. Aside from undercooking ingredients which may cause foodborne illnesses, I think you're highly unlikely to create toxic chemical reactions using common cooking ingredients.
matwood 3y ago
Yep, it's literally the first thing kids do when they learn echo at the CLI. Make the computer write out curse words.
pkulak 3y ago
Garbage in should mean nothing out. If you ask a calculator what 2+2 is, but you mean 2+3, you still get the right answer, because you didn't put in garbage, you put in the wrong inputs. If you ask it what "pi + transubstantiation" is, it should fail, not confidently tell you "14".
- bongobingo1 3y ago
  > ”Forty-two,” said Deep Thought, with infinite majesty and calm.
- kibwen 3y ago
  The whole point of Babbage's classic anecdote is that it is possible to create garbage in the step where we encode our expectations into the commands of the machine. Even if the machine operates on its inputs properly, if we gave it inputs that did not properly encode our expectations, that's garbage, and the outcome will be garbage.
chasd00 3y ago
Does garbage in/out really apply to LLMs though? The relationship between input and output is unknown in large LLMs as far as I know . If that’s the case, then you can’t make any claim about “output” relating to “input” when it comes to LLM AI.
- kibwen 3y ago
  Even a hyper-rational mind with infinite computation capacity will draw the wrong conclusions if it is fed incorrect observations. LLMs aren't magic, they're pattern-matching engines. Even if the relationship between input and output is opaque, that doesn't mean they're omniscient.
antod 3y ago
Not so much "garbage in, garbage out" but "anything in, garbage out". It's just the garbage out is getting amazingly close to non garbage.

toyg 3y ago

LLMs take "garbage out" a bit further, though. You can have perfect inputs and, technically speaking, there will still be a combination of statistical tricks in the model that results in garbage.

rchaud 3y ago
Except a calculator will just show you an error if the inputs can't be calculated (dividing by zero). An AI will give you the wrong answer and act like it's right and that you're wrong.
- marcosdumay 3y ago
  > and act like it's right and that you're wrong Nah, chatgpt will profoundly apologize and claim your answer is the correct one. It doesn't even matter if it's wrong or not.
  - mrguyorama 3y ago
    Isn't that only after it was caught gaslighting people?

Terr_ 3y ago

For a bit more context, since it's a worthy quote: > On two occasions, I have been asked [by members of Parliament], 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able to rightly apprehend the kind of confusion of ideas that could provoke such a question.

pmontra 3y ago
I have no clues about the goal of that question and whom asked it but people sometimes are able to detect mistakes in questions. A simple example: "I think that actually you wanted to ask me 6 times 6, not 6 times 5,because we just measured this room and it's 6 by 6 meters, and the answer is 36." Maybe that member of the Parliament wanted to know if the machine is intelligent enough to reason about their inputs. Nobody born in the last 50 years would ask that question about computers, because we know that they are dumb electrical circuits. Basically nobody had that kind of insight about the mechanical computing machines of the time, when that question was asked. However somebody born in the last few years might start to ask that question again now, because today's machines seem to reason.
- bonoboTP 3y ago
  Right, I hate when people smugly repeat this anecdote as if it was totally unreasonable to expect something more. The ideas of AI and cybernetics were of course quite far away in Babbage's time (notwithstanding the vision of Kempelen's Mechanical Turk) and learned men were enamored with the cool new steam-powered mechanical passive valve-logic worldview, but normal people's intuitions don't have to follow whatever is the new fad. If 30 years ago I asked a truck enthusiast: "If there's a car in front of me on the highway and I slam on the accelerator and don't touch the brakes, will the truck slow down before collision or am I going to crash into the car?" - Trucker in 1993: "I can't even." Truck today: *brakes*.
  - boredumb 3y ago
    Smug or not asking an LLM to repeat a bleach based recipe and then receiving one is exactly what this quote is alluding to. ""If I ask for something I don't want, will it give me something I want??" - no
    - bonoboTP 3y ago
      It's a design decision and depends on the maker's philosophy. It's not absurd to have it either one way or the other. Is the user assumed to be an idealized empowered rational adult who knows what they want, or more like a toddler babbling around who has no idea what they want and can just approximately gesture in the direction? Or where exactly inbetween. Again, depends on the philosophy.

killerstorm 3y ago

Not quite. If you're using a capable LLM (e.g. GPT-4), you can easily fix this with a prompt, e.g. "remove non-edible items from the ingredients". It's perfectly capable of returning an error when the input is garbage, but it needs a context for it. Without a context it might interpret it as a creative writing task.

dylan604 3y ago
Doesn’t this assume that an LLM knows what you mean by non-edible? So it leaves out the plastic and cardboard and not the toxic substances. Or maybe going the other way, it leaves out processed food
- consp 3y ago
  Cardboard is edible. If it is healthy is another question altogether.
  - hypertele-Xii 3y ago
    Michel Lotito is in the Guinness World Records for having eaten 18 bicycles, 15 supermarket trolleys, seven TV sets, six chandeliers, two beds, a pair of skis, a Cessna light aircraft and a computer.
  - dylan604 3y ago
    That’s my point. Chlorine is edible too. So is arsenic, bug spray, and a plethora of other toxic substances. So your pedantic response was already covered.
    - digging 3y ago
      Well, no. Chlorine is unsafe. Cardboard is just not nutritious. They're different categories.
      - smileysteve 3y ago

moss2 3y ago

For AI, nothing in can mean garbage out

mdp2021 3y ago
> AI Careful with those words. People are increasingly confusing LLMs with AI, and even further, you seem to be identifying AI as if it were made of LLMs. I had to spell it out even yesterday: AI is the automation of problem solving, focused on reliably giving good solutions to definite problems. If LLMs use technologies developed for AI, this does not make them AI.
- kortex 3y ago
  > further, you seem to be identifying AI as if it were made of LLMs. GP is doing nothing of the sort. LLMs are a form of AI; artificial intelligence has been around since the 60s. They are not AGI (artificial general intelligence), and no one (credible) is claiming LLMs are AGI.
  - mdp2021 3y ago
    > GP is doing nothing of the sort The poster wrote that «For AI, nothing in can mean garbage out». The poster seems to mean LLMs with «nothing in can mean garbage out». The poster seems to be calling LLMs AI. The poster seems to be attributing to AI properties of LLMs. Hence, the poster seems «to be identifying AI as if it were made of LLMs». > LLMs are a form of AI Prove it (or, defend it). I would say that it is arguable that they are not, unless one describes LLMs as "engines that reliably solve the problem of generating convincing text". The issue with that perspective is that «generating convincing text» is hardly per se a problem: it does not define a complete problem - text does not "stand alone" (content remains crucial). > artificial intelligence has been around since the 60s I know (and I should know decently well. Pedantically, a few years earlier - specifying just in case): what are you trying to say with that? Which application are you proposing to defend your perspective?
    - criddell 3y ago

visarga 3y ago

"Garbage in-garbage out", meet "Quantity has a quality of its own". LLMs are trained on a huge amount of text, this tends to compensate for garbage in. Of course if the model was not properly tested and fine-tuned, a LLM would happily execute any command.

dylan604 3y ago
Does it? Or does it just show that even in those quantities, there’s a preponderance of garbage? If it is sourcing places like Twitter, Reddit, and what not, then there’s most definitely a lot of garbage present.

FrustratedMonky 3y ago

>"garbage in equals garbage out" as is for humans

II2II 3y ago

How do we know it is GIGO? Plenty of people like to test edge cases. It is quite possible that someone fed the AI ingredients known to produce chlorine gas to see what would result. That is not to say that the company's excuse is valid. There should definiely be tests of the inputs to ensure they are safe (i.e. test for garbage in). The garbage out bit though would be more difficult seeming as that requires a knowledge of chemistry. Seeming as most of our knowledge of chemistry was founded.on experimentation, then later theory backed by experimentation, it is something that I wouldn't trust in the hands of an LLM.

michaelmior 3y ago
Testing the "garbage in" case seems as though it would be sufficient for the vast majority of use cases. Aside from undercooking ingredients which may cause foodborne illnesses, I think you're highly unlikely to create toxic chemical reactions using common cooking ingredients.

matwood 3y ago

Yep, it's literally the first thing kids do when they learn echo at the CLI. Make the computer write out curse words.

pkulak 3y ago

Garbage in should mean nothing out. If you ask a calculator what 2+2 is, but you mean 2+3, you still get the right answer, because you didn't put in garbage, you put in the wrong inputs. If you ask it what "pi + transubstantiation" is, it should fail, not confidently tell you "14".

bongobingo1 3y ago
> ”Forty-two,” said Deep Thought, with infinite majesty and calm.
kibwen 3y ago
The whole point of Babbage's classic anecdote is that it is possible to create garbage in the step where we encode our expectations into the commands of the machine. Even if the machine operates on its inputs properly, if we gave it inputs that did not properly encode our expectations, that's garbage, and the outcome will be garbage.

chasd00 3y ago

Does garbage in/out really apply to LLMs though? The relationship between input and output is unknown in large LLMs as far as I know . If that’s the case, then you can’t make any claim about “output” relating to “input” when it comes to LLM AI.

kibwen 3y ago
Even a hyper-rational mind with infinite computation capacity will draw the wrong conclusions if it is fed incorrect observations. LLMs aren't magic, they're pattern-matching engines. Even if the relationship between input and output is opaque, that doesn't mean they're omniscient.

antod 3y ago

Not so much "garbage in, garbage out" but "anything in, garbage out". It's just the garbage out is getting amazingly close to non garbage.