3 ms·

Averaging out bad output cannot produce good output. Or as the old saying goes, garbage in, garbage out.

jiggawatts 3y ago
It's not necessarily "garbage". A known limitation of word-at-a-time GPT models is that they can't backtrack, edit, or iterate on what they're doing. You can see these limitations when asking them to solve puzzles involving "term substitution". They can only do these up to a certain complexity, and then they run out of steam. However, if you run their output back through for a second round, they can continue substituting! Essentially, GPT models make mistakes, but can also recognise those mistakes and fix them on subsequent iterations. Much like humans. Another factor is that the "temperature" setting means that the outputs have an element to them that's purely random. Combining multiple outputs "anneals" out this randomness by converging on output that is the most consistent. Essentially, it's like asking a team of people to brainstorm, and then combining their partial solutions into a complete one. Once again, very human-like! This has been tested (semi) scientifically. Running GPT 4 across some puzzles or exams might yield a 60% success rate if you give it one chance. If you let it solve each problem by running it four times and then letting it combine its own outputs itself (not via human-selection!), then it will get up to an 80% success rate. See: https://www.youtube.com/watch?v=wVzuvf9D9BU https://www.youtube.com/watch?v=wVzuvf9D9BU
- namaria 3y ago
  There's nothing scientific about this. People are just holding a black box and saying "if I jiggle it like this, I can get better results". Approximating better results in simple contrived puzzles is hardly evidence that recycling the output makes it better in any absolute sense. Besides, translation being open ended and heavily context dependent will likely suffer from such recycling.
  - jiggawatts 3y ago
    They're taking objective measurements of experimental outcomes while varying aspects of the setup. How is that not scientific!? "If I jiggle this is behaves differently" is like... half of all science! At the end of the day, better is better, and I don't think that can be hand-waved away. That's like saying someone is not smarter because they just "studied harder" and that doesn't count somehow.
    - namaria 3y ago
      None of the papers are offering falsifiable premises and testing them. This is not doing science, this is merely assembling metadata. I find shameful that serious academics and institutions are indulging in this kind of number games. "Let's publish a bunch of measurements we took" might be rigorous and look like what advancing science would look like if you squint. But this is just gaming the publishing game.
      - jiggawatts 3y ago
        A decent chunk of biology was simply collecting bugs and pinning them on a board. A lot of science is just taking down measurements and recording the result. It's not always a giant leap, some grand insight, or a world-changing theorem.
        namaria 3y ago
        No, a large chunk of biology was figuring out species evolve and were not created as they are right now. Or that every living thing is made of cells that share the same basic chemistry and structure, not special kinds of tissue. The busywork in between might have looked like science and occupied a lot of time and space. Relativity and quantum mechanics generated a lot of falsifiable claims that were eventually tested. Yet we have been building larger and larger colliders to "collect and publish data" and the best outcome so far has been a job program for engineers (useful and good) and the Web (useful and good). But we shouldn't confuse collecting data and engineering with advancing science.

It's not necessarily "garbage". A known limitation of word-at-a-time GPT models is that they can't backtrack, edit, or iterate on what they're doing. You can see these limitations when asking them to solve puzzles involving "term substitution". They can only do these up to a certain complexity, and then they run out of steam. However, if you run their output back through for a second round, they can continue substituting! Essentially, GPT models make mistakes, but can also recognise those mistakes and fix them on subsequent iterations. Much like humans. Another factor is that the "temperature" setting means that the outputs have an element to them that's purely random. Combining multiple outputs "anneals" out this randomness by converging on output that is the most consistent. Essentially, it's like asking a team of people to brainstorm, and then combining their partial solutions into a complete one. Once again, very human-like! This has been tested (semi) scientifically. Running GPT 4 across some puzzles or exams might yield a 60% success rate if you give it one chance. If you let it solve each problem by running it four times and then letting it combine its own outputs itself (not via human-selection!), then it will get up to an 80% success rate. See: https://www.youtube.com/watch?v=wVzuvf9D9BU https://www.youtube.com/watch?v=wVzuvf9D9BU

namaria 3y ago
There's nothing scientific about this. People are just holding a black box and saying "if I jiggle it like this, I can get better results". Approximating better results in simple contrived puzzles is hardly evidence that recycling the output makes it better in any absolute sense. Besides, translation being open ended and heavily context dependent will likely suffer from such recycling.
- jiggawatts 3y ago
  They're taking objective measurements of experimental outcomes while varying aspects of the setup. How is that not scientific!? "If I jiggle this is behaves differently" is like... half of all science! At the end of the day, better is better, and I don't think that can be hand-waved away. That's like saying someone is not smarter because they just "studied harder" and that doesn't count somehow.
  - namaria 3y ago
    None of the papers are offering falsifiable premises and testing them. This is not doing science, this is merely assembling metadata. I find shameful that serious academics and institutions are indulging in this kind of number games. "Let's publish a bunch of measurements we took" might be rigorous and look like what advancing science would look like if you squint. But this is just gaming the publishing game.
    - jiggawatts 3y ago
      A decent chunk of biology was simply collecting bugs and pinning them on a board. A lot of science is just taking down measurements and recording the result. It's not always a giant leap, some grand insight, or a world-changing theorem.
      - namaria 3y ago
        No, a large chunk of biology was figuring out species evolve and were not created as they are right now. Or that every living thing is made of cells that share the same basic chemistry and structure, not special kinds of tissue. The busywork in between might have looked like science and occupied a lot of time and space. Relativity and quantum mechanics generated a lot of falsifiable claims that were eventually tested. Yet we have been building larger and larger colliders to "collect and publish data" and the best outcome so far has been a job program for engineers (useful and good) and the Web (useful and good). But we shouldn't confuse collecting data and engineering with advancing science.