4 ms·

That's not a fitting analogy because there is no equivalent to the laws of thermodynamics that says a system can't be self improving. You're dismissing a techno

That's not a fitting analogy because there is no equivalent to the laws of thermodynamics that says a system can't be self improving. You're dismissing a technological possibility without much reason. We are extremely similar genetically to our ancestors of 100k years ago. The big difference is cultural inheritance. We come up with ideas and objects and pass them down. Humans have improved the capabilities of humans. There's no reason to think computer programs aren't capable of the same.

yreg 3y ago
We also have proven self improving AIs - GANs. Or even evolutionary algorithms.
- phreeza 3y ago
  I would not say GANs are self-improving in ways that vanilla NNs are not.
geraneum 3y ago
It’s not that straightforward. Quoting Yann LeCun from a LinkedIn post [1]: > I have claimed that Auto-Regressive LLMs are exponentially diverging diffusion processes. Here is the argument: Let e be the probability that any generated token exits the tree of "correct" answers. Then the probability that an answer of length n is correct is (1-e)^n. > Errors accumulate. The probability of correctness decreases exponentially. One can mitigate the problem by making e smaller (through training) but one simply cannot eliminate the problem entirely. A solution would require to make LLMs non auto-regressive while preserving their fluency. Moreover, I think the human analogy doesn’t quite fit here. There’s no evidence that today’s human is cognitively more capable than their human ancestors. It’s a conflation of technology and transference of knowledge with intelligence. 1: https://www.linkedin.com/posts/yann-lecun_i-have-claimed-that-auto-regressive-llms-activity-7045908925660950528-hJGk https://www.linkedin.com/posts/yann-lecun_i-have-claimed-tha...
- spi 3y ago
  I guess I should be wiser than contradicting LeCun on a public forum, but his math doesn't really work out. It only works if there is a unique correct answer to any question, in which case e=1/dict_size which is clearly false - even humans can just put "Let me think..." in front of anything else and get logically "the same" answer; after how many such filler words do you deem it "wrong"? Even taking colloquial speech aside, in Math, there is apparently a book that collects 367 different proofs of the Pythagorean Theorem; that tree of correct answers is certainly quite complicated. You can't approximate it with a fixed probability and take the exponential - the number of possibly correct next tokens varies very strongly depending on the previous string. LLMs and autoregression are very good at avoiding the 99.999[...]% of the strings that are simple gibberish. If you were to generate a string made of 20 random tokens from GPT2 tokenizer, you would get something like: "automakersGrand carries liberties Occupations ongoingOULDessing heartbeat Pillar intrigued Trotskymediatelyearable founding examinations lavAg redesign folds" and of course any half decent language model does much better than that. If the "paths to truth" were as unlikely as LeCun puts it there would be no hope. A non-autoregressive model would certainly be "better" because it would be faster, which is where language models started from (BERT & co.), it just doesn't seem to work as well... similarly to how a human sometimes needs to write something down and only realizes the correct answer to a complicated question on the go. If anything, we'd need to allow LLMs to realize mistakes and correct themselves out of them, i.e. making the generation non-linear. If you ask GPT4 something complicated (like math) it's not rare at all that it logically contradicts itself in their answers. I would be surprised if, somewhere deep in the model, it doesn't "realize" this, but it can't fix it, so it falls back to what humans do at an exam or interview that started badly: try to bullshit their way out of the thing, sweeping the inconsistency under the carpet, unless you explicitly point it to them (and often even after that, both GPT4 and humans). P.S. Mathematician rant: who on Earth calls a probability "e"??
- Cyphase 3y ago
  > There’s no evidence that today’s human is cognitively more capable than their human ancestors. It’s a conflation of technology and transference of knowledge with intelligence. That was their point. See here: > We are extremely similar genetically to our ancestors of 100k years ago. The big difference is cultural inheritance. We come up with ideas and objects and pass them down.
wg0 3y ago
Now you're falsely equating biological brains with giant arrays of floating point numbers. we have NOT evolved where we are today by generating inputs to ourselves, watching our own output and modifying our inputs to watch again our outputs. Rather - we had fundamental strong reasoning almost flawless rigorous (formally documented) capabilities from the get go. Interference, induction, deduction. LLMs have none of that and it's documented all over multiple times over. We have built at that on top of those faculties again - by not giving input to us and observing our outputs rather poking around into the world with are reasoning and cognitive faculties and arranging/categorising carefully what we discovered. So for fanboys, this surely is very huge and I respect that sentiment wholeheartedly. EDIT: Note about reasoning
- travisjungroth 3y ago
  > Rather - we had fundamental strong reasoning almost flawless rigorous (formally documented) capabilities from the get go. What are you talking about? This doesn't describe the vast majority of human knowledge. You're dismissing the possibility of new technology via a real loose analogy and insulting anyone who disagrees. This isn't what good reasoning looks like. GANs already exist. The linked project already exists. > A model will generate its own input and will watch it's own output and in process, will become more intelligent than it really is. If you sub out the needless contradiction of "more intelligent than it really is" for "more intelligent than it was", then this is something that already happens. It will continue to happen whether you believe in it or not. You're acting like you're arguing against people with sketches of perpetual motion devices. You're not. This is like arguing against heavier-than-air flight in 1905. You are way behind.
  - wg0 3y ago
    > What are you talking about? This doesn't describe the vast majority of human knowledge. I hope you don't mean to say that vast majority of knowledge is devoid of any consistent reasoning and is just hallucinated on the way as an LLM does. > It will continue to happen whether you believe in it or not. We'll see. I believe post transformers, next AI winter is around the corner - for a while. What I see above is an LLM chewing its own output with light modifications as input and that's not going to lead anywhere as the README of the project itself clearly notes. It gets stuck.

yreg 3y ago

We also have proven self improving AIs - GANs. Or even evolutionary algorithms.

phreeza 3y ago
I would not say GANs are self-improving in ways that vanilla NNs are not.

geraneum 3y ago

It’s not that straightforward. Quoting Yann LeCun from a LinkedIn post [1]: > I have claimed that Auto-Regressive LLMs are exponentially diverging diffusion processes. Here is the argument: Let e be the probability that any generated token exits the tree of "correct" answers. Then the probability that an answer of length n is correct is (1-e)^n. > Errors accumulate. The probability of correctness decreases exponentially. One can mitigate the problem by making e smaller (through training) but one simply cannot eliminate the problem entirely. A solution would require to make LLMs non auto-regressive while preserving their fluency. Moreover, I think the human analogy doesn’t quite fit here. There’s no evidence that today’s human is cognitively more capable than their human ancestors. It’s a conflation of technology and transference of knowledge with intelligence. 1: https://www.linkedin.com/posts/yann-lecun_i-have-claimed-that-auto-regressive-llms-activity-7045908925660950528-hJGk https://www.linkedin.com/posts/yann-lecun_i-have-claimed-tha...

spi 3y ago
I guess I should be wiser than contradicting LeCun on a public forum, but his math doesn't really work out. It only works if there is a unique correct answer to any question, in which case e=1/dict_size which is clearly false - even humans can just put "Let me think..." in front of anything else and get logically "the same" answer; after how many such filler words do you deem it "wrong"? Even taking colloquial speech aside, in Math, there is apparently a book that collects 367 different proofs of the Pythagorean Theorem; that tree of correct answers is certainly quite complicated. You can't approximate it with a fixed probability and take the exponential - the number of possibly correct next tokens varies very strongly depending on the previous string. LLMs and autoregression are very good at avoiding the 99.999[...]% of the strings that are simple gibberish. If you were to generate a string made of 20 random tokens from GPT2 tokenizer, you would get something like: "automakersGrand carries liberties Occupations ongoingOULDessing heartbeat Pillar intrigued Trotskymediatelyearable founding examinations lavAg redesign folds" and of course any half decent language model does much better than that. If the "paths to truth" were as unlikely as LeCun puts it there would be no hope. A non-autoregressive model would certainly be "better" because it would be faster, which is where language models started from (BERT & co.), it just doesn't seem to work as well... similarly to how a human sometimes needs to write something down and only realizes the correct answer to a complicated question on the go. If anything, we'd need to allow LLMs to realize mistakes and correct themselves out of them, i.e. making the generation non-linear. If you ask GPT4 something complicated (like math) it's not rare at all that it logically contradicts itself in their answers. I would be surprised if, somewhere deep in the model, it doesn't "realize" this, but it can't fix it, so it falls back to what humans do at an exam or interview that started badly: try to bullshit their way out of the thing, sweeping the inconsistency under the carpet, unless you explicitly point it to them (and often even after that, both GPT4 and humans). P.S. Mathematician rant: who on Earth calls a probability "e"??
Cyphase 3y ago
> There’s no evidence that today’s human is cognitively more capable than their human ancestors. It’s a conflation of technology and transference of knowledge with intelligence. That was their point. See here: > We are extremely similar genetically to our ancestors of 100k years ago. The big difference is cultural inheritance. We come up with ideas and objects and pass them down.

wg0 3y ago

Now you're falsely equating biological brains with giant arrays of floating point numbers. we have NOT evolved where we are today by generating inputs to ourselves, watching our own output and modifying our inputs to watch again our outputs. Rather - we had fundamental strong reasoning almost flawless rigorous (formally documented) capabilities from the get go. Interference, induction, deduction. LLMs have none of that and it's documented all over multiple times over. We have built at that on top of those faculties again - by not giving input to us and observing our outputs rather poking around into the world with are reasoning and cognitive faculties and arranging/categorising carefully what we discovered. So for fanboys, this surely is very huge and I respect that sentiment wholeheartedly. EDIT: Note about reasoning

travisjungroth 3y ago
> Rather - we had fundamental strong reasoning almost flawless rigorous (formally documented) capabilities from the get go. What are you talking about? This doesn't describe the vast majority of human knowledge. You're dismissing the possibility of new technology via a real loose analogy and insulting anyone who disagrees. This isn't what good reasoning looks like. GANs already exist. The linked project already exists. > A model will generate its own input and will watch it's own output and in process, will become more intelligent than it really is. If you sub out the needless contradiction of "more intelligent than it really is" for "more intelligent than it was", then this is something that already happens. It will continue to happen whether you believe in it or not. You're acting like you're arguing against people with sketches of perpetual motion devices. You're not. This is like arguing against heavier-than-air flight in 1905. You are way behind.
- wg0 3y ago
  > What are you talking about? This doesn't describe the vast majority of human knowledge. I hope you don't mean to say that vast majority of knowledge is devoid of any consistent reasoning and is just hallucinated on the way as an LLM does. > It will continue to happen whether you believe in it or not. We'll see. I believe post transformers, next AI winter is around the corner - for a while. What I see above is an LLM chewing its own output with light modifications as input and that's not going to lead anywhere as the README of the project itself clearly notes. It gets stuck.