6 ms·

What I find really entertaining is the "just predicting the next token" argument. If just predicting the next token can produce similar or better results than

What I find really entertaining is the "just predicting the next token" argument. If just predicting the next token can produce similar or better results than the almighty human intelligence on some tasks, then maybe there's a bit of hubris in how smart we think we actually are.

tines 3y ago
> If just predicting the next token can produce similar or better results than the almighty human intelligence on some tasks But it's not better than almighty human intelligence, it _is_ human intelligence, because it was trained on a mass of some of the best human intelligence in all recorded history (I say this because the good stuff like Aristotle got preserved while the garbage disappeared (this was true until the recent internet age, in which garbage survives as well as the gold)). > then maybe there's a bit of hubris in how smart we think we actually are I feel like you could say this if ChatGPT or whatever obtained its knowledge some other way than direct guidance from humans, but since we hand-fed it the answers, it falls a little flat for me. I'm open to persuasion.
- autokad 3y ago
  chatgpt doesnt just feed us back answers we already taught it. It learned relationships and semantics so it can apply that knowledge to do something novel. For instance, I took the basic of a dream and told it to turn it into a short story. The short story wasn't bad. I said make it more exciting, it updated the story such that one of the cars exploded. I guess chatgpt learned excitement from michael bay.
  - tines 3y ago
    (I'm going to be brusque for the sake of the argument, I very much could be wrong and I don't even know how much I believe of the argument I'm making.) > chatgpt doesnt just feed us back answers we already taught it True, there is some structure to the answers we already taught it that it statistically mimics as well. > It learned relationships and semantics so it can apply that knowledge to do something novel Can you provide an example of this novelty? I think we underestimate the depth and variety of things that humans have written about and put on the internet, and so while anything you ask ChatGPT to do might be outside of your own experience, it's highly likely that it's already been thought before and uploaded to the internet, and that ChatGPT is just parrotting back something to you that is very similar to what it has already seen. This effect of ChatGPT having so much more experience/training data than any single human being such that it can convince any single human that it is original is an interesting one. This is why I think, for example, that image generation will result in (a period of) "artistic inbreeding." Because there is so much that other humans have done that is outside of any individual's experience, we will accept e.g. Midjourney's output as something moving and original, when in reality it's just a slight variation on something that someone else has done before that we haven't seen. (Again apologies for any rudeness, I respect your opinion and experiences and am enjoying the conversation.)
    - autokad 3y ago
      I took the basics of a dream I had, and asked it to turn it into a short story. the result was pretty good. Is it using stuff already to seed its responses? sure, but thats what we do to. Nothing you do or say wasn't taught to you. But these are not simply parroting responses. I said this to chatgpt: I had a dream that me and my friend were in a car accident, and we had a choice in deciding how to use 1 hour. we could use it to live out our last final moments or spend that hour on trying to think of a solution to survive the wreck. turn this into a short story. As they sat trapped in their overturned car, Maria and her best friend Rachel had a dilemma. They only had one hour left to live and had to choose how to spend it. Rachel suggested they spend their final moments together, talking about all their favorite memories. But Maria had a different idea. She proposed they use the hour to think of a solution to survive the wreck. Rachel was hesitant at first, but ultimately agreed. For the next hour, they brainstormed every possible escape plan they could think of. They tried pushing against the doors, breaking the windows, and even attempting to signal for help. Just as they were about to give up hope, Maria remembered a tool in the trunk of the car. She crawled over to retrieve it and used it to pry open the door. Rachel followed her lead, and they finally escaped the car just in time. As they stumbled away from the wreck, both girls were in shock that they had made it out alive. They hugged each other tightly, grateful for the hour they spent trying to find a solution instead of giving up. From that moment on, they made a promise to never take a single moment for granted.
      - tines 3y ago
        > Nothing you do or say wasn't taught to you. If nothing we do or say wasn't taught to us then where did all human knowledge come from in the first place? This doesn't hold up. (Again, being direct for the sake of argument, please forgive any unkindness.)
        brycedriesenga 3y ago
        From our environment, genetics, and other people. We simply are able to take in more inputs (i.e. not just text) than LLMs.
- iliane5 3y ago
  > But it's not better than almighty human intelligence, it _is_ human intelligence, because it was trained on a mass of some of the best human intelligence in all recorded history Sure, I was saying "better" in the sense that if for X task, it can do better than Y% of humans. > since we hand-fed it the answers, it falls a little flat for me We didn't really hand-fed it any answers though did we? If you put a human in a white box all its life, with access to the entire dataset on a screen but no social interaction, nothing to see aside from the text, nothing to hear, nothing to feel, nothing to taste, etc, it'd be very impressed if they were then able to create answers that seem to display such thoughtful and complex understanding of the world.
  - simonh 3y ago
    I think the human would make a lot of the same fundamental errors LLMs make, for similar reasons. The level to which LLMs seem to understand the world is highly superficial because it is entirely linguistic. Also human written texts about the world and human affairs miss out huge swathes of contextual information that we safely assume actual humans have. LLMs don’t have any of that, which is why they fall flat on their faces in so many ways.
    - iliane5 3y ago
      Absolutely. What’s fascinating is that they’re getting such good understanding of many things through just text. Multimodal models that can process text, images, sounds, video, etc. are gonna be very interesting for that very reason
goldfeld 3y ago
[0]if we get a bit quantum (or God for some), then backtracking could happen by collapsing the dead-ends and "changing" history to stay with what turns out to be the solid plan. Could emergent conscience on AI's neurons do the planning and reasoning that it rather seems to be doing but ML experts will say it is not? If our conscience could by any chance reside not in the electrical currents of the wetware, could AI's reason also not reside in tokens? Is there some mysterious process possibly taking place and will philosophy probe it? 0: pasted from another thread
simonh 3y ago
I think it’s undeniable that LLMs encode knowledge, but the way they do so and what their answers imply, compared to what the same answer from a human would imply, are completely different. For example if a human explains the process for solving a mathematical problem, we know that person knows how to solve that problem. That’s not necessarily true of an LLM. They can give such explanations because they have been trained on many texts explaining those procedures, therefore they can generate texts of that form. However texts containing an actual mathematical problem and the workings for solving it are a completely different class of text for an LLM. The probabilistic token weightings for the maths text explanation don’t help at all. So yes these are fascinating, knowledgeable and even in some ways very intelligent systems. However it a radically different form of intelligence from us, in ways we find difficult to reason about.
- iliane5 3y ago
  Well it's like birds and airplanes. Do airplanes "fly" in the same sense that birds do? Of course not, birds flap their wings and airplanes need to be built, fueled and flown by humans. You could argue that the way birds fly is "more natural" or superior in some ways but I've yet to see a bird fly Mach 3. If you replace the analogy with humans and LLMs, LLMs won't ever reason or understand things in the same way we do, but if/when their output gets much smarter than us across the board, will it really matter?
  - simonh 3y ago
    I think the issue is there are good reasons to think LLMs architected and trained the way they are now can never approach human reasoning capability. That’s because the corpus of human written material is simply grossly inadequate to communicate or encode the knowledge necessary for that. Our written material assumes huge swathes of contextual knowledge, real world experience, and human lived experience that LLMs don’t and can’t have. At least architected and trained as they are now. Thats on top of the crippling inability LLMs have to generalise an ability to perform a task from the ability to generate a description of how to do the task. Plus many other similar limitations that would be inexplicable if displayed by a human. Of course LLMs aren’t the final word in AI development. I think they’re a vitally important step towards general AI, and we’ll get there eventually as we develop ever more capable architectures.
pegasus 3y ago
There's definitely hubris in how clever we consider ourselves. And encountering these AIs will hopefully bring a healthy adjustment there. But another manifestation of our hubris is the way we over-valorize our cleverness, making us feel oh so superior to other species, for example. Emotions, desires, agency, which we share with our animal cousins (and plants maybe also), but which software systems lack, are equally important to our life experience.
majormajor 3y ago
We've known for a long time that computers can do calculations far, far, far faster than us. We continue to figure out new ways to make those calculations do more complicated things faster than humans. What is intelligence beyond calculation is an ancient question, but not the one I'm most interested in at the moment, re: today's tools. I'm curious right now about if there's meaning to other people in human creation vs automation creation. E.g. is there a meaningful difference between an algorithm curating a feed of human-made TikTok videos and an algorithm both curating and creating a feed of human-made TikTok videos. Both qualitatively in terms of "would people engage with it to the same level" and quantitatively in terms of "how many new trends would emerge, how would they vary, how does that machine ecosystem of content generation behave compared to a human one" if you remove any human curation/training/feedback/nudging/etc from the flow beyond just "how many views/likes did you get?"
- iliane5 3y ago
  I think as soon as text2video gets really good (like midjourney level), there’s gonna be so much AI generated content that unless it’s all extremely good, human made content will be something people search specifically for. As for curation, I think the success of TikTok proves that you don’t need that much data to pretty preceding pinpoint what someone wants to watch (or what will get them to spend the most time on the app at least).
  - majormajor 3y ago
    Do you mean with humans generating the prompts or with some sort of no-human-in-the-loop "generate the text prompt to generate the video" automation? I think a super accessible animation tool would get a lot of use and result in a lot of cool stuff, but it's the latter that I'm really curious about in terms of how people interact with it.
opportune 3y ago
I don’t think there’s anything making it impossible for actual intelligence to arise from a task as simple as “predicting the next token (to model human thought/speech/writing)” because with enough compute resources, smart AI implementations, and training that task basically would be optimized by becoming a general intelligence. But it’s clear based on current implementations that once you work backwards from the knowledge that it’s “just predicting the next token” you can easily find situations in which the AI doesn’t demonstrate general intelligence. This is most obvious when it comes to math, but it’s also apparent in hallucinations and the model not being able to reason through/synthesize ideas very well, deviate from the script (instead of just answering a question with what it has already, in some cases it should not even try to answer and instead ask more clarifying questions). To be fair, there are plenty of humans with excellent writing or speaking skills that are bad at that kind of stuff too.
- simonh 3y ago
  The problem is such an approach is limited by the content of the training texts. As I mentioned elsewhere, our written texts assume huge swathes of contextual and experiential information and knowledge that LLMs don’t have. It’s possible some of it might be inferred from the texts, but not all of it by a long shot. If somehow you could generate a training text encoding a complete and thorough understanding of the physical world, human psychology and sociology, and reasoning then that might get you quite far. But the existing it even near future human textual corpus isn’t really that. Even then I still think you’d hit the limitations of the LLM cognitive architecture pretty hard.
  - giardini 3y ago
    simonh says >"The problem is such an approach is limited by the content of the training texts."< Aside: I would like to see ChatGPTs with distinct training texts, e.g., a ChatGPTs trained on the "great books" of Western philosophy and science knowledge up to the time of Victorian England.
feoren 3y ago
That'd be like saying that search engines are smarter than the almighty human intelligence because they know the capitals of every country while most humans don't. No, it just has access to a lot of data near-instantaneously. Just like GPT-4 does. It's the enormity of compiled human knowledge that is "smart" in GPT-4. It absolutely is "just predicting the next token", and it turns out that's enough to be an astoundingly intelligent-seeming system when trained on thousands of years of human knowledge. Of course it is! It's like in Avatar: The Last Airbender when he consults with his thousand past-lives at once for wisdom. GPT-4 lets us consult with the collective knowledge of humanity! It's absolutely amazing! And it's also "just predicting the next token". Those are both true.

tines 3y ago

> If just predicting the next token can produce similar or better results than the almighty human intelligence on some tasks But it's not better than almighty human intelligence, it _is_ human intelligence, because it was trained on a mass of some of the best human intelligence in all recorded history (I say this because the good stuff like Aristotle got preserved while the garbage disappeared (this was true until the recent internet age, in which garbage survives as well as the gold)). > then maybe there's a bit of hubris in how smart we think we actually are I feel like you could say this if ChatGPT or whatever obtained its knowledge some other way than direct guidance from humans, but since we hand-fed it the answers, it falls a little flat for me. I'm open to persuasion.

autokad 3y ago
chatgpt doesnt just feed us back answers we already taught it. It learned relationships and semantics so it can apply that knowledge to do something novel. For instance, I took the basic of a dream and told it to turn it into a short story. The short story wasn't bad. I said make it more exciting, it updated the story such that one of the cars exploded. I guess chatgpt learned excitement from michael bay.
- tines 3y ago
  (I'm going to be brusque for the sake of the argument, I very much could be wrong and I don't even know how much I believe of the argument I'm making.) > chatgpt doesnt just feed us back answers we already taught it True, there is some structure to the answers we already taught it that it statistically mimics as well. > It learned relationships and semantics so it can apply that knowledge to do something novel Can you provide an example of this novelty? I think we underestimate the depth and variety of things that humans have written about and put on the internet, and so while anything you ask ChatGPT to do might be outside of your own experience, it's highly likely that it's already been thought before and uploaded to the internet, and that ChatGPT is just parrotting back something to you that is very similar to what it has already seen. This effect of ChatGPT having so much more experience/training data than any single human being such that it can convince any single human that it is original is an interesting one. This is why I think, for example, that image generation will result in (a period of) "artistic inbreeding." Because there is so much that other humans have done that is outside of any individual's experience, we will accept e.g. Midjourney's output as something moving and original, when in reality it's just a slight variation on something that someone else has done before that we haven't seen. (Again apologies for any rudeness, I respect your opinion and experiences and am enjoying the conversation.)
  - autokad 3y ago
    I took the basics of a dream I had, and asked it to turn it into a short story. the result was pretty good. Is it using stuff already to seed its responses? sure, but thats what we do to. Nothing you do or say wasn't taught to you. But these are not simply parroting responses. I said this to chatgpt: I had a dream that me and my friend were in a car accident, and we had a choice in deciding how to use 1 hour. we could use it to live out our last final moments or spend that hour on trying to think of a solution to survive the wreck. turn this into a short story. As they sat trapped in their overturned car, Maria and her best friend Rachel had a dilemma. They only had one hour left to live and had to choose how to spend it. Rachel suggested they spend their final moments together, talking about all their favorite memories. But Maria had a different idea. She proposed they use the hour to think of a solution to survive the wreck. Rachel was hesitant at first, but ultimately agreed. For the next hour, they brainstormed every possible escape plan they could think of. They tried pushing against the doors, breaking the windows, and even attempting to signal for help. Just as they were about to give up hope, Maria remembered a tool in the trunk of the car. She crawled over to retrieve it and used it to pry open the door. Rachel followed her lead, and they finally escaped the car just in time. As they stumbled away from the wreck, both girls were in shock that they had made it out alive. They hugged each other tightly, grateful for the hour they spent trying to find a solution instead of giving up. From that moment on, they made a promise to never take a single moment for granted.
    - tines 3y ago
      > Nothing you do or say wasn't taught to you. If nothing we do or say wasn't taught to us then where did all human knowledge come from in the first place? This doesn't hold up. (Again, being direct for the sake of argument, please forgive any unkindness.)
      - brycedriesenga 3y ago
        From our environment, genetics, and other people. We simply are able to take in more inputs (i.e. not just text) than LLMs.
iliane5 3y ago
> But it's not better than almighty human intelligence, it _is_ human intelligence, because it was trained on a mass of some of the best human intelligence in all recorded history Sure, I was saying "better" in the sense that if for X task, it can do better than Y% of humans. > since we hand-fed it the answers, it falls a little flat for me We didn't really hand-fed it any answers though did we? If you put a human in a white box all its life, with access to the entire dataset on a screen but no social interaction, nothing to see aside from the text, nothing to hear, nothing to feel, nothing to taste, etc, it'd be very impressed if they were then able to create answers that seem to display such thoughtful and complex understanding of the world.
- simonh 3y ago
  I think the human would make a lot of the same fundamental errors LLMs make, for similar reasons. The level to which LLMs seem to understand the world is highly superficial because it is entirely linguistic. Also human written texts about the world and human affairs miss out huge swathes of contextual information that we safely assume actual humans have. LLMs don’t have any of that, which is why they fall flat on their faces in so many ways.
  - iliane5 3y ago
    Absolutely. What’s fascinating is that they’re getting such good understanding of many things through just text. Multimodal models that can process text, images, sounds, video, etc. are gonna be very interesting for that very reason

goldfeld 3y ago

[0]if we get a bit quantum (or God for some), then backtracking could happen by collapsing the dead-ends and "changing" history to stay with what turns out to be the solid plan. Could emergent conscience on AI's neurons do the planning and reasoning that it rather seems to be doing but ML experts will say it is not? If our conscience could by any chance reside not in the electrical currents of the wetware, could AI's reason also not reside in tokens? Is there some mysterious process possibly taking place and will philosophy probe it? 0: pasted from another thread

simonh 3y ago

I think it’s undeniable that LLMs encode knowledge, but the way they do so and what their answers imply, compared to what the same answer from a human would imply, are completely different. For example if a human explains the process for solving a mathematical problem, we know that person knows how to solve that problem. That’s not necessarily true of an LLM. They can give such explanations because they have been trained on many texts explaining those procedures, therefore they can generate texts of that form. However texts containing an actual mathematical problem and the workings for solving it are a completely different class of text for an LLM. The probabilistic token weightings for the maths text explanation don’t help at all. So yes these are fascinating, knowledgeable and even in some ways very intelligent systems. However it a radically different form of intelligence from us, in ways we find difficult to reason about.

iliane5 3y ago
Well it's like birds and airplanes. Do airplanes "fly" in the same sense that birds do? Of course not, birds flap their wings and airplanes need to be built, fueled and flown by humans. You could argue that the way birds fly is "more natural" or superior in some ways but I've yet to see a bird fly Mach 3. If you replace the analogy with humans and LLMs, LLMs won't ever reason or understand things in the same way we do, but if/when their output gets much smarter than us across the board, will it really matter?
- simonh 3y ago
  I think the issue is there are good reasons to think LLMs architected and trained the way they are now can never approach human reasoning capability. That’s because the corpus of human written material is simply grossly inadequate to communicate or encode the knowledge necessary for that. Our written material assumes huge swathes of contextual knowledge, real world experience, and human lived experience that LLMs don’t and can’t have. At least architected and trained as they are now. Thats on top of the crippling inability LLMs have to generalise an ability to perform a task from the ability to generate a description of how to do the task. Plus many other similar limitations that would be inexplicable if displayed by a human. Of course LLMs aren’t the final word in AI development. I think they’re a vitally important step towards general AI, and we’ll get there eventually as we develop ever more capable architectures.

pegasus 3y ago

There's definitely hubris in how clever we consider ourselves. And encountering these AIs will hopefully bring a healthy adjustment there. But another manifestation of our hubris is the way we over-valorize our cleverness, making us feel oh so superior to other species, for example. Emotions, desires, agency, which we share with our animal cousins (and plants maybe also), but which software systems lack, are equally important to our life experience.

majormajor 3y ago

We've known for a long time that computers can do calculations far, far, far faster than us. We continue to figure out new ways to make those calculations do more complicated things faster than humans. What is intelligence beyond calculation is an ancient question, but not the one I'm most interested in at the moment, re: today's tools. I'm curious right now about if there's meaning to other people in human creation vs automation creation. E.g. is there a meaningful difference between an algorithm curating a feed of human-made TikTok videos and an algorithm both curating and creating a feed of human-made TikTok videos. Both qualitatively in terms of "would people engage with it to the same level" and quantitatively in terms of "how many new trends would emerge, how would they vary, how does that machine ecosystem of content generation behave compared to a human one" if you remove any human curation/training/feedback/nudging/etc from the flow beyond just "how many views/likes did you get?"

iliane5 3y ago
I think as soon as text2video gets really good (like midjourney level), there’s gonna be so much AI generated content that unless it’s all extremely good, human made content will be something people search specifically for. As for curation, I think the success of TikTok proves that you don’t need that much data to pretty preceding pinpoint what someone wants to watch (or what will get them to spend the most time on the app at least).
- majormajor 3y ago
  Do you mean with humans generating the prompts or with some sort of no-human-in-the-loop "generate the text prompt to generate the video" automation? I think a super accessible animation tool would get a lot of use and result in a lot of cool stuff, but it's the latter that I'm really curious about in terms of how people interact with it.

opportune 3y ago

I don’t think there’s anything making it impossible for actual intelligence to arise from a task as simple as “predicting the next token (to model human thought/speech/writing)” because with enough compute resources, smart AI implementations, and training that task basically would be optimized by becoming a general intelligence. But it’s clear based on current implementations that once you work backwards from the knowledge that it’s “just predicting the next token” you can easily find situations in which the AI doesn’t demonstrate general intelligence. This is most obvious when it comes to math, but it’s also apparent in hallucinations and the model not being able to reason through/synthesize ideas very well, deviate from the script (instead of just answering a question with what it has already, in some cases it should not even try to answer and instead ask more clarifying questions). To be fair, there are plenty of humans with excellent writing or speaking skills that are bad at that kind of stuff too.

simonh 3y ago
The problem is such an approach is limited by the content of the training texts. As I mentioned elsewhere, our written texts assume huge swathes of contextual and experiential information and knowledge that LLMs don’t have. It’s possible some of it might be inferred from the texts, but not all of it by a long shot. If somehow you could generate a training text encoding a complete and thorough understanding of the physical world, human psychology and sociology, and reasoning then that might get you quite far. But the existing it even near future human textual corpus isn’t really that. Even then I still think you’d hit the limitations of the LLM cognitive architecture pretty hard.
- giardini 3y ago
  simonh says >"The problem is such an approach is limited by the content of the training texts."< Aside: I would like to see ChatGPTs with distinct training texts, e.g., a ChatGPTs trained on the "great books" of Western philosophy and science knowledge up to the time of Victorian England.

feoren 3y ago

That'd be like saying that search engines are smarter than the almighty human intelligence because they know the capitals of every country while most humans don't. No, it just has access to a lot of data near-instantaneously. Just like GPT-4 does. It's the enormity of compiled human knowledge that is "smart" in GPT-4. It absolutely is "just predicting the next token", and it turns out that's enough to be an astoundingly intelligent-seeming system when trained on thousands of years of human knowledge. Of course it is! It's like in Avatar: The Last Airbender when he consults with his thousand past-lives at once for wisdom. GPT-4 lets us consult with the collective knowledge of humanity! It's absolutely amazing! And it's also "just predicting the next token". Those are both true.