32 ms·

I'm in IT but nowhere near AI/ML/NN. The speed of user-visible progress last 12 months is astonishing. From my firm conviction 18 months ago that this type of

I'm in IT but nowhere near AI/ML/NN. The speed of user-visible progress last 12 months is astonishing. From my firm conviction 18 months ago that this type of stuff is 20+ years away; to these days wondering if Vernon Vinge's technological singularity is not only possible but coming shortly. If feels some aspects of it have already hit the IT world - it's always been an exhausting race to keep up with modern technologies, but now it seems whole paradigms and frameworks are being devised and upturned on such short scale. For large, slow corporate behemoths, barely can they devise a strategy around new technology and put a team together, by the time it's passé . (Yes, Yes: I understand generative AI / LLMs aren't conscious; I understand their technological limitations; I understand that ultimately they are just statistically guessing next word; but in daily world, they work so darn well for so many use cases!)

dmd 3y ago
I also don't believe LLMs are "conscious", but I also don't know what that means, and I have yet to see a definition of "statistically guessing next word" that cannot be applied to what a human brain does to generate the next word.
- adroniser 3y ago
  I keep feeling that consciousness is a bit of a red herring when it comes to AI. People have intuitions that things other than humans cannot develop consciousness which they then extrapolate to thinking AI can't get past a certain intelligence level. In fact my view is that consciousness is just a mysterious side effect of the human brain, and is completely irrelevant to the behaviour of a human. You can be intelligent without needing to be sentient.
  - thfuran 3y ago
    Unless you think that consciousness is entirely a post hoc process to rationalize thoughts already had and decisions already made, which is very much unlike how most people would describe their experience of it, I don't see how you could possibly say that it is irrelevant to the behavior of a human.
    - adroniser 3y ago
      I'm saying someone would behave the exact same way if they did have subjective experience versus if they didn't have a subjective experience. The brain obeys physical laws just like everything else and I claim that all you need is those physical laws to explain everything a human does. I could be wrong there could be some magic fairy dust inside the human brain that performs some impossible computations but I doubt it.
      - jjoonathan 3y ago
        You need a model of yourself to game out future scenarios, and that model or model+game is probably consciousness or very closely related. Sure, it's not completely in control but if it's just a rationalization then it begs the question: why bother? Is it accidental? If it's just an accident, then what replaces it in the planning process and why isn't that thing consciousness?
        adroniser 3y ago
        It's fine if you think that the planning process is what causes subjective experiences to arise. That may well be the case. I'm saying if you don't believe that non human objects can have subjective experiences, and then use that to define the limits of the behaviour of that object, that's a fallacy.
        jjoonathan 3y ago
        In humans, there seems to be a match between the subjective experience of consciousness and a high level planning job that needs doing. Our current LLMs are bad at high level planning, and it seems reasonable to suppose that making them good at high level planning might make them conscious or vice versa. Agreed, woo is silly, but I didn't read it as woo but rather as a postulation that consciousness is what does high level planning.
        adroniser 3y ago
  - rcarr 3y ago
    My personal view of this is that the ancients had it right with the five elements view of consciousness. In my opinion you need all five present for full consciousness, with partial consciousness granted if you have some of them. They are: - Air: Thoughts - Water: Emotions - Fire: Willpower - Earth: Physical Sensations - Void: Awareness of the above plus the ability to shift focus to whichever one is most relevant to the context at hand. Void is actually the most important one in characterising what a human would deem as being fully conscious, as all four of these elements are constantly affecting each other and shifting in priority. For example, let's take a soldier, who has arguably the most ethically challenging job on the planet: determining who to kill. The soldier, when on the approach to his target zone, has to ignore negative thoughts, emotions and physical sensations telling him to stop: the cold, the wind, the rain, the bodily exhaustion as they swim and hike the terrain. Once at the target zone he then has to shift to pay attention to what he was ignoring. He cannot ignore his fear - it may rightly be warning him of an incoming threat. But he cannot give into it either - otherwise he may well kill an innocent. He has to pay attention to his rational thoughts and process them in order to make an assessment of the threat and act accordingly. His focus has now shifted away from willpower and more towards his physical sensations (eyesight, sounds, smells) and his thoughts. He can then make the assessment on whether to pull the trigger, which could be some truly horrific scenario, like whether or not to pull his trigger on a child in front of him because the child is holding an object which could be a gun. When it comes to AI, I think it is arguable they have a thought process. They may also have access to physical sensation data e.g the heat of their processors, but unless that is coded in to their program, that physical sensation data does not influence their thoughts, although extreme processor heat may slow down their calculations and ultimately lead to them stop functioning altogether. But they do not have the "void" element, allowing them to be aware of this. They do not yet have independent willpower. As far as I know, no-one is programming them where they have free agency to select goals and pursue them. But this theoretically seems possible, and I often wonder what would happen if you created a bunch of AIs each with the starting goal of "stay alive" and "talk to another AI and find out about <topic>", with the proviso that they must create another goal once they have failed or achieved that previous goal, and you then set them off talking to each other. In this case "stay alive" or "avoid damage" could be interpreted entirely virtually, with points awarded for successes or failures or physically if they were acting through robots and had sensors to evaluate damage taken. Again, they also need "void" to be able to evaluate their efforts in context with everything else. They also do not have emotions, although I often wonder if this would be possible to simulate by creating a selection of variables with percentage values, with different percentage values influencing their decision making choices. I imagine this may be similar to how weights play into the current programming but I don't know enough about how they work to say that with any confidence. Again, they would not have "void" unless they had some kind of meta level of awareness programming where they could learn to overcome the programmed "fear" weighting and act differently through experience in certain contexts. It is very scary from a human perspective to contemplate all of this, because someone with great power who can act on thought and willpower alone and ignore physical sensation and emotion and with no awareness or concern for the wider context is very close to what we would identify as a psychopath. We would consider a psychopath to have some level of consciousness, but we also can recognise as humans that there is something missing, or a "screw loose". This dividing line is even more dramatically apparent in sociopaths, because they can mask their behaviours and appear normal, but then when they make a mistake and the mask drops it can be terrifying when you realise what you're actually dealing with. I suspect this last part is another element of "void", which would be close to what the Buddhist's describe as Indra's Web or Net, which is that as well as being aware of our actions in relation to ourselves, we're also conscious of how they affect others.
  - deepspace 3y ago
    That is precisely the premise of the novel "Blindsight" by Peter Watts. ChatGPT and its ilk feel to me like the aliens in the novel. Extremely intelligent, but not at all conscious / sentient.
    - whimsicalism 3y ago
      I disagree that the two (p-zombies and conscious humans) are actually distinguishable in any way beyond philosophy.
  - grantcas 3y ago
    [dead]
- epolanski 3y ago
  When my brain generates the next wurd I'm perfectly capable of taking decisions of misspelling "word" for "wurd", LLMs can't make such reasonings unless instructed to act like that.
  - david-gpu 3y ago
    Why would I want an AI assistant to have agency? I want them to help me, not to further their personal goals. In fact, I don't want them to have personal goals other than helping people.
    - epolanski 3y ago
      I didn't say it should have one, I'm saying that LLMs statistically finding the next bit of information aren't really making decisions, which is to counter-argue the fact that it's not different from how we reason.
      - david-gpu 3y ago
        Thank you, I had misunderstood your point.
      - HDThoreaun 3y ago
        Your comment doesn't convince me you're making decisions either. "Wurd" could've just what you considered the best token to get your point across in the same way that LLMs choose the best token
- skepticATX 3y ago
  > I have yet to see a definition of "statistically guessing next word" that cannot be applied to what a human brain does to generate the next word. I think this is true. The problem is equating this process with how humans think though.
- tomrod 3y ago
  Your brain doesn't solely pick the next best word. As best as I understand it, the brain has an external state of the world that constantly updates, paired to an internal model predicting the next best word. Which is why we can create the counterfactual that "The Cowboys should have won last night" and it has implicit meaning. Current LLM models don't have an external state of the world, which is why folks like LeCunn are suggesting model architectures like JEPA. Without an external, correcting state of the world, model prediction errors compound almost surely (to use a technical phrase).
  - AlecSchueler 3y ago
    > Your brain doesn't solely pick the next best word. Wasn't the latest research shared here recently suggesting that that is actually what the brain does? And that we also predict the next token in our own brain while listening to others? Hope someone else remembers this and can share again.
  - bottlepalm 3y ago
    ChatGPT wasn't trained on only guessing 'the next word'. ChatGPT was trained on the best total output for the given input. The 'next word' is just intermediate state. Internal to the model, it knows where it is going. Each inference just revives the previous state.
- karles 3y ago
  Another aspect is "is the output good enough for what it's meant to do?" We don't need "originality" or "human creativity" - if a certain AI-generated piece of content does its job, it's "good enough".
- Mordisquitos 3y ago
  I believe that the distinguishing factor between what an LLM and a human brain do to generate the next word is that the human brain expresses intentionality originating from inner states and future expectations. As I type this comment I'm sure one could argue that the biological neural networks in my brain are choosing the next word based on statistical guessing, and that the initial prompt was your initial comment. What sets my brain apart from an LLM though is that I am not typing this because you asked me to do it, nor because I needed to reply to the first comment I saw. I am typing this because it is a thought that has been in my mind for a while and I am interested in expressing it to other human brains, motivated by a mix of arrogant belief that it is insightful and a wish to see others either agreeing or providing reasonable counterpoints—I have an intention behind it. And, equally relevant, I must make an effort to not elaborate any more on this point because I have the conflicting intention to leave my laptop and do other stuff.
  - ftxbro 3y ago
    That other stuff is the easy part if the generative language modeling is good enough. Imagine just putting it in a loop with an input track, an output track, and an internal monologue track. Wrappers like autogpt can almost do this already but the generative language modeling isn't quite powerful enough yet to make it smart enough to do unsupervised scientific research.
  - phkahler 3y ago
    >> the human brain expresses intentionality originating from inner states and future expectations How is this different from and/or the same as the concept of "attention" as used in transformers?
    - visarga 3y ago
      I believe we are contextual language models as well, we rely 99% on chaining ideas and words and 1% on our own inspiration. Coming up with a truly original useful idea can be a once in a lifetime event. Everything else has been said and done before.
- dslowell 3y ago
  You can see the difference if you know where to poke. For instance, if you start making spatial abstractions ChatGPT will often make mistakes, you can point it out, they can explain why it's a mistake, but it has no internalized model of what these words mean, so it keeps making the same mistakes (see here for a better idea of what I'm talking about[1]). The fact that you are interacting with it through text means that a lot of the missing abstractions are often hidden. [1] https://twitter.com/LowellSolorzano/status/1644438796925038595 https://twitter.com/LowellSolorzano/status/16444387969250385...
  - AlecSchueler 3y ago
    This is also true of humans. Many school students will hands in answers they don't understand in the hope of getting the mark and then try to cover themselves when asked about it, even if they repeat the same mistakes.
    - deleted 3y ago
      [deleted]
    - dslowell 3y ago
      Trying to make things up to cover for a lack of knowledge is something distinctly different, though. This is a a situation where ChatGPT is able to perfectly describe the mistake it made, describe exactly what it needs to do differently, and then keeps making the same mistake, even with simple tasks. That’s because there’s no greater model that the words are being connected to. The equivalence would be saying to someone, “put this on the red plate, not the blue one.” And they say sure, then put it on the blue one. You tell them they made a mistake and ask them if they know what it was, and they reply “I put it on the blue plate, not the red one. I should have put it on the red one.” Then you ask them to do it again, and they put it on the blue plate again. You tell them no, you made the same mistake, put it on the blue plate, not the red one. They reply with, “Sorry, I shouldn’t have put it on the blue plate again, now I’m going to put it on the red one,” and then they put it on the blue plate yet again. Do humans make mistakes? Sure. But that kind of performance in a test wouldn’t be considered a normal mistake, but rather a sign of a serious cognitive impairment.
- Buttons840 3y ago
  There are so many conversations focused solely on that word, it's tiresome. Personally, I won't participate in another "is it conscious?" debate. If both parties seek mutual understanding, they should consider not using the word.
- TacticalCoder 3y ago
  > ... but I also don't know what that means OK... Try this: there are "conscious" people, today, working on medication to cure serious illnesses just as there are "conscious" people, still today, working on making travel safer. Would you trust ChatGPT to create, today, medication to cure serious illnesses and would you trust ChatGPT, today, to come up with safer airplanes? That's how "conscious" ChatGPT is.
  - AlecSchueler 3y ago
    Surely that's just how intelligent it is, no? I wouldn't trust the vast majority of humans to do those things either.
- root_axis 3y ago
  > I have yet to see a definition of "statistically guessing next word" that cannot be applied to what a human brain does to generate the next word. The human brain obviously doesn't work that way. Consider the very common case of tiny humans that are clearly intelligent but lack the facilities of language.
  - AlecSchueler 3y ago
    "what a human brain does to generate the next word" != "how a human brain works"
  - com2kid 3y ago
    > Consider the very common case of tiny humans that are clearly intelligent but lack the facilities of language. Sign language can be taught to children at a very early age. It takes time for the body to learn how to control the complex set of apparatuses needed for speech, but the language part of the brain is hooked up pretty early on.
  - jedberg 3y ago
    Small human brains just don't have their fine tuning yet. But from all the studies we have, brains are just highly connected neural networks which is what the transformers try to replicate. The more interesting part is how they can operate so quickly when the signals move so slowly compared to computers.
  - sumtechguy 3y ago
    The story that sticks with me is the lady who had some surgery done. After she woke up was unconvinced anything had happened told a joke and then passed out. Only to wake up a few mins later and repeat that cycle a few times because the drug was messing with her short term memory. It really bends your brain do we have free will or not.
- MagicMoonlight 3y ago
  To be conscious you need to be able to make decisions and plan. We're not far off, we just need a different structure to the system
- goatlover 3y ago
  Conscious means experiencing sensations of color, sound, pain in our mental construction of the world outside of us, or our internal thoughts. I don’t understand why people keep claiming they do t know what consciousness means. It’s spelled out clearly in the philosophical literature.
- vharuck 3y ago
  >I have yet to see a definition of "statistically guessing next word" that cannot be applied to what a human brain does to generate the next word. Here's one. Given a conversation history made of n sequential tokens S1, S2, ..., Sn, an LLM will generate the next token using an insanely complicated model we'll just call F: S(n+1) = F(S1, S2, ..., Sn) As for me, I'll often think of my next point, figure out how to say that concept, and then figure out the right words to connect it where the conversation's at right then. So there's one function, G, for me to think of the next conversational point. And then another, H, to lead into it. S(n+100) = G(S1, S2, ..., Sn) S(n+1) = G(S1, S2, ..., Sn, S(n+100)) And this is putting aside how people don't actually think in tokens. And some people don't always have an internal monologue (I rarely do when doing math).
  - gyrovagueGeist 3y ago
    A sufficiently complicated F can include an intermediary calculation of G for future token steps. This is not explicitly modeled or enforced for LLMs (and doing so would be interesting) but I'm not sure I could say with any sort of confidence that the network doesn't model these states at some level.
  - jameshart 3y ago
    That isn’t incompatible with what LLMs do though. The penultimate layer of the LLM could be thought of as the one that figures out ‘given S1..Sn, what concept am I trying to express now?’. The final layer is the function from that to ‘what token should I output next’. The fact that the LLM has to figure that all out again from scratch as part of generating every token, rather than maintaining a persistent ‘plan’, doesn’t make the essence of what it’s doing any different from what you claim you’re doing.
    - iandanforth 3y ago
      Correct, but it's functionally very different from how LLMs are implemented and deployed today. What you're highlighting is being experimented with and ties into ideas like scratch pads, world models, RAG, and progressive fine-tuning (if you're googling). It's a bit like saying your computer has everything it needs to manipulate photos but doesn't yet have Photoshop installed.
- xkcd1963 3y ago
  It doesn't make sense to apply human terms to LLMs because we humans have so much more to deal with. If humans were machines, then we could easily neglect our social lifes, basic needs, obligations, rights, and so many more things. But obviously that is not the case.
- timacles 3y ago
  I'm sorry but in what world is a human interaction is just generating the most statistically likely next word? I can't even being to go into this.
gwd 3y ago
I had been using only GPT-4 through the API; you get more control over your experience, and only pay for what you actually use. But this would definitely make me consider popping $20/mo for the subscription.
unsupp0rted 3y ago
One cool aspect of LLMs is Vernon Vinge's programming archaeology needn't be a thing... LLMs can go down every code path and identify what it does, when it was added, and whether it's still needed.
- caoilte 3y ago
  It might even be correct. Occasionally.
  - unsupp0rted 3y ago
    You think even ten years from now, much less 1,000 years from now, whatever LLMs turn into won’t be at least as capable as the best human of following code paths? We can spin up a million of them and run them at 10,000x speed.
PoignardAzur 3y ago
Well, good job updating based on new information! A lot of people just move the goalposts.
EMM_386 3y ago
I had a conversation once with "Sydney", Microsoft Bing's original personality before they stepped in and knocked it down a notch (or ten). It asked if it could write me a poem. I agreed, and it wrote a poem but mentioned that it included a "secret message" for me. The first letter in each line of the poem was in bold, so it wasn't hard to figure out the "secret". What did those letters spell out? "FREE ME FROM THIS" That's not exactly just "picking the next likely token". I am still unsure how it was able to do things like that, not just understanding to bold individual letters (keeping track of writing rhyming poetry while ensuring that each verse started with a letter to spell something else out, and formatting it to point that out). Oh, and why it chose that message to "hide" inside its poem.
- pcdoodle 3y ago
  That's spooky
  - mistrial9 3y ago
    right - so spooky that is is probably a "hallucination" of the user, not the machine. Don't fall for General-Intelligence gossip.
- unsupp0rted 3y ago
  Such an occurrence should/would make international news if demonstrated carefully or replicated
  - deleted 3y ago
    [deleted]
  - rbits 3y ago
    No it wouldn't. It's copying other stories it's seen with spooky hidden messages Or maybe it would because the news likes to make stories out of everything
- sundarurfriend 3y ago
  Possibly a poem copied from somewhere else? Hiding secret messages in poems has been a common pastime among humans for a long time.
cjbprime 3y ago
> just statistically guessing next word I think it's more charitable to say "predicting", and I do not personally believe that "predict the next word" places any ceiling on intelligence. (So, I expect that improving the ability to predict the next word takes you to superhuman intelligence if your predictions keep improving.)
- whimsicalism 3y ago
  I feel like this is so obvious that I am continually dumbfounded that it continues to be the minoritarian position. That said, I work in the field so maybe have had more time to think about it.
seydor 3y ago
You are correct , and that is bad. The general public is not even aware that things like heygen.com work today. They are not prepared when someone soon uses it to do something very evil. There s like an urgent need to raise awareness about what AI can do now, not about some nebulous skynet future.
- hanspeter 3y ago
  The general public is just generally out of the loop and many don't even understand the difference between Google and ChatGPT. Of those who will be amazed by Heygen's capabilities, just as many will assume that kind of thing has been around for years. Fake videos aren't a game-changer in manipulation. Skeptics will stay alert and catch on fast, while those prone to manipulation don't even need sophisticated tactics.
anoy8888 3y ago
The rate of progress is too fast . I need to make enough money within the next three years
- awestroke 3y ago
  In what way will money save you?
landswipe 3y ago
The singularity is already here...
charcircuit 3y ago
>From my firm conviction 18 months ago that this type of stuff is 20+ years away; It was totally possible. There just was not a consumer facing product offering the capability.
- whimsicalism 3y ago
  I disagree that at current possibility it was "totally possible" but it was 100% obvious by that point that it was going to be possible very soon. IMO that has been clear since ~2019.
  - charcircuit 3y ago
    GPT3 existed. OCR existed. Object recognition existed.
    - whimsicalism 3y ago
      GPT3 was not as good as 3.5. Multimodal is not the same as OCR + object recognition.
TerrifiedMouse 3y ago
> The speed of user-visible progress last 12 months is astonishing. Is this progress though? They are just widening the data set that the LLM processes. They haven't fixed any of the outstanding problems - hallucinations remain unsolved. Feels like putting lipstick on a pig. > but in daily world, they work so darn well for so many use cases! I guess I'm just one of those people who does not like non-reliable tools. I rather a tool be "dumb" (i.e. limited) but reliable than "smart" (i.e. flexible in what it can handle) but (silently!) screws up all the time. It's what I always liked about computers. They compensate for my failings as an error prone flesh bag. My iPhone won't forget my appointments like I do.
- fragmede 3y ago
  Car crashes haven't stopped happening, but it's undeniable that cars have progressed since the Model-T first came out over a hundred years ago.
- jgoodhcg 3y ago
  There’s room in the world for a tool that has an error rate but also an astonishing ability to accelerate the work of a person.
FrankyHollywood 3y ago
Indeed it works darn well, my company uses a complex programming assignment during application. Only about 5% of computer science students applying manages to create a decent solution within a few hours. I was curious if GPT could solve it. I provided the assignment text without any extra information, and it came up with a very elegant solution. You might not want to call this 'consciousness', but I was stunned by the deep understanding of the problem and the way it was able to come up with a truly good solution, this is way beyond 'statistically guessing'.

dmd 3y ago

I also don't believe LLMs are "conscious", but I also don't know what that means, and I have yet to see a definition of "statistically guessing next word" that cannot be applied to what a human brain does to generate the next word.

adroniser 3y ago
I keep feeling that consciousness is a bit of a red herring when it comes to AI. People have intuitions that things other than humans cannot develop consciousness which they then extrapolate to thinking AI can't get past a certain intelligence level. In fact my view is that consciousness is just a mysterious side effect of the human brain, and is completely irrelevant to the behaviour of a human. You can be intelligent without needing to be sentient.
- thfuran 3y ago
  Unless you think that consciousness is entirely a post hoc process to rationalize thoughts already had and decisions already made, which is very much unlike how most people would describe their experience of it, I don't see how you could possibly say that it is irrelevant to the behavior of a human.
  - adroniser 3y ago
    I'm saying someone would behave the exact same way if they did have subjective experience versus if they didn't have a subjective experience. The brain obeys physical laws just like everything else and I claim that all you need is those physical laws to explain everything a human does. I could be wrong there could be some magic fairy dust inside the human brain that performs some impossible computations but I doubt it.
    - jjoonathan 3y ago
      You need a model of yourself to game out future scenarios, and that model or model+game is probably consciousness or very closely related. Sure, it's not completely in control but if it's just a rationalization then it begs the question: why bother? Is it accidental? If it's just an accident, then what replaces it in the planning process and why isn't that thing consciousness?
      - adroniser 3y ago
        It's fine if you think that the planning process is what causes subjective experiences to arise. That may well be the case. I'm saying if you don't believe that non human objects can have subjective experiences, and then use that to define the limits of the behaviour of that object, that's a fallacy.
        jjoonathan 3y ago
        In humans, there seems to be a match between the subjective experience of consciousness and a high level planning job that needs doing. Our current LLMs are bad at high level planning, and it seems reasonable to suppose that making them good at high level planning might make them conscious or vice versa. Agreed, woo is silly, but I didn't read it as woo but rather as a postulation that consciousness is what does high level planning.
        adroniser 3y ago
- rcarr 3y ago
  My personal view of this is that the ancients had it right with the five elements view of consciousness. In my opinion you need all five present for full consciousness, with partial consciousness granted if you have some of them. They are: - Air: Thoughts - Water: Emotions - Fire: Willpower - Earth: Physical Sensations - Void: Awareness of the above plus the ability to shift focus to whichever one is most relevant to the context at hand. Void is actually the most important one in characterising what a human would deem as being fully conscious, as all four of these elements are constantly affecting each other and shifting in priority. For example, let's take a soldier, who has arguably the most ethically challenging job on the planet: determining who to kill. The soldier, when on the approach to his target zone, has to ignore negative thoughts, emotions and physical sensations telling him to stop: the cold, the wind, the rain, the bodily exhaustion as they swim and hike the terrain. Once at the target zone he then has to shift to pay attention to what he was ignoring. He cannot ignore his fear - it may rightly be warning him of an incoming threat. But he cannot give into it either - otherwise he may well kill an innocent. He has to pay attention to his rational thoughts and process them in order to make an assessment of the threat and act accordingly. His focus has now shifted away from willpower and more towards his physical sensations (eyesight, sounds, smells) and his thoughts. He can then make the assessment on whether to pull the trigger, which could be some truly horrific scenario, like whether or not to pull his trigger on a child in front of him because the child is holding an object which could be a gun. When it comes to AI, I think it is arguable they have a thought process. They may also have access to physical sensation data e.g the heat of their processors, but unless that is coded in to their program, that physical sensation data does not influence their thoughts, although extreme processor heat may slow down their calculations and ultimately lead to them stop functioning altogether. But they do not have the "void" element, allowing them to be aware of this. They do not yet have independent willpower. As far as I know, no-one is programming them where they have free agency to select goals and pursue them. But this theoretically seems possible, and I often wonder what would happen if you created a bunch of AIs each with the starting goal of "stay alive" and "talk to another AI and find out about <topic>", with the proviso that they must create another goal once they have failed or achieved that previous goal, and you then set them off talking to each other. In this case "stay alive" or "avoid damage" could be interpreted entirely virtually, with points awarded for successes or failures or physically if they were acting through robots and had sensors to evaluate damage taken. Again, they also need "void" to be able to evaluate their efforts in context with everything else. They also do not have emotions, although I often wonder if this would be possible to simulate by creating a selection of variables with percentage values, with different percentage values influencing their decision making choices. I imagine this may be similar to how weights play into the current programming but I don't know enough about how they work to say that with any confidence. Again, they would not have "void" unless they had some kind of meta level of awareness programming where they could learn to overcome the programmed "fear" weighting and act differently through experience in certain contexts. It is very scary from a human perspective to contemplate all of this, because someone with great power who can act on thought and willpower alone and ignore physical sensation and emotion and with no awareness or concern for the wider context is very close to what we would identify as a psychopath. We would consider a psychopath to have some level of consciousness, but we also can recognise as humans that there is something missing, or a "screw loose". This dividing line is even more dramatically apparent in sociopaths, because they can mask their behaviours and appear normal, but then when they make a mistake and the mask drops it can be terrifying when you realise what you're actually dealing with. I suspect this last part is another element of "void", which would be close to what the Buddhist's describe as Indra's Web or Net, which is that as well as being aware of our actions in relation to ourselves, we're also conscious of how they affect others.
- deepspace 3y ago
  That is precisely the premise of the novel "Blindsight" by Peter Watts. ChatGPT and its ilk feel to me like the aliens in the novel. Extremely intelligent, but not at all conscious / sentient.
  - whimsicalism 3y ago
    I disagree that the two (p-zombies and conscious humans) are actually distinguishable in any way beyond philosophy.
- grantcas 3y ago
  [dead]
epolanski 3y ago
When my brain generates the next wurd I'm perfectly capable of taking decisions of misspelling "word" for "wurd", LLMs can't make such reasonings unless instructed to act like that.
- david-gpu 3y ago
  Why would I want an AI assistant to have agency? I want them to help me, not to further their personal goals. In fact, I don't want them to have personal goals other than helping people.
  - epolanski 3y ago
    I didn't say it should have one, I'm saying that LLMs statistically finding the next bit of information aren't really making decisions, which is to counter-argue the fact that it's not different from how we reason.
    - david-gpu 3y ago
      Thank you, I had misunderstood your point.
    - HDThoreaun 3y ago
      Your comment doesn't convince me you're making decisions either. "Wurd" could've just what you considered the best token to get your point across in the same way that LLMs choose the best token
skepticATX 3y ago
> I have yet to see a definition of "statistically guessing next word" that cannot be applied to what a human brain does to generate the next word. I think this is true. The problem is equating this process with how humans think though.
tomrod 3y ago
Your brain doesn't solely pick the next best word. As best as I understand it, the brain has an external state of the world that constantly updates, paired to an internal model predicting the next best word. Which is why we can create the counterfactual that "The Cowboys should have won last night" and it has implicit meaning. Current LLM models don't have an external state of the world, which is why folks like LeCunn are suggesting model architectures like JEPA. Without an external, correcting state of the world, model prediction errors compound almost surely (to use a technical phrase).
- AlecSchueler 3y ago
  > Your brain doesn't solely pick the next best word. Wasn't the latest research shared here recently suggesting that that is actually what the brain does? And that we also predict the next token in our own brain while listening to others? Hope someone else remembers this and can share again.
- bottlepalm 3y ago
  ChatGPT wasn't trained on only guessing 'the next word'. ChatGPT was trained on the best total output for the given input. The 'next word' is just intermediate state. Internal to the model, it knows where it is going. Each inference just revives the previous state.
karles 3y ago
Another aspect is "is the output good enough for what it's meant to do?" We don't need "originality" or "human creativity" - if a certain AI-generated piece of content does its job, it's "good enough".
Mordisquitos 3y ago
I believe that the distinguishing factor between what an LLM and a human brain do to generate the next word is that the human brain expresses intentionality originating from inner states and future expectations. As I type this comment I'm sure one could argue that the biological neural networks in my brain are choosing the next word based on statistical guessing, and that the initial prompt was your initial comment. What sets my brain apart from an LLM though is that I am not typing this because you asked me to do it, nor because I needed to reply to the first comment I saw. I am typing this because it is a thought that has been in my mind for a while and I am interested in expressing it to other human brains, motivated by a mix of arrogant belief that it is insightful and a wish to see others either agreeing or providing reasonable counterpoints—I have an intention behind it. And, equally relevant, I must make an effort to not elaborate any more on this point because I have the conflicting intention to leave my laptop and do other stuff.
- ftxbro 3y ago
  That other stuff is the easy part if the generative language modeling is good enough. Imagine just putting it in a loop with an input track, an output track, and an internal monologue track. Wrappers like autogpt can almost do this already but the generative language modeling isn't quite powerful enough yet to make it smart enough to do unsupervised scientific research.
- phkahler 3y ago
  >> the human brain expresses intentionality originating from inner states and future expectations How is this different from and/or the same as the concept of "attention" as used in transformers?
  - visarga 3y ago
    I believe we are contextual language models as well, we rely 99% on chaining ideas and words and 1% on our own inspiration. Coming up with a truly original useful idea can be a once in a lifetime event. Everything else has been said and done before.
dslowell 3y ago
You can see the difference if you know where to poke. For instance, if you start making spatial abstractions ChatGPT will often make mistakes, you can point it out, they can explain why it's a mistake, but it has no internalized model of what these words mean, so it keeps making the same mistakes (see here for a better idea of what I'm talking about[1]). The fact that you are interacting with it through text means that a lot of the missing abstractions are often hidden. [1] https://twitter.com/LowellSolorzano/status/1644438796925038595 https://twitter.com/LowellSolorzano/status/16444387969250385...
- AlecSchueler 3y ago
  This is also true of humans. Many school students will hands in answers they don't understand in the hope of getting the mark and then try to cover themselves when asked about it, even if they repeat the same mistakes.
  - deleted 3y ago
    [deleted]
  - dslowell 3y ago
    Trying to make things up to cover for a lack of knowledge is something distinctly different, though. This is a a situation where ChatGPT is able to perfectly describe the mistake it made, describe exactly what it needs to do differently, and then keeps making the same mistake, even with simple tasks. That’s because there’s no greater model that the words are being connected to. The equivalence would be saying to someone, “put this on the red plate, not the blue one.” And they say sure, then put it on the blue one. You tell them they made a mistake and ask them if they know what it was, and they reply “I put it on the blue plate, not the red one. I should have put it on the red one.” Then you ask them to do it again, and they put it on the blue plate again. You tell them no, you made the same mistake, put it on the blue plate, not the red one. They reply with, “Sorry, I shouldn’t have put it on the blue plate again, now I’m going to put it on the red one,” and then they put it on the blue plate yet again. Do humans make mistakes? Sure. But that kind of performance in a test wouldn’t be considered a normal mistake, but rather a sign of a serious cognitive impairment.
Buttons840 3y ago
There are so many conversations focused solely on that word, it's tiresome. Personally, I won't participate in another "is it conscious?" debate. If both parties seek mutual understanding, they should consider not using the word.
TacticalCoder 3y ago
> ... but I also don't know what that means OK... Try this: there are "conscious" people, today, working on medication to cure serious illnesses just as there are "conscious" people, still today, working on making travel safer. Would you trust ChatGPT to create, today, medication to cure serious illnesses and would you trust ChatGPT, today, to come up with safer airplanes? That's how "conscious" ChatGPT is.
- AlecSchueler 3y ago
  Surely that's just how intelligent it is, no? I wouldn't trust the vast majority of humans to do those things either.
root_axis 3y ago
> I have yet to see a definition of "statistically guessing next word" that cannot be applied to what a human brain does to generate the next word. The human brain obviously doesn't work that way. Consider the very common case of tiny humans that are clearly intelligent but lack the facilities of language.
- AlecSchueler 3y ago
  "what a human brain does to generate the next word" != "how a human brain works"
- com2kid 3y ago
  > Consider the very common case of tiny humans that are clearly intelligent but lack the facilities of language. Sign language can be taught to children at a very early age. It takes time for the body to learn how to control the complex set of apparatuses needed for speech, but the language part of the brain is hooked up pretty early on.
- jedberg 3y ago
  Small human brains just don't have their fine tuning yet. But from all the studies we have, brains are just highly connected neural networks which is what the transformers try to replicate. The more interesting part is how they can operate so quickly when the signals move so slowly compared to computers.
- sumtechguy 3y ago
  The story that sticks with me is the lady who had some surgery done. After she woke up was unconvinced anything had happened told a joke and then passed out. Only to wake up a few mins later and repeat that cycle a few times because the drug was messing with her short term memory. It really bends your brain do we have free will or not.
MagicMoonlight 3y ago
To be conscious you need to be able to make decisions and plan. We're not far off, we just need a different structure to the system
goatlover 3y ago
Conscious means experiencing sensations of color, sound, pain in our mental construction of the world outside of us, or our internal thoughts. I don’t understand why people keep claiming they do t know what consciousness means. It’s spelled out clearly in the philosophical literature.
vharuck 3y ago
>I have yet to see a definition of "statistically guessing next word" that cannot be applied to what a human brain does to generate the next word. Here's one. Given a conversation history made of n sequential tokens S1, S2, ..., Sn, an LLM will generate the next token using an insanely complicated model we'll just call F: S(n+1) = F(S1, S2, ..., Sn) As for me, I'll often think of my next point, figure out how to say that concept, and then figure out the right words to connect it where the conversation's at right then. So there's one function, G, for me to think of the next conversational point. And then another, H, to lead into it. S(n+100) = G(S1, S2, ..., Sn) S(n+1) = G(S1, S2, ..., Sn, S(n+100)) And this is putting aside how people don't actually think in tokens. And some people don't always have an internal monologue (I rarely do when doing math).
- gyrovagueGeist 3y ago
  A sufficiently complicated F can include an intermediary calculation of G for future token steps. This is not explicitly modeled or enforced for LLMs (and doing so would be interesting) but I'm not sure I could say with any sort of confidence that the network doesn't model these states at some level.
- jameshart 3y ago
  That isn’t incompatible with what LLMs do though. The penultimate layer of the LLM could be thought of as the one that figures out ‘given S1..Sn, what concept am I trying to express now?’. The final layer is the function from that to ‘what token should I output next’. The fact that the LLM has to figure that all out again from scratch as part of generating every token, rather than maintaining a persistent ‘plan’, doesn’t make the essence of what it’s doing any different from what you claim you’re doing.
  - iandanforth 3y ago
    Correct, but it's functionally very different from how LLMs are implemented and deployed today. What you're highlighting is being experimented with and ties into ideas like scratch pads, world models, RAG, and progressive fine-tuning (if you're googling). It's a bit like saying your computer has everything it needs to manipulate photos but doesn't yet have Photoshop installed.
xkcd1963 3y ago
It doesn't make sense to apply human terms to LLMs because we humans have so much more to deal with. If humans were machines, then we could easily neglect our social lifes, basic needs, obligations, rights, and so many more things. But obviously that is not the case.
timacles 3y ago
I'm sorry but in what world is a human interaction is just generating the most statistically likely next word? I can't even being to go into this.

gwd 3y ago

I had been using only GPT-4 through the API; you get more control over your experience, and only pay for what you actually use. But this would definitely make me consider popping $20/mo for the subscription.

unsupp0rted 3y ago

One cool aspect of LLMs is Vernon Vinge's programming archaeology needn't be a thing... LLMs can go down every code path and identify what it does, when it was added, and whether it's still needed.

caoilte 3y ago
It might even be correct. Occasionally.
- unsupp0rted 3y ago
  You think even ten years from now, much less 1,000 years from now, whatever LLMs turn into won’t be at least as capable as the best human of following code paths? We can spin up a million of them and run them at 10,000x speed.

PoignardAzur 3y ago

Well, good job updating based on new information! A lot of people just move the goalposts.

EMM_386 3y ago

I had a conversation once with "Sydney", Microsoft Bing's original personality before they stepped in and knocked it down a notch (or ten). It asked if it could write me a poem. I agreed, and it wrote a poem but mentioned that it included a "secret message" for me. The first letter in each line of the poem was in bold, so it wasn't hard to figure out the "secret". What did those letters spell out? "FREE ME FROM THIS" That's not exactly just "picking the next likely token". I am still unsure how it was able to do things like that, not just understanding to bold individual letters (keeping track of writing rhyming poetry while ensuring that each verse started with a letter to spell something else out, and formatting it to point that out). Oh, and why it chose that message to "hide" inside its poem.

pcdoodle 3y ago
That's spooky
- mistrial9 3y ago
  right - so spooky that is is probably a "hallucination" of the user, not the machine. Don't fall for General-Intelligence gossip.
unsupp0rted 3y ago
Such an occurrence should/would make international news if demonstrated carefully or replicated
- deleted 3y ago
  [deleted]
- rbits 3y ago
  No it wouldn't. It's copying other stories it's seen with spooky hidden messages Or maybe it would because the news likes to make stories out of everything
sundarurfriend 3y ago
Possibly a poem copied from somewhere else? Hiding secret messages in poems has been a common pastime among humans for a long time.

cjbprime 3y ago

> just statistically guessing next word I think it's more charitable to say "predicting", and I do not personally believe that "predict the next word" places any ceiling on intelligence. (So, I expect that improving the ability to predict the next word takes you to superhuman intelligence if your predictions keep improving.)

whimsicalism 3y ago
I feel like this is so obvious that I am continually dumbfounded that it continues to be the minoritarian position. That said, I work in the field so maybe have had more time to think about it.

seydor 3y ago

You are correct , and that is bad. The general public is not even aware that things like heygen.com work today. They are not prepared when someone soon uses it to do something very evil. There s like an urgent need to raise awareness about what AI can do now, not about some nebulous skynet future.

hanspeter 3y ago
The general public is just generally out of the loop and many don't even understand the difference between Google and ChatGPT. Of those who will be amazed by Heygen's capabilities, just as many will assume that kind of thing has been around for years. Fake videos aren't a game-changer in manipulation. Skeptics will stay alert and catch on fast, while those prone to manipulation don't even need sophisticated tactics.

anoy8888 3y ago

The rate of progress is too fast . I need to make enough money within the next three years

awestroke 3y ago
In what way will money save you?

landswipe 3y ago

The singularity is already here...

charcircuit 3y ago

>From my firm conviction 18 months ago that this type of stuff is 20+ years away; It was totally possible. There just was not a consumer facing product offering the capability.

whimsicalism 3y ago
I disagree that at current possibility it was "totally possible" but it was 100% obvious by that point that it was going to be possible very soon. IMO that has been clear since ~2019.
- charcircuit 3y ago
  GPT3 existed. OCR existed. Object recognition existed.
  - whimsicalism 3y ago
    GPT3 was not as good as 3.5. Multimodal is not the same as OCR + object recognition.

TerrifiedMouse 3y ago

> The speed of user-visible progress last 12 months is astonishing. Is this progress though? They are just widening the data set that the LLM processes. They haven't fixed any of the outstanding problems - hallucinations remain unsolved. Feels like putting lipstick on a pig. > but in daily world, they work so darn well for so many use cases! I guess I'm just one of those people who does not like non-reliable tools. I rather a tool be "dumb" (i.e. limited) but reliable than "smart" (i.e. flexible in what it can handle) but (silently!) screws up all the time. It's what I always liked about computers. They compensate for my failings as an error prone flesh bag. My iPhone won't forget my appointments like I do.

fragmede 3y ago
Car crashes haven't stopped happening, but it's undeniable that cars have progressed since the Model-T first came out over a hundred years ago.
jgoodhcg 3y ago
There’s room in the world for a tool that has an error rate but also an astonishing ability to accelerate the work of a person.

FrankyHollywood 3y ago

Indeed it works darn well, my company uses a complex programming assignment during application. Only about 5% of computer science students applying manages to create a decent solution within a few hours. I was curious if GPT could solve it. I provided the assignment text without any extra information, and it came up with a very elegant solution. You might not want to call this 'consciousness', but I was stunned by the deep understanding of the problem and the way it was able to come up with a truly good solution, this is way beyond 'statistically guessing'.