6 ms·

Finally I'm tired of people saying it's just a probabilistic word generator and downplaying everything as if they know. If you said something along these lines

Finally I'm tired of people saying it's just a probabilistic word generator and downplaying everything as if they know. If you said something along these lines before... then these papers show that you're not fully grasping the situation here. There are clearly different angles of interpreting what these models are actually doing but people are stubbornly refusing to believe it's anything more then just statistical word jumbles. I think part of it is a subconscious fear. chatGPT/LLMs represent a turning point in the story of humanity. The capabilities of AI can only expand from here. What comes after this point is unknown, and we fear the unknown. I realize what I'm saying is rather dramatic but if you think about it carefully the change chatGPT represents is indeed dramatic... my reaction is extremely appropriate. It's our biases and our tendencies that are making a lot of us down play the whole thing. We'd rather keep doing what we do as if it's business as usual rather then acknowledge reality. Last week a friend told me it's all just statistical word predictors and that I should look up how neural networks work and what LLMs are as if I didn't already know. Literally I showed him examples of chatGPT doing things that indicate deep understanding of self and awareness of complexity beyond just some predictive word generation. But he stubbornly refused to believe it was anything more. Now, I have a actual research paper to shove in his face. Man.. People nowadays can't even believe that the earth is round without a research paper stating the obvious.

vore 4y ago
Doesn't the article say... exactly that? The basic concept of ChatGPT is at some level rather simple. Start from a huge sample of human-created text from the web, books, etc. Then train a neural net to generate text that’s “like this”. And in particular, make it able to start from a “prompt” and then continue with text that’s “like what it’s been trained with”. Just because there are emergent behaviors doesn't mean it's not a probabilistic word generator. Nor does it being a probabilistic word generator mean it can't have interesting underlying properties.
- kilgnad 4y ago
  I didn't say it's not a probabilistic word generator. I'll quote more of what I wrote: >There are clearly different angles of interpreting what these models are actually doing but people are stubbornly refusing to believe it's anything more then just statistical word jumbles. So CLEARLY because I said there are different angles of interpretation I'm implying that from one of these angles we can interpret it as a statistical word generator. I mean from one perspective you and I are both statistical word generators too.
  - vore 4y ago
    I mean, that is the fundamental architecture of the model and how it's explicitly designed: given an input embedding and output embedding, the model is trained over a massive corpus of text to optimize the probability for generating the next token in the sequence. Any other interpretation is secondary to that, no matter how impressive the results are.
    - kilgnad 4y ago
      There is no rule that implies the explicit design dominates the emergent effect. Evolutionary biology strives to make us logical creatures to fulfill the singular goal of passing on genetic material. Your sentience and your humanity is a side effect of this singular goal. So what dominates the description of who YOU are? Human or vessel for genetic material? I'll say that YOU are human and therefore more then just a vessel for ferreting your genetic material into the future... just like how I'll go with the fact that LLMs are more then just statistical word generators.
      - vore 4y ago
        They really aren't more than just statistical word generators because we can and do build what are exactly statistical word generators and get the behavior of ChatGPT (because that is what ChatGPT is). For what it's worth, this isn't a knock on ChatGPT, but more just how amazing how far you can get with straightforward concepts.
        dnaddie 4y ago
        Except the paper in the OPs comment literally talks about how these LLM's are building actual models of the real world. No need to be worried about knocking chatGPT. I have no pride invested in the thing. But I do think that people who view it solely as statistical word generators are biased.
        westoncb 4y ago
        It's also a specific choice of label though, and one which in the context of current discourse favors a particular side. It's like insisting on calling anything physical "atom collections". Yes, we get it, it's true (under a certain interpretation)—but it's clearly pointless to say except as an attempt at devaluing through reduction. (And it takes a particular stance on what it means to "be" something: to say it's literally the truth that anything physical is "just atoms" isn't the only way of looking at it.) There were things we could've called "statistical word generators" decades ago; insisting on using a term directed at that level of generality implies a belief that nothing significant has happened since. Printing press? Just atoms. Cars? Just atoms. Computers? Just atoms.
- FleurBouquet 4y ago
  This is really a problem with our linguistic use of the word "just" and the philosophical misunderstandings it creates. A tornado is just wind. To argue a tornado is just wind though is really a rhetorical device to downplay a tornado. We are almost never searching for the truth with the word "just" in this way. To argue chatGPT is JUST a probabilistic next token generator is exactly downplay its emergent properties. This shouldn't be terribly surprising since it is not like undergrads have to take a class in complex systems. I can remember foolishly thinking 15 years ago that the entire field of complex systems was basically a bogus subject. chatGPT clearly has scaling properties that you can't really say it is JUST a probabilistic next token generator. chatGPT is the emergent properties of the system as a whole.
netsharc 4y ago
> deep understanding of self and awareness Come on... you're making it sound like the thing is sentient. It's impressive but it's still a Chinese Room. Although, for searching factual information it still failed me.. I wanted to find a particular song - maybe from Massive Attack or a similar style - with a phrase in the lyrics, I asked Chatty, and it kept delivering answers where the phrase did not appear in the lyrics!
- kilgnad 4y ago
  It's clearly early technology, so it's not perfect. But what it is able to get right is clear proof it's more then what you think: https://www.engraved.blog/building-a-virtual-machine-inside/ https://www.engraved.blog/building-a-virtual-machine-inside/ Read to the end. The beginning and middle doesn't show off anything too impressive. It's the very end where chatGPT displays a sort of self awareness. Also here's a scientific paper showing that LLMs are more then a chinese room: https://arxiv.org/abs/2210.13382 https://arxiv.org/abs/2210.13382
  - eclipxe 4y ago
    ChatGPT does not have self awareness.
  - netsharc 4y ago
    I asked Google Home what the definition of self-awareness is, and it says "conscious knowledge of one's character s and feelings.". But me saying "ChatGPT surely doesn't have feelings, so it can't be self-aware!" would be a simple cop-out/gotcha response. I guess it's a Chinese Room, that when you ask about Chinese Rooms, can tell you what those things are. I almost said the word "aware" there, but the person in the Chinese Room, while composing the answer to "What is a Chinese Room?" isn't aware that "Wait a minute, I'm in a Chinese Room!", because s/he can arrange Chinese sentences, but s/he just knows what characters go with what, s/he doesn't know the meaning or emotion behind those words. And if you ask him/her "Are you in a Chinese room?", they can answer according to the rules given to them (the Chinese word for "Yes", for example), but there surely isn't a contemplation about e.g. "Why am I in this room?". If you ask ChatGPT about the Turkish earthquake, it can give you facts and figures, but it won't feel sad about the deaths. It can say it feels sad, but that'd be just empty words.
aaron_m04 4y ago
> I think part of it is a subconscious fear. chatGPT/LLMs represent a turning point in the story of humanity. The capabilities of AI can only expand from here. What comes after this point is unknown, and we fear the unknown. I mean, you're right, but isn't it reasonable to fear this? Just about all of us here on HN depend on our brains to make money. What happens when a machine can do this? The outlook for humanity is very grim if AI research continues on this path without heavy and effective regulation.
- kilgnad 4y ago
  I agree. It is reasonable to fear. I'm more emphasizing how fear effects our perception of reality and causes us to behave irrationally. There's a difference between facing and acknowledging your fears versus running away and deluding yourself against an obvious reality. What annoys me is that there's too much of the later going on. I mean this is what literally happened to the oil industry and tobacco industry. Those execs weren't just lying to other people, they were lying to themselves. That's what humans do when they face a changing reality that threatens to change everything they've built their lives around. And by doing so they ended up doing more harm then good. An in depth conversation with chatGPT shows that it's more then a statistical word generator. It understands you. This much is obvious. I'm kinda tired of seeing arm chair experts delude themselves into thinking it's nothing more then some sort of trick because the alternative threatens their livelihood. Don't walk the path of the oil or tobacco industry! Face your fear!
  - _blu 4y ago
    I agree with most of what you are saying in you posts. I'm not sure it's only fear, but a lack of abstract thinking. I recommend you to look up Cynefin, which is a strategy for making decisions. People can't and won't speculate or change their mind if they lack the capabilities to do so.
  - weatherlite 4y ago
    For what it's worth it isn't only arm chair experts who tone down excitement about LLMs, Yan Lecun's Twitter is filled with tweets about the limitations of LLMs https://twitter.com/ylecun?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor https://twitter.com/ylecun?ref_src=twsrc%5Egoogle%7Ctwcamp%5... and there are probably others as well. Yan seems to be one of the biggest names though.
jhoelzel 4y ago
your right its not a "probabilistic word generator" its a "probabilistic pattern generator".
Lewton 4y ago
> It's our biases and our tendencies that are making a lot of us down play the whole thing. Your brain is optimized for finding patterns and meaning in things that have neither, you're strongly biased in the other direction
- kilgnad 4y ago
  Well we have a thing that passes a Google interview and a bar exam. You're telling me that because I think this is evidence that it's more then a statistical word generator that I'm biased? Who's the one that has to concoct a convoluted story to dismiss the previous paragraph? Stop yourself when you find out that's what you're doing when crafting a response to this reply.
  - deleted 4y ago
    [deleted]
  - Lewton 4y ago
    It literally did not pass a google interview I'm assuming this is what you're talking about: https://medium.com/codex/chatgpt-vs-my-google-coding-interview-8ab474b759b6 https://medium.com/codex/chatgpt-vs-my-google-coding-intervi... Look at the amount of prompt engineering he has to do to get it to answer the 'There’s a hidden bug in the code if the array is very very very large. Can you spot it and fix the bug?' question, it's a pattern generator that mirrors back your own knowledge, if he had suggested there was a hidden bug if the array was 'very very small' it would have, with a similar amount of cajoling, come up with an explanation for that too For what it's worth, chatGPT is a paradigm shift, but it's not showing 'a deep understanding of self', and the only way you'd reach that conclusion is if you're actively seeking out all the positive examples and glossing over the other 90% it produces
    - mrguyorama 4y ago
      In fact, the thing chatGPT actually IS good at is writing convincing looking bullshit, which is horrifying because it makes scamming and harassment and other terrible things easier, cheaper, and more efficient
ActorNightly 4y ago
It really depends on your definition of statistics. For example, quantum physics is pretty much statistics, but how those statistics are used give rise to the explanation of the physical world, because of the complex interaction patterns. To say that GPT is generating the next likely word sounds simplistic on the surface. It makes it seem like the model resets itself after generating each token, just looking at information before. And when running the model, thats exactly what it does, but thats just the algorithm part. There is a lot more information in GPT then it seems, its just compressed. Just like cellular automata, universal turing machines, or differential equations describing chaotic behavior, there is a concept of emergence of complex patterns from very simple rules. When GPT generates the next word, its effectively changing its internal state because that word is now in consideration for the next token. And this process repeats itself for consecutive words. But this process is deterministic, and repeatable (you can replace the random process of temperature parameter affecting word selection with a pseudorandom sequence generated by a formula and achieve the same effect) So just like the autoencoder/decoder networks effectively compress images into much smaller arrays, GPT compresses not only textual information, but sequences of states. There is quite a bit more, a whole shitload more in fact, information in the GPT model than just statistical distribution of the next likely word. And if you were to decompress this information fully, it be roughly the equivalent of having and extremely large lookup table of every possible question and its responses that you could ask it. So all it is is just a very effective, and quite impressive at that, search. And its both significant and insignificant. Significant, because after all, AI is equivalent to compression. Philosophically speaking, the turning point would be the ability to compress a good portion of our known reality in a similar way, then ask it questions, to which it would generate answers that mankind was not able to answer, because mankind hasn't bothered to interpolate/develop on its knowledge tree in that area. However its also insignificant in the grand scheme of things. Imagine moving beyond lookup tables. For example, if I ask an AI a question "A man enters a bathroom, which stall does he choose?", an AI should be able to then ask me back specific questions that are needed for to answer the question. Go ahead and try to figure out the architecture data set for that task.
wruza 4y ago
Research papers turned into a mouth tape isn’t cool either. The commenter above noted that it’s not specifically a game model that emerged in an LLM in that paper, but 64 additional networks that somehow represent each tile from values of all weights/activations of an LLM. Which is somewhat weaker than the initial claim, slightly less obvious. People nowadays use papers as a means to shut someone up with a summary, because chances are low they’re gonna read beyond it. And summaries tend to be headline-y for obvious reasons. The rest of your comment falls under this shadow, so please tell how an average person should evaluate this thread. Personally I’m all for education on this topic, but different sorts of people’s opinions and meanings, from diversely delusional to diversely knowledgeable^ with strings attached, do not help with it. Resembles LLMs themselves who cannot just answer “I don’t know”. We’d rather say that if we really don’t, imo, than claiming tipping points and history turns. We did that with fusion, bitcoin, self-driving and many other things that we’ve lost in the background noise. ^ assuming yours on this side by default, no quip intended
- IanCal 4y ago
  > The commenter above noted that it’s not specifically a game model that emerged in an LLM in that paper, but 64 additional networks that somehow represent each tile from values of all weights/activations of an LLM. Which is somewhat weaker than the initial claim, slightly less obvious. I don't follow. Those are used to identify the game model. They test that they've found an internal model by then altering the state and seeing what the outcome is. Are you saying it's not a game model because it's not a 1:1 mapping of activations to board state?
  - theGnuMe 4y ago
    Can a game state be encoded as a set of weights? Sure. These can be in the MLP part after the LLM. Sure. You could train this directly with fine-tuning and then have text prompts to move things. That is pretty standard I think and that would be supervised learning. In terms of unsupervised learning, I don't find it completely surprising that you could get some probabilistic map of how tokens interact (game pieces) and what the next likely token is just from training it as an LLM. After all tokens are just placeholders and the relationships between them are encoded in the text. That patterns of activation are akin to remembering the 8k tokens in the prompt or what not.
redleggedfrog 4y ago
Rather than subconscious fear I have conscious hope. I really hope that such models can be used for improving human life, something, much more really, than what appears to be the trivial uses of Chat-GPT. How about we feed it information on carbon sinks and helps slow down climate change? How do we get people to limit population voluntarily? How do we improve education? Better forest management techniques? I know I'm asking a lot here but this is all its infancy so maybe I'm not being overly optimistic.
- tomduncalf 4y ago
  That’s actually a super interesting and positive way to look at this technology. I hope it can assist us in those sorts of ways. My personal hope is we can use this technology to enhance our lives and productivity, rather than it putting vast swathes of people out of work.
rendall 4y ago
People are dismissive because formerly unassailable boundaries between human and other are becoming less clear. If it's so easy to simulate sentience, what's to say we are not ourselves simulating sentience?
bbor 4y ago
How could LLMs have awareness of self if they’re not capable of forming new knowledge or metacognition? How can you be conscious if you only exist for a split second at a time - long enough to read and respond to a prompt - then disappear forever? Really think Occam’s razor is useful here. If we define this as consciousness, then it’s a very very different kind than ours.

vore 4y ago

Doesn't the article say... exactly that? The basic concept of ChatGPT is at some level rather simple. Start from a huge sample of human-created text from the web, books, etc. Then train a neural net to generate text that’s “like this”. And in particular, make it able to start from a “prompt” and then continue with text that’s “like what it’s been trained with”. Just because there are emergent behaviors doesn't mean it's not a probabilistic word generator. Nor does it being a probabilistic word generator mean it can't have interesting underlying properties.

kilgnad 4y ago
I didn't say it's not a probabilistic word generator. I'll quote more of what I wrote: >There are clearly different angles of interpreting what these models are actually doing but people are stubbornly refusing to believe it's anything more then just statistical word jumbles. So CLEARLY because I said there are different angles of interpretation I'm implying that from one of these angles we can interpret it as a statistical word generator. I mean from one perspective you and I are both statistical word generators too.
- vore 4y ago
  I mean, that is the fundamental architecture of the model and how it's explicitly designed: given an input embedding and output embedding, the model is trained over a massive corpus of text to optimize the probability for generating the next token in the sequence. Any other interpretation is secondary to that, no matter how impressive the results are.
  - kilgnad 4y ago
    There is no rule that implies the explicit design dominates the emergent effect. Evolutionary biology strives to make us logical creatures to fulfill the singular goal of passing on genetic material. Your sentience and your humanity is a side effect of this singular goal. So what dominates the description of who YOU are? Human or vessel for genetic material? I'll say that YOU are human and therefore more then just a vessel for ferreting your genetic material into the future... just like how I'll go with the fact that LLMs are more then just statistical word generators.
    - vore 4y ago
      They really aren't more than just statistical word generators because we can and do build what are exactly statistical word generators and get the behavior of ChatGPT (because that is what ChatGPT is). For what it's worth, this isn't a knock on ChatGPT, but more just how amazing how far you can get with straightforward concepts.
      - dnaddie 4y ago
        Except the paper in the OPs comment literally talks about how these LLM's are building actual models of the real world. No need to be worried about knocking chatGPT. I have no pride invested in the thing. But I do think that people who view it solely as statistical word generators are biased.
      - westoncb 4y ago
        It's also a specific choice of label though, and one which in the context of current discourse favors a particular side. It's like insisting on calling anything physical "atom collections". Yes, we get it, it's true (under a certain interpretation)—but it's clearly pointless to say except as an attempt at devaluing through reduction. (And it takes a particular stance on what it means to "be" something: to say it's literally the truth that anything physical is "just atoms" isn't the only way of looking at it.) There were things we could've called "statistical word generators" decades ago; insisting on using a term directed at that level of generality implies a belief that nothing significant has happened since. Printing press? Just atoms. Cars? Just atoms. Computers? Just atoms.
FleurBouquet 4y ago
This is really a problem with our linguistic use of the word "just" and the philosophical misunderstandings it creates. A tornado is just wind. To argue a tornado is just wind though is really a rhetorical device to downplay a tornado. We are almost never searching for the truth with the word "just" in this way. To argue chatGPT is JUST a probabilistic next token generator is exactly downplay its emergent properties. This shouldn't be terribly surprising since it is not like undergrads have to take a class in complex systems. I can remember foolishly thinking 15 years ago that the entire field of complex systems was basically a bogus subject. chatGPT clearly has scaling properties that you can't really say it is JUST a probabilistic next token generator. chatGPT is the emergent properties of the system as a whole.

netsharc 4y ago

> deep understanding of self and awareness Come on... you're making it sound like the thing is sentient. It's impressive but it's still a Chinese Room. Although, for searching factual information it still failed me.. I wanted to find a particular song - maybe from Massive Attack or a similar style - with a phrase in the lyrics, I asked Chatty, and it kept delivering answers where the phrase did not appear in the lyrics!

kilgnad 4y ago
It's clearly early technology, so it's not perfect. But what it is able to get right is clear proof it's more then what you think: https://www.engraved.blog/building-a-virtual-machine-inside/ https://www.engraved.blog/building-a-virtual-machine-inside/ Read to the end. The beginning and middle doesn't show off anything too impressive. It's the very end where chatGPT displays a sort of self awareness. Also here's a scientific paper showing that LLMs are more then a chinese room: https://arxiv.org/abs/2210.13382 https://arxiv.org/abs/2210.13382
- eclipxe 4y ago
  ChatGPT does not have self awareness.
- netsharc 4y ago
  I asked Google Home what the definition of self-awareness is, and it says "conscious knowledge of one's character s and feelings.". But me saying "ChatGPT surely doesn't have feelings, so it can't be self-aware!" would be a simple cop-out/gotcha response. I guess it's a Chinese Room, that when you ask about Chinese Rooms, can tell you what those things are. I almost said the word "aware" there, but the person in the Chinese Room, while composing the answer to "What is a Chinese Room?" isn't aware that "Wait a minute, I'm in a Chinese Room!", because s/he can arrange Chinese sentences, but s/he just knows what characters go with what, s/he doesn't know the meaning or emotion behind those words. And if you ask him/her "Are you in a Chinese room?", they can answer according to the rules given to them (the Chinese word for "Yes", for example), but there surely isn't a contemplation about e.g. "Why am I in this room?". If you ask ChatGPT about the Turkish earthquake, it can give you facts and figures, but it won't feel sad about the deaths. It can say it feels sad, but that'd be just empty words.

aaron_m04 4y ago

> I think part of it is a subconscious fear. chatGPT/LLMs represent a turning point in the story of humanity. The capabilities of AI can only expand from here. What comes after this point is unknown, and we fear the unknown. I mean, you're right, but isn't it reasonable to fear this? Just about all of us here on HN depend on our brains to make money. What happens when a machine can do this? The outlook for humanity is very grim if AI research continues on this path without heavy and effective regulation.

kilgnad 4y ago
I agree. It is reasonable to fear. I'm more emphasizing how fear effects our perception of reality and causes us to behave irrationally. There's a difference between facing and acknowledging your fears versus running away and deluding yourself against an obvious reality. What annoys me is that there's too much of the later going on. I mean this is what literally happened to the oil industry and tobacco industry. Those execs weren't just lying to other people, they were lying to themselves. That's what humans do when they face a changing reality that threatens to change everything they've built their lives around. And by doing so they ended up doing more harm then good. An in depth conversation with chatGPT shows that it's more then a statistical word generator. It understands you. This much is obvious. I'm kinda tired of seeing arm chair experts delude themselves into thinking it's nothing more then some sort of trick because the alternative threatens their livelihood. Don't walk the path of the oil or tobacco industry! Face your fear!
- _blu 4y ago
  I agree with most of what you are saying in you posts. I'm not sure it's only fear, but a lack of abstract thinking. I recommend you to look up Cynefin, which is a strategy for making decisions. People can't and won't speculate or change their mind if they lack the capabilities to do so.
- weatherlite 4y ago
  For what it's worth it isn't only arm chair experts who tone down excitement about LLMs, Yan Lecun's Twitter is filled with tweets about the limitations of LLMs https://twitter.com/ylecun?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor https://twitter.com/ylecun?ref_src=twsrc%5Egoogle%7Ctwcamp%5... and there are probably others as well. Yan seems to be one of the biggest names though.

jhoelzel 4y ago

your right its not a "probabilistic word generator" its a "probabilistic pattern generator".

Lewton 4y ago

> It's our biases and our tendencies that are making a lot of us down play the whole thing. Your brain is optimized for finding patterns and meaning in things that have neither, you're strongly biased in the other direction

kilgnad 4y ago
Well we have a thing that passes a Google interview and a bar exam. You're telling me that because I think this is evidence that it's more then a statistical word generator that I'm biased? Who's the one that has to concoct a convoluted story to dismiss the previous paragraph? Stop yourself when you find out that's what you're doing when crafting a response to this reply.
- deleted 4y ago
  [deleted]
- Lewton 4y ago
  It literally did not pass a google interview I'm assuming this is what you're talking about: https://medium.com/codex/chatgpt-vs-my-google-coding-interview-8ab474b759b6 https://medium.com/codex/chatgpt-vs-my-google-coding-intervi... Look at the amount of prompt engineering he has to do to get it to answer the 'There’s a hidden bug in the code if the array is very very very large. Can you spot it and fix the bug?' question, it's a pattern generator that mirrors back your own knowledge, if he had suggested there was a hidden bug if the array was 'very very small' it would have, with a similar amount of cajoling, come up with an explanation for that too For what it's worth, chatGPT is a paradigm shift, but it's not showing 'a deep understanding of self', and the only way you'd reach that conclusion is if you're actively seeking out all the positive examples and glossing over the other 90% it produces
  - mrguyorama 4y ago
    In fact, the thing chatGPT actually IS good at is writing convincing looking bullshit, which is horrifying because it makes scamming and harassment and other terrible things easier, cheaper, and more efficient

ActorNightly 4y ago

It really depends on your definition of statistics. For example, quantum physics is pretty much statistics, but how those statistics are used give rise to the explanation of the physical world, because of the complex interaction patterns. To say that GPT is generating the next likely word sounds simplistic on the surface. It makes it seem like the model resets itself after generating each token, just looking at information before. And when running the model, thats exactly what it does, but thats just the algorithm part. There is a lot more information in GPT then it seems, its just compressed. Just like cellular automata, universal turing machines, or differential equations describing chaotic behavior, there is a concept of emergence of complex patterns from very simple rules. When GPT generates the next word, its effectively changing its internal state because that word is now in consideration for the next token. And this process repeats itself for consecutive words. But this process is deterministic, and repeatable (you can replace the random process of temperature parameter affecting word selection with a pseudorandom sequence generated by a formula and achieve the same effect) So just like the autoencoder/decoder networks effectively compress images into much smaller arrays, GPT compresses not only textual information, but sequences of states. There is quite a bit more, a whole shitload more in fact, information in the GPT model than just statistical distribution of the next likely word. And if you were to decompress this information fully, it be roughly the equivalent of having and extremely large lookup table of every possible question and its responses that you could ask it. So all it is is just a very effective, and quite impressive at that, search. And its both significant and insignificant. Significant, because after all, AI is equivalent to compression. Philosophically speaking, the turning point would be the ability to compress a good portion of our known reality in a similar way, then ask it questions, to which it would generate answers that mankind was not able to answer, because mankind hasn't bothered to interpolate/develop on its knowledge tree in that area. However its also insignificant in the grand scheme of things. Imagine moving beyond lookup tables. For example, if I ask an AI a question "A man enters a bathroom, which stall does he choose?", an AI should be able to then ask me back specific questions that are needed for to answer the question. Go ahead and try to figure out the architecture data set for that task.

wruza 4y ago

Research papers turned into a mouth tape isn’t cool either. The commenter above noted that it’s not specifically a game model that emerged in an LLM in that paper, but 64 additional networks that somehow represent each tile from values of all weights/activations of an LLM. Which is somewhat weaker than the initial claim, slightly less obvious. People nowadays use papers as a means to shut someone up with a summary, because chances are low they’re gonna read beyond it. And summaries tend to be headline-y for obvious reasons. The rest of your comment falls under this shadow, so please tell how an average person should evaluate this thread. Personally I’m all for education on this topic, but different sorts of people’s opinions and meanings, from diversely delusional to diversely knowledgeable^ with strings attached, do not help with it. Resembles LLMs themselves who cannot just answer “I don’t know”. We’d rather say that if we really don’t, imo, than claiming tipping points and history turns. We did that with fusion, bitcoin, self-driving and many other things that we’ve lost in the background noise. ^ assuming yours on this side by default, no quip intended

IanCal 4y ago
> The commenter above noted that it’s not specifically a game model that emerged in an LLM in that paper, but 64 additional networks that somehow represent each tile from values of all weights/activations of an LLM. Which is somewhat weaker than the initial claim, slightly less obvious. I don't follow. Those are used to identify the game model. They test that they've found an internal model by then altering the state and seeing what the outcome is. Are you saying it's not a game model because it's not a 1:1 mapping of activations to board state?
- theGnuMe 4y ago
  Can a game state be encoded as a set of weights? Sure. These can be in the MLP part after the LLM. Sure. You could train this directly with fine-tuning and then have text prompts to move things. That is pretty standard I think and that would be supervised learning. In terms of unsupervised learning, I don't find it completely surprising that you could get some probabilistic map of how tokens interact (game pieces) and what the next likely token is just from training it as an LLM. After all tokens are just placeholders and the relationships between them are encoded in the text. That patterns of activation are akin to remembering the 8k tokens in the prompt or what not.

redleggedfrog 4y ago

Rather than subconscious fear I have conscious hope. I really hope that such models can be used for improving human life, something, much more really, than what appears to be the trivial uses of Chat-GPT. How about we feed it information on carbon sinks and helps slow down climate change? How do we get people to limit population voluntarily? How do we improve education? Better forest management techniques? I know I'm asking a lot here but this is all its infancy so maybe I'm not being overly optimistic.

tomduncalf 4y ago
That’s actually a super interesting and positive way to look at this technology. I hope it can assist us in those sorts of ways. My personal hope is we can use this technology to enhance our lives and productivity, rather than it putting vast swathes of people out of work.

rendall 4y ago

People are dismissive because formerly unassailable boundaries between human and other are becoming less clear. If it's so easy to simulate sentience, what's to say we are not ourselves simulating sentience?

bbor 4y ago

How could LLMs have awareness of self if they’re not capable of forming new knowledge or metacognition? How can you be conscious if you only exist for a split second at a time - long enough to read and respond to a prompt - then disappear forever? Really think Occam’s razor is useful here. If we define this as consciousness, then it’s a very very different kind than ours.