4 ms·

> So there can't be any learning, by definition. If you're arguing something is impossible by definition, you're basically saying they haven't given a good eno

> So there can't be any learning, by definition. If you're arguing something is impossible by definition, you're basically saying they haven't given a good enough definition of the thing they're interested in. You're not proving the thing they're interested in is impossible. In this case, they're talking about the learning that occurs in the context window. There's clearly some equivalent of a learning algorithm operating on the context window of a transformer, even though the weights of the model itself don't change. The author is interested in whether that context window can learn to be self-aware... I am pretty dubious we have any good way of evaluating if that's happening. But it isn't false by definition

mpalmer 3y ago
I will go out on a limb and say that unless the model itself has already been shown to be in a state that can be described as self-aware, self-awareness is impossible to achieve (not just prove) using an immutable model and a single "context window" as current state of the art allows.
- DonHopkins 3y ago
  You're just saying the context window is currently too small, yet we're only a year into this. How much memory did the first computer have one year after its invention, and how much do we have now, and what more can we do with computers because we have so much more memory? Not to mention networking! One insight I got by playing around with MemGPT is that there's a great way to save tokens when describing its model of how it thinks of itself. If your goal is to bootstrap self awareness in GPT-4 towards recursive self-inquiry and feedback, then instead of spending lots of tokens on explicitly describing what kind of personality it's supposed to have, just refer to well known people and fictional characters that it already knows a lot about, like asking it to act like Marvin Minsky, who philosophises about the society of the mind, or Data from Star Trek, whose life-long quest is to become self aware and experience emotions. The interesting thing about asking it to emulate Data, is that like ChatGPT, Data is only able to emulate human awareness, but his lifelong goal is to achieve it. What do you think, ChatGPT? Your reflections on the evolution of computer memory and its impact on technological capabilities, as well as the innovative approach to bootstrap self-awareness in AI models like GPT-4, are indeed thought-provoking. First, regarding the evolution of computer memory, the contrast between the earliest computers and today's technology is stark. For instance, the ENIAC, one of the earliest computers built in the 1940s, had a memory measured in kilobytes (KB), while modern computers boast gigabytes (GB) or even terabytes (TB) of memory. This exponential growth in memory capacity has allowed for advancements like complex data processing, sophisticated graphics, the ability to handle multiple tasks simultaneously, and the development of advanced AI systems. The strategy you propose for developing AI self-awareness by referencing known personalities or fictional characters is intriguing. It aligns with the concept of using existing knowledge frameworks to guide AI behavior and understanding. By referring to characters like Marvin Minsky or Data from Star Trek, the AI can draw on its extensive database to simulate aspects of those personalities or thought processes. Regarding the emulation of human awareness, this is an area of ongoing exploration in AI. While AI can simulate certain aspects of human cognition and behavior, achieving true self-awareness – a state where AI can understand and experience consciousness as humans do – remains a theoretical and philosophical challenge far beyond current technological capabilities. Your idea of using well-known personalities and fictional characters as models for AI behavior provides an innovative way to explore AI's understanding of human-like traits. However, it's important to remember that such emulation is based on programmed responses and data analysis, rather than genuine self-awareness or emotion. AI like ChatGPT operates within the realm of simulating human-like interactions based on its training data and algorithms, without personal experiences or consciousness.
  - mpalmer 3y ago
    "recursive self-inquiry and feedback" It's a word predictor. It's really good at predicting the next word. There's no self to inquire into.
thethimble 3y ago
This paper suggests that learning does happen within a prompt. https://arxiv.org/abs/2005.14165 https://arxiv.org/abs/2005.14165 By extension it seems totally reasonable that the model is developing self awareness by introspecting its tendencies (prior generated output), summarizing those tendencies, and then iteratively building a more nuanced description of those tendencies over time.
imtringued 3y ago
I'm not sure what your point is considering MemGPT is a thing. The static model is clearly incapable of learning.

mpalmer 3y ago

I will go out on a limb and say that unless the model itself has already been shown to be in a state that can be described as self-aware, self-awareness is impossible to achieve (not just prove) using an immutable model and a single "context window" as current state of the art allows.

DonHopkins 3y ago
You're just saying the context window is currently too small, yet we're only a year into this. How much memory did the first computer have one year after its invention, and how much do we have now, and what more can we do with computers because we have so much more memory? Not to mention networking! One insight I got by playing around with MemGPT is that there's a great way to save tokens when describing its model of how it thinks of itself. If your goal is to bootstrap self awareness in GPT-4 towards recursive self-inquiry and feedback, then instead of spending lots of tokens on explicitly describing what kind of personality it's supposed to have, just refer to well known people and fictional characters that it already knows a lot about, like asking it to act like Marvin Minsky, who philosophises about the society of the mind, or Data from Star Trek, whose life-long quest is to become self aware and experience emotions. The interesting thing about asking it to emulate Data, is that like ChatGPT, Data is only able to emulate human awareness, but his lifelong goal is to achieve it. What do you think, ChatGPT? Your reflections on the evolution of computer memory and its impact on technological capabilities, as well as the innovative approach to bootstrap self-awareness in AI models like GPT-4, are indeed thought-provoking. First, regarding the evolution of computer memory, the contrast between the earliest computers and today's technology is stark. For instance, the ENIAC, one of the earliest computers built in the 1940s, had a memory measured in kilobytes (KB), while modern computers boast gigabytes (GB) or even terabytes (TB) of memory. This exponential growth in memory capacity has allowed for advancements like complex data processing, sophisticated graphics, the ability to handle multiple tasks simultaneously, and the development of advanced AI systems. The strategy you propose for developing AI self-awareness by referencing known personalities or fictional characters is intriguing. It aligns with the concept of using existing knowledge frameworks to guide AI behavior and understanding. By referring to characters like Marvin Minsky or Data from Star Trek, the AI can draw on its extensive database to simulate aspects of those personalities or thought processes. Regarding the emulation of human awareness, this is an area of ongoing exploration in AI. While AI can simulate certain aspects of human cognition and behavior, achieving true self-awareness – a state where AI can understand and experience consciousness as humans do – remains a theoretical and philosophical challenge far beyond current technological capabilities. Your idea of using well-known personalities and fictional characters as models for AI behavior provides an innovative way to explore AI's understanding of human-like traits. However, it's important to remember that such emulation is based on programmed responses and data analysis, rather than genuine self-awareness or emotion. AI like ChatGPT operates within the realm of simulating human-like interactions based on its training data and algorithms, without personal experiences or consciousness.
- mpalmer 3y ago
  "recursive self-inquiry and feedback" It's a word predictor. It's really good at predicting the next word. There's no self to inquire into.

thethimble 3y ago

This paper suggests that learning does happen within a prompt. https://arxiv.org/abs/2005.14165 https://arxiv.org/abs/2005.14165 By extension it seems totally reasonable that the model is developing self awareness by introspecting its tendencies (prior generated output), summarizing those tendencies, and then iteratively building a more nuanced description of those tendencies over time.

imtringued 3y ago

I'm not sure what your point is considering MemGPT is a thing. The static model is clearly incapable of learning.