5 ms·

Curiosity is inherently proactive. An LLM, fundamentally, is a file. It's a FILE. It just sits there until you ask for output. Saying anything more about that p

Curiosity is inherently proactive. An LLM, fundamentally, is a file. It's a FILE. It just sits there until you ask for output. Saying anything more about that process is just anthropomorphizing. No one ever accused the Google index of having "curiosity", but the idea is basically the same - you give it a query and it gives you back a response. Then it just sits idle until the next query.

powerapple 2y ago
proactive can be: we take infinite inputs at a resolution, having eyes, ears, all sensors constantly "asking" for output
swid 2y ago
I disagree on the premise that computers are able to simulate many different things; it would be just as easy to say that the universe cannot have curiosity; that the universe is just things playing out according to some preset rules. But of course, people exist within the universe, and while our brains do function according to those rules; likely all rules that can be expressed with math and statistics; we do have curiosity. You can look at the level of abstraction of the universe, and you will not find subjective experience or choice, yet the “platform” of the universe demonstrably allows for that. When I see arguments like yours and the parent’s, I cannot help but think the arguments would seem to apply just as well to an accurate simulation of the universe, which shows the argument must be flawed. You are a file in the universe, loaded into memory and being run in parallel with my file. If you believe physics can be expressed through math and humans have subjective experienced, then the right kind of simulation can also have these things. Of course any simulation can be represented digitally and saved to disk.
- HarHarVeryFunny 2y ago
  > it would be just as easy to say that the universe cannot have curiosity; that the universe is just things playing out according to some preset rules. Behavior/dynamics depends on structure, and structure is scale dependent. At the scale of galaxies and stars, the universe has a relatively simple structure, with relatively boring dynamics mostly governed by gravity and nuclear fusion. When you zoom down to the level of dynamics on a planet full of life, or of a living entity such as a human, then things get a lot more interesting since at those scales there is far more structure causing a much greater variety of dynamics. It's the structure of an LLM vs the structure of a human brain that makes the difference.
  - swid 2y ago
    But I am saying that is the wrong comparison. The LLM doesn’t need to implement a human brain directly. It needs to implement a sophisticated enough simulation of people that the simulation itself contains ”people” who believe in themselves. I don’t know LLMs do that, but they are excellent function approximators, and the laws of physics which allow for my brain to be conscious also can be considered some function to approximate. If the LLM can approximate that function well enough, then simulated humans would truly believe in their own existence, as I do. And it isn’t really important whether or not consciousness is part of the simulation or not, if the end result is the simulator is capable of emulating people to a greater extent.
    - HarHarVeryFunny 2y ago
      If you wanted to build a behavioral simulation of a human, and knew how to do it, then what advantage would there be to trying to get an LLM to emulate this simulator ?! Better just code up your simulator to run efficiently on a supercomputer!
      - swid 2y ago
        The LLM teaches itself the rules of the simulation by learning to predict what happens next so well. Presumable, running a human simulation by brute forcing physics at a scale large enough to represent a human is completely infeasible, but we can maybe conceive how it would work. LLMs are an impressive engine for predicting “next” that is actually computationally feasible.
HarHarVeryFunny 2y ago
Sure, but even though an LLM is only a function that "acts" (outputs a word) when called, it could (if it weren't just an LLM!) use that opportunity to act to pursue some curiosity-based line of discussion. One limiting factor as far as curiosity goes isn't just that an LLM is a passive function, but also that it's just a statistical sequence-to-sequence machine (a transformer) - that's all that exists under the hood. It doesn't have any mechanism for innate traits to influence the generation process. All you could do would be to train that one mechanism it does have to mimic human responses judged to reflect curiosity about specific fine-tuning inputs.
CooCooCaCha 2y ago
A plane, are you crazy? It's just a metal tube with sheets on both sides, a TUBE! Claiming it can fly like a bird is just anthropomorphizing.
- HarHarVeryFunny 2y ago
  I don't think that's a good analogy. We're talking about innate traits, not coarse functionality. A plane and a bird can both fly, but a plane has no innate desire to do so, whether to take advantage of good soaring conditions, or to escape ground-based predators, etc. An LLM and a human can both generate words, but the LLM is just trying to minimize repeating statistical errors it made when being pre-trained. The human's actions, including speech, are towards adaptive behavior to keep it alive per innate traits discovered by evolution. There's a massive difference.
  - CooCooCaCha 2y ago
    Honestly I don't really care what current LLMs can do, I'm more interested in fundamental limitations of AI and I think the "it's just a file" argument is nonsense and the analogy makes sense in that regard.
    - HarHarVeryFunny 2y ago
      I think you're focusing on the wrong part of his/her "it's just a file" argument. The actual point wasn't about the file packaging/form, but about the fact that it's just passive - just a function sitting there waiting to be called, not an active agent able to act out on it's curiosity. Curiosity is a trait of an agentic system where curiously is driving exploration leading to learning.
      - CooCooCaCha 2y ago
        I'm focusing on what they said. They said "an LLM, fundamentally, is a file". Which is true, but the implication is that LLMs can't be agentic, which may or may not be true.
GuB-42 2y ago
A LLM is not just a file, it also has context (the attention window in transformers). A LLM can have curiosity in the sense that it will ask questions. It can be a useful trait as a problem often seen in current "chat"-type LLMs is that they tend to talk a bit too authoritatively about things they don't know about (aka. hallucinations). Encouraging users to give a bit more context for better quality answers can counteract this. The point could be to make chatbots not like search engines, search engines will answer garbage with garbage, a chatbot can ask for precision when it can't give a satisfactory answer. For example: - How much is a pint in mL? - A US pint is 473 mL vs - Is it for a drink in the US? If so, a US pint in 473 mL, but in other contexts and locations, the answer can vary. The second answer is "curious", and by requesting extra context, it tells that the question is incomplete and with that extra context, it can give a better answer. For example, a pint of beer in France is likely to be 500mL, even though it is not really a pint, it is how it is understood.
- HarHarVeryFunny 2y ago
  Yeah, but that's not the model being curious about things that it itself doesn't know, it'd be the model copying a human response (training sample) based on what that human didn't know. To consider the difference, there might be some domain that the model was trained extensively on, and "knew" a lot about, but in a given context it might still act dumb and "show curiosity" just because it's mimicking a human response in that situation, not basing it's response on what it actually knows!
krisoft 2y ago
> Saying anything more about that process is just anthropomorphizing. Take "this LLM is more curious" as a shorthand for "the output generated by this LLM mimics the kind of behaviour we would describe in a human as being curious". > It just sits there until you ask for output. That is indeed a property of the current interfaces. But this can be very easily changed. If we choose to we can just pipe in the clock, and then we can train the model to write to you after a while. Or we can make a system where certain outputs from the LLM cause the execution environment fetch it data from outside sources and input it into the LLM. And then there would be model weights which make the system just sit there and do nothing, and there would be model weights which browse wikipedia all day. I think it would be apt to call this second kind a "curios" model while the pervious one is not.
TeMPOraL 2y ago
> Curiosity is inherently proactive. An LLM, fundamentally, is a file. It's a FILE. It just sits there until you ask for output. And the difference between proactive and reactive here boils down entirely to an infinite loop and some kind of I/O (which could be as trivial as "web search" function call). It so happens that, in the way LLMs are deployed, you're supplying the loop through interaction. But nothing stops you from making a script and putting while(True) on top. Put another way, if you had a brain debugger and paused execution to step it, the paused brain would also "just sit there until you ask for output". LLM interactions are like that. It doesn't make it limited in any fundamental way, much like an interactive application isn't limited in a fundamental way just because you only ever use it by putting a breakpoint in its event loop and running a cycle at a time.

powerapple 2y ago

proactive can be: we take infinite inputs at a resolution, having eyes, ears, all sensors constantly "asking" for output

swid 2y ago

I disagree on the premise that computers are able to simulate many different things; it would be just as easy to say that the universe cannot have curiosity; that the universe is just things playing out according to some preset rules. But of course, people exist within the universe, and while our brains do function according to those rules; likely all rules that can be expressed with math and statistics; we do have curiosity. You can look at the level of abstraction of the universe, and you will not find subjective experience or choice, yet the “platform” of the universe demonstrably allows for that. When I see arguments like yours and the parent’s, I cannot help but think the arguments would seem to apply just as well to an accurate simulation of the universe, which shows the argument must be flawed. You are a file in the universe, loaded into memory and being run in parallel with my file. If you believe physics can be expressed through math and humans have subjective experienced, then the right kind of simulation can also have these things. Of course any simulation can be represented digitally and saved to disk.

HarHarVeryFunny 2y ago
> it would be just as easy to say that the universe cannot have curiosity; that the universe is just things playing out according to some preset rules. Behavior/dynamics depends on structure, and structure is scale dependent. At the scale of galaxies and stars, the universe has a relatively simple structure, with relatively boring dynamics mostly governed by gravity and nuclear fusion. When you zoom down to the level of dynamics on a planet full of life, or of a living entity such as a human, then things get a lot more interesting since at those scales there is far more structure causing a much greater variety of dynamics. It's the structure of an LLM vs the structure of a human brain that makes the difference.
- swid 2y ago
  But I am saying that is the wrong comparison. The LLM doesn’t need to implement a human brain directly. It needs to implement a sophisticated enough simulation of people that the simulation itself contains ”people” who believe in themselves. I don’t know LLMs do that, but they are excellent function approximators, and the laws of physics which allow for my brain to be conscious also can be considered some function to approximate. If the LLM can approximate that function well enough, then simulated humans would truly believe in their own existence, as I do. And it isn’t really important whether or not consciousness is part of the simulation or not, if the end result is the simulator is capable of emulating people to a greater extent.
  - HarHarVeryFunny 2y ago
    If you wanted to build a behavioral simulation of a human, and knew how to do it, then what advantage would there be to trying to get an LLM to emulate this simulator ?! Better just code up your simulator to run efficiently on a supercomputer!
    - swid 2y ago
      The LLM teaches itself the rules of the simulation by learning to predict what happens next so well. Presumable, running a human simulation by brute forcing physics at a scale large enough to represent a human is completely infeasible, but we can maybe conceive how it would work. LLMs are an impressive engine for predicting “next” that is actually computationally feasible.

HarHarVeryFunny 2y ago

Sure, but even though an LLM is only a function that "acts" (outputs a word) when called, it could (if it weren't just an LLM!) use that opportunity to act to pursue some curiosity-based line of discussion. One limiting factor as far as curiosity goes isn't just that an LLM is a passive function, but also that it's just a statistical sequence-to-sequence machine (a transformer) - that's all that exists under the hood. It doesn't have any mechanism for innate traits to influence the generation process. All you could do would be to train that one mechanism it does have to mimic human responses judged to reflect curiosity about specific fine-tuning inputs.

CooCooCaCha 2y ago

A plane, are you crazy? It's just a metal tube with sheets on both sides, a TUBE! Claiming it can fly like a bird is just anthropomorphizing.

HarHarVeryFunny 2y ago
I don't think that's a good analogy. We're talking about innate traits, not coarse functionality. A plane and a bird can both fly, but a plane has no innate desire to do so, whether to take advantage of good soaring conditions, or to escape ground-based predators, etc. An LLM and a human can both generate words, but the LLM is just trying to minimize repeating statistical errors it made when being pre-trained. The human's actions, including speech, are towards adaptive behavior to keep it alive per innate traits discovered by evolution. There's a massive difference.
- CooCooCaCha 2y ago
  Honestly I don't really care what current LLMs can do, I'm more interested in fundamental limitations of AI and I think the "it's just a file" argument is nonsense and the analogy makes sense in that regard.
  - HarHarVeryFunny 2y ago
    I think you're focusing on the wrong part of his/her "it's just a file" argument. The actual point wasn't about the file packaging/form, but about the fact that it's just passive - just a function sitting there waiting to be called, not an active agent able to act out on it's curiosity. Curiosity is a trait of an agentic system where curiously is driving exploration leading to learning.
    - CooCooCaCha 2y ago
      I'm focusing on what they said. They said "an LLM, fundamentally, is a file". Which is true, but the implication is that LLMs can't be agentic, which may or may not be true.

GuB-42 2y ago

A LLM is not just a file, it also has context (the attention window in transformers). A LLM can have curiosity in the sense that it will ask questions. It can be a useful trait as a problem often seen in current "chat"-type LLMs is that they tend to talk a bit too authoritatively about things they don't know about (aka. hallucinations). Encouraging users to give a bit more context for better quality answers can counteract this. The point could be to make chatbots not like search engines, search engines will answer garbage with garbage, a chatbot can ask for precision when it can't give a satisfactory answer. For example: - How much is a pint in mL? - A US pint is 473 mL vs - Is it for a drink in the US? If so, a US pint in 473 mL, but in other contexts and locations, the answer can vary. The second answer is "curious", and by requesting extra context, it tells that the question is incomplete and with that extra context, it can give a better answer. For example, a pint of beer in France is likely to be 500mL, even though it is not really a pint, it is how it is understood.

HarHarVeryFunny 2y ago
Yeah, but that's not the model being curious about things that it itself doesn't know, it'd be the model copying a human response (training sample) based on what that human didn't know. To consider the difference, there might be some domain that the model was trained extensively on, and "knew" a lot about, but in a given context it might still act dumb and "show curiosity" just because it's mimicking a human response in that situation, not basing it's response on what it actually knows!

krisoft 2y ago

> Saying anything more about that process is just anthropomorphizing. Take "this LLM is more curious" as a shorthand for "the output generated by this LLM mimics the kind of behaviour we would describe in a human as being curious". > It just sits there until you ask for output. That is indeed a property of the current interfaces. But this can be very easily changed. If we choose to we can just pipe in the clock, and then we can train the model to write to you after a while. Or we can make a system where certain outputs from the LLM cause the execution environment fetch it data from outside sources and input it into the LLM. And then there would be model weights which make the system just sit there and do nothing, and there would be model weights which browse wikipedia all day. I think it would be apt to call this second kind a "curios" model while the pervious one is not.

TeMPOraL 2y ago

> Curiosity is inherently proactive. An LLM, fundamentally, is a file. It's a FILE. It just sits there until you ask for output. And the difference between proactive and reactive here boils down entirely to an infinite loop and some kind of I/O (which could be as trivial as "web search" function call). It so happens that, in the way LLMs are deployed, you're supplying the loop through interaction. But nothing stops you from making a script and putting while(True) on top. Put another way, if you had a brain debugger and paused execution to step it, the paused brain would also "just sit there until you ask for output". LLM interactions are like that. It doesn't make it limited in any fundamental way, much like an interactive application isn't limited in a fundamental way just because you only ever use it by putting a breakpoint in its event loop and running a cycle at a time.