3 ms·

I don't get how you can see one of those CLI coding tools in action and still parrot the "no agency" line. The goal-oriented behavior is rather obvious. Sure,

I don't get how you can see one of those CLI coding tools in action and still parrot the "no agency" line. The goal-oriented behavior is rather obvious. Sure, they aren't very good at agentic behavior yet, and the time horizon is pretty low. But that keeps improving with each frontier release.

simonsarris 1y ago
Well, the goal-oriented behavior of the AIM-9 Sidewinder air-to-air missile is even more obvious. It might even have a higher success rate than CLI coding tools. But it's not helpful to claim it has any agency.
lo_zamoyski 1y ago
> The goal-oriented behavior is rather obvious. Obvious? Is an illusion obviously the real thing? There is nothing substantially different in LLMs from any other run of the mill algorithm or software.
- Eisenstein 1y ago
  Can you give an example of something that would be substantially different under your definition?
  - lo_zamoyski 1y ago
    But that's the point: there isn't anything substantially different within the scope of computation. If you are given a set of LEGOs and all you can do is snap the pieces together, then there's nothing other than snapping pieces together that you can do. Adding more of the same LEGOs bricks to the set doesn't change the game. It only changes how large the structures you build can be, but scale isn't some kind of magical incantation that can transcend the limits of the system. Computation is an abstract, syntactic mathematical model. These models formalize the notion of "effective method". Nowhere is semantic content included in these models or conceptually entailed by them, certainly not in physical simulations of them like the device you are reading this post on. So, we can say that intentionality would be something substantially different. We absolutely do not have intentionality in LLMs or any computational construct. It is shear magical thinking to somehow think it does.
    - Eisenstein 1y ago
      I think it is well established that scale can transcend limits. Look at insect colonies, animals, or any complex system and you will find it is made out of much simpler components.
      - lo_zamoyski 1y ago
        Scale qua scale does not transcend the limits of the activity occurring in the system. Meaning, each ant in an ant colony has a set of fixed powers to do X, Y, and Z. Adding more ants doesn't change what an individual ant is capable of per se. Now, you might say that while an individual ant exercising its power X cannot lift a tree branch, a colony of a thousand, with each ant exercising its power X, might. But this doesn't transcend the limitation of the individual ant - each ant is still only exercising some power X - nor is the aggregate manifesting some new kind if power. It is only multiplying the exercise of X. In the case of computers, a program is a set of formal rules that take some sequence of uninterpreted symbols and maps them to some other sequence of uninterpreted symbols. Adding more rules and more symbols scales the program - and it is the only scaling you can perform - but you don't somehow magically create interpreted symbols. How could it? This is magical thinking.
        Eisenstein 1y ago
        I think you misunderstand. I am not talking about physical ability. Ant colonies act as a complex system even though ants individually are incredibly simple. * https://en.wikipedia.org/wiki/Self-organization https://en.wikipedia.org/wiki/Self-organization * https://en.wikipedia.org/wiki/Patterns_of_self-organization_in_ants https://en.wikipedia.org/wiki/Patterns_of_self-organization_...
        lo_zamoyski 1y ago
        I don't misunderstand. And if your comment has any relevance to the discussion, you're using it as an analogy to suggest that something like syntax can somehow produce semantics, which is impossible - actually worse, it's incoherent. It's an intellectual muddle. Obviously simple entities can compose into complex aggregates and produce net effects that no individual could on its own, but it doesn't follow that they can generate things that were not in power of these individuals to generate. That's my point. Otherwise, you are claiming that something can come from nothing.
- ACCount37 1y ago
  In the same way there's "nothing substantially different" in humans from any other run of the mill matter. I find that all this talk of "illusion" is nothing but anthropocentric cope. Humans want to be those special little snowflakes, so when an LLM does X, there are crowds of humans itching to scream "it's not REAL X".
  - lo_zamoyski 1y ago
    > In the same way there's "nothing substantially different" in humans from any other run of the mill matter. This is an incredibly intellectual vacuous take. If there is no substantial difference between a human being and any other cluster of matter, then it is you who is saddled with the problem of explaining the obvious differences. If there is no difference between intelligent life and a pile of rocks, then what the hell are you even talking about? Why are we talking about AI and intelligence at all? Either everything is intelligent, or nothing is, if we accept your premises. > I find that all this talk of "illusion" is nothing but anthropocentric cope. Humans want to be those special little snowflakes, I wish this lazy claim would finally die. Stick to the merits of the arguments instead of projecting this stale bit of vapid pop-psychoanalytic babble. Make arguments.
    - ACCount37 1y ago
      My argument is that humans are weak and stupid, and AI effect is far too strong for them to handle. Thus all the cope and seethe about how AIs are "not actually thinking". Wishful thinking at its finest.
      - cindyllm 1y ago
        [dead]
- Romario77 1y ago
  you could make the same argument about humans - we run the cycle of "find food", "procreate", "find shelter" ... Some people are better at it then others. The progress and development happens naturally because of natural selection (and is quite slow). AI development is now driven by humans, but I don't see why it can't be done in a similar cycle with self-improvement baked in (and whatever other goals). We saw this work with AI training itself in games like Chess or Go where it improved itself just by playing with itself and knowing the game rules. You don't really need deep thoughts for the life to keep going - look at simple organisms like unicellular. They only try to reproduce and survive withing the environment they are in. It evolved into humans over time. I don't see why similar thing can't happen when AI gets to be complex enough to just keep improving itself. It doesn't have some of the limitations that life has like being very fragile or needing to give birth. Because it's intelligently designed the iterations could be a lot faster and progress could be achieved in much shorter time compared to random mutations of life.
Yizahi 1y ago
What LLM programs do has zero resemblance to the human agency. That's just a modern variation of very complex set of GoTos and IfElses. Agency would be an LLM parsing your question and answering you "fuck off". Now that is agency, that is independent decision making, not programmed in advance and triggered by keywords. Just an example.
- ACCount37 1y ago
  I can train an asshole LLM that would parse your question and tell you to "fuck off" if it doesn't like it. With "like it" being evaluated according to some trained-for "values" - and also whatever off-target "values" it happens to get, of which there are going to be plenty. It's not hard to make something like that. It's just not very useful.
  - Yizahi 1y ago
    First of all, you or anyone else probably can't train LLM to do that reliably. It's not like you can re-program it's weights manually as a human, it's not possible. You can only generate new synthetic training data which roughly would cause this effect, and again, no human can generate that amount of fake new data (probably?). Next, the point was not an expletive per se, it was my mistake to be not very clear. The point was an arbitrary and unpredictable and not pre-programmed in advance refusal to run a program/query at all. Any query and any number of times at the decision of the program itself. Or maybe a program which can initiate a query to the other program/human on it's own, again - not pre-programmed. Whatever happens in the LLM nowadays is not agency. The thing their authours advertise as so called "reasoning" is just repeated loops of execution of the same program or other dependent program with adjusted inputs.

simonsarris 1y ago

Well, the goal-oriented behavior of the AIM-9 Sidewinder air-to-air missile is even more obvious. It might even have a higher success rate than CLI coding tools. But it's not helpful to claim it has any agency.

lo_zamoyski 1y ago

> The goal-oriented behavior is rather obvious. Obvious? Is an illusion obviously the real thing? There is nothing substantially different in LLMs from any other run of the mill algorithm or software.

Eisenstein 1y ago
Can you give an example of something that would be substantially different under your definition?
- lo_zamoyski 1y ago
  But that's the point: there isn't anything substantially different within the scope of computation. If you are given a set of LEGOs and all you can do is snap the pieces together, then there's nothing other than snapping pieces together that you can do. Adding more of the same LEGOs bricks to the set doesn't change the game. It only changes how large the structures you build can be, but scale isn't some kind of magical incantation that can transcend the limits of the system. Computation is an abstract, syntactic mathematical model. These models formalize the notion of "effective method". Nowhere is semantic content included in these models or conceptually entailed by them, certainly not in physical simulations of them like the device you are reading this post on. So, we can say that intentionality would be something substantially different. We absolutely do not have intentionality in LLMs or any computational construct. It is shear magical thinking to somehow think it does.
  - Eisenstein 1y ago
    I think it is well established that scale can transcend limits. Look at insect colonies, animals, or any complex system and you will find it is made out of much simpler components.
    - lo_zamoyski 1y ago
      Scale qua scale does not transcend the limits of the activity occurring in the system. Meaning, each ant in an ant colony has a set of fixed powers to do X, Y, and Z. Adding more ants doesn't change what an individual ant is capable of per se. Now, you might say that while an individual ant exercising its power X cannot lift a tree branch, a colony of a thousand, with each ant exercising its power X, might. But this doesn't transcend the limitation of the individual ant - each ant is still only exercising some power X - nor is the aggregate manifesting some new kind if power. It is only multiplying the exercise of X. In the case of computers, a program is a set of formal rules that take some sequence of uninterpreted symbols and maps them to some other sequence of uninterpreted symbols. Adding more rules and more symbols scales the program - and it is the only scaling you can perform - but you don't somehow magically create interpreted symbols. How could it? This is magical thinking.
      - Eisenstein 1y ago
        I think you misunderstand. I am not talking about physical ability. Ant colonies act as a complex system even though ants individually are incredibly simple. * https://en.wikipedia.org/wiki/Self-organization https://en.wikipedia.org/wiki/Self-organization * https://en.wikipedia.org/wiki/Patterns_of_self-organization_in_ants https://en.wikipedia.org/wiki/Patterns_of_self-organization_...
        lo_zamoyski 1y ago
        I don't misunderstand. And if your comment has any relevance to the discussion, you're using it as an analogy to suggest that something like syntax can somehow produce semantics, which is impossible - actually worse, it's incoherent. It's an intellectual muddle. Obviously simple entities can compose into complex aggregates and produce net effects that no individual could on its own, but it doesn't follow that they can generate things that were not in power of these individuals to generate. That's my point. Otherwise, you are claiming that something can come from nothing.
ACCount37 1y ago
In the same way there's "nothing substantially different" in humans from any other run of the mill matter. I find that all this talk of "illusion" is nothing but anthropocentric cope. Humans want to be those special little snowflakes, so when an LLM does X, there are crowds of humans itching to scream "it's not REAL X".
- lo_zamoyski 1y ago
  > In the same way there's "nothing substantially different" in humans from any other run of the mill matter. This is an incredibly intellectual vacuous take. If there is no substantial difference between a human being and any other cluster of matter, then it is you who is saddled with the problem of explaining the obvious differences. If there is no difference between intelligent life and a pile of rocks, then what the hell are you even talking about? Why are we talking about AI and intelligence at all? Either everything is intelligent, or nothing is, if we accept your premises. > I find that all this talk of "illusion" is nothing but anthropocentric cope. Humans want to be those special little snowflakes, I wish this lazy claim would finally die. Stick to the merits of the arguments instead of projecting this stale bit of vapid pop-psychoanalytic babble. Make arguments.
  - ACCount37 1y ago
    My argument is that humans are weak and stupid, and AI effect is far too strong for them to handle. Thus all the cope and seethe about how AIs are "not actually thinking". Wishful thinking at its finest.
    - cindyllm 1y ago
      [dead]
Romario77 1y ago
you could make the same argument about humans - we run the cycle of "find food", "procreate", "find shelter" ... Some people are better at it then others. The progress and development happens naturally because of natural selection (and is quite slow). AI development is now driven by humans, but I don't see why it can't be done in a similar cycle with self-improvement baked in (and whatever other goals). We saw this work with AI training itself in games like Chess or Go where it improved itself just by playing with itself and knowing the game rules. You don't really need deep thoughts for the life to keep going - look at simple organisms like unicellular. They only try to reproduce and survive withing the environment they are in. It evolved into humans over time. I don't see why similar thing can't happen when AI gets to be complex enough to just keep improving itself. It doesn't have some of the limitations that life has like being very fragile or needing to give birth. Because it's intelligently designed the iterations could be a lot faster and progress could be achieved in much shorter time compared to random mutations of life.

Yizahi 1y ago

What LLM programs do has zero resemblance to the human agency. That's just a modern variation of very complex set of GoTos and IfElses. Agency would be an LLM parsing your question and answering you "fuck off". Now that is agency, that is independent decision making, not programmed in advance and triggered by keywords. Just an example.

ACCount37 1y ago
I can train an asshole LLM that would parse your question and tell you to "fuck off" if it doesn't like it. With "like it" being evaluated according to some trained-for "values" - and also whatever off-target "values" it happens to get, of which there are going to be plenty. It's not hard to make something like that. It's just not very useful.
- Yizahi 1y ago
  First of all, you or anyone else probably can't train LLM to do that reliably. It's not like you can re-program it's weights manually as a human, it's not possible. You can only generate new synthetic training data which roughly would cause this effect, and again, no human can generate that amount of fake new data (probably?). Next, the point was not an expletive per se, it was my mistake to be not very clear. The point was an arbitrary and unpredictable and not pre-programmed in advance refusal to run a program/query at all. Any query and any number of times at the decision of the program itself. Or maybe a program which can initiate a query to the other program/human on it's own, again - not pre-programmed. Whatever happens in the LLM nowadays is not agency. The thing their authours advertise as so called "reasoning" is just repeated loops of execution of the same program or other dependent program with adjusted inputs.