3 ms·

This just shows that the models (not AI, statistical models of text used without consent) are not that smart, it's the tooling around them which allows using th

This just shows that the models (not AI, statistical models of text used without consent) are not that smart, it's the tooling around them which allows using these models as a heuristic for brute force search of the solution space. Just last week, I prompted (not asked, it is not sentient) Claude to generate (not tell me or find out or any other anthropomorphization) whether I need to call Dispose on objects passed to me from 2 different libraries for industrial cameras. Being industrial, most people using them typically don't post their code publicly, which means the models have poor statistical coverage around these topics. The LLM generated a response which triggered the tooling around it to perform dozens of internet searches and based on my initial prompt, the search results and lots of intermediate tokens ("thinking"), generated a reply which said that yes, I need to call Dispose in both cases. It was phrased authoritatively and confidently. So I tried it, one library segfaulted, the other returned an exception on a later call. I performed my own internet search (a single one) and immediately found documentation from one of the libraries clearly stating I don't need to call Dispose. The other library being much more poorly documented didn't mention this explicitly but had examples which didn't call Dispose. I am sure if I used LLMs "properly" "agentically", then they would have triggered the tooling around them to build and execute the code, gotten the same results as me much faster, then equally authoritatively and confidently stated that I don't need to call Dispose. This is not thinking. It's a form of automation but not thinking and not intelligence.

Isamu 9mo ago
>brute force search of the solution space “Brute force” is mostly what makes it all work, and what is most disappointing to me currently. Including the brute force necessary to train an LLM, the vast quantity of text necessary to approach almost human quality, the massive scale of data centers necessary to deploy these models, etc. I am hoping this is a transitional period, where LLMs could be used to create better models that are more finesse and less brute force.
- martin-t 9mo ago
  To be honest, these models being bad is what gives me some hope we can figure out how to approach a potential future AI as a society before it arrives. Because right now everything in the west is structured around rich people owning things they have not built while people who did the actual work with their hands and their minds are left in the dust. For a brief period of time (a couple decades), tech was a path for anyone from any background to get at least enough to not struggle. Not become truly rich as for that you need to own real estate or companies but having all your reasonable material needs taken care of and being able to save up for retirement (or in countries without free education, to pay for kids' college). And that might be coming to an end, with people who benefited from this opportunity cheering it on.
  - logicprog 9mo ago
    I make $1500/mo, working part time for a friend being vastly underpaid for technical work and part time as a meat packer in the back of a supermarket, because I had to drop out of university a year before finishing due to a disability and loss of financial resources, and thanks to being trans, disabled, and a dropout, as well as the job market that's fucked up due to AI and all the other reasons for layoffs, so I can't get a tech job, despite having been programming since I was seven and being very good at it. I don't think it's really fair to talk about "people who benefited from this opportunity cheering it on" in the comments on one of my posts. I'm an agentic AI coding enthusiast because I find it fascinating, it allows me to focus more on what I like most about programming (software architecture, systems thinking, etc), and the decreased cognitive load and increased productivity allows me to continue to do interesting projects in the time and energy I have left after my jobs and disability take.
    - martin-t 9mo ago
      The current crop of genML criticism is a false dichotomy. People either focus on the models not meeting their standards (which is very subject matter dependent) or being exploitative. The thing is, both can be true at the same time and the models can in theory get good enough. Just like the water usage argument - I am not saying it's right or wrong, I am saying it can change if somebody comes up with more energy efficient models. We should focus on the fundamentals. And those are that LLMs are build on the work of millions (hundreds of millions?) of people without their consent and that their goal is to reduce or completely eliminate the economic (and therefore political) power of those people. I understand why you do it, I use stolen code too because ultimately nobody is likely to be punished and I'd just be disadvantaging myself for no reason. But we need to think about the endgame - not ours - theirs - because theirs dictates ours.
ozozozd 9mo ago
What a sober and accurate observation of the real capabilities of LLMs. And it’s nothing to sneeze at because it allows me to stay in the terminal rather than go back and forth between the terminal and Google.
- martin-t 9mo ago
  Have you heard about "run or raise" scripts/utilities? I have each program i use regularly bound to a keyboard shortcut and the search bar is ctrl+k in most browsers. If we're talking purely about the time saving of not having to open/focus another program and its search bar, than those costs can be negligible.
TonyStr 9mo ago
Yes, I think you are spot on. I've been toying with Claude Code recently to counter my own bias against agentic coding. It will confidently create a broken project, run it, read the error messages, fix it, run it, read the error messages and keep going until it runs. I used it to create a firefox addon, which meant that it invoked me very frequently to validate its output. This was much more tedious than letting it work on problems that it could validate with the console. It also kinda sucks at googling and looking up documentation. AI "reasoning" in it's current state is a hack meant to overcome the problem of contextual learning[0]. It somewhat works given enough time and good automatic tooling. When this problem is solved, I think we will see a significant boost in productivity from these tools. In it's current state, I'm not convinced that they are worth my time (and money). [0] - https://hy.tencent.com/research/100025?langVersion=en https://hy.tencent.com/research/100025?langVersion=en
logicprog 9mo ago
> I am sure if I used LLMs "properly" "agentically", then they would have triggered the tooling around them to build and execute the code, gotten the same results as me much faster, then equally authoritatively and confidently stated that I don't need to call Dispose. Yes, usually my agents directly read the source code of libraries that don't have lots of good documentation or information in their training data, and/or create test programs as minimal viable examples and compile and run them themselves to see what happens, it's quite useful. But you're right overall; LLMs placed inside agents are essentially providing a sort of highly steerable plausible prior for a genetic algorithm to automatically solve problems and do automation tasks. It's not as brute force as a classic genetic algorithm, but it can't always one-shot things, there is sometimes an element of guess-and-check. But IME at least that element is usually not more iterations than it takes me to figure something out (2-3), on average, although sometimes it needs more iterations than I would've on simple problems, and other times much less on harder ones, or vice versa.
- martin-t 9mo ago
  > usually my agents directly read the source code of libraries that don't have lots of good documentation That only works if the source is available. And since software licenses are meaningless now, we might see more and more closed source software.

Isamu 9mo ago

>brute force search of the solution space “Brute force” is mostly what makes it all work, and what is most disappointing to me currently. Including the brute force necessary to train an LLM, the vast quantity of text necessary to approach almost human quality, the massive scale of data centers necessary to deploy these models, etc. I am hoping this is a transitional period, where LLMs could be used to create better models that are more finesse and less brute force.

martin-t 9mo ago
To be honest, these models being bad is what gives me some hope we can figure out how to approach a potential future AI as a society before it arrives. Because right now everything in the west is structured around rich people owning things they have not built while people who did the actual work with their hands and their minds are left in the dust. For a brief period of time (a couple decades), tech was a path for anyone from any background to get at least enough to not struggle. Not become truly rich as for that you need to own real estate or companies but having all your reasonable material needs taken care of and being able to save up for retirement (or in countries without free education, to pay for kids' college). And that might be coming to an end, with people who benefited from this opportunity cheering it on.
- logicprog 9mo ago
  I make $1500/mo, working part time for a friend being vastly underpaid for technical work and part time as a meat packer in the back of a supermarket, because I had to drop out of university a year before finishing due to a disability and loss of financial resources, and thanks to being trans, disabled, and a dropout, as well as the job market that's fucked up due to AI and all the other reasons for layoffs, so I can't get a tech job, despite having been programming since I was seven and being very good at it. I don't think it's really fair to talk about "people who benefited from this opportunity cheering it on" in the comments on one of my posts. I'm an agentic AI coding enthusiast because I find it fascinating, it allows me to focus more on what I like most about programming (software architecture, systems thinking, etc), and the decreased cognitive load and increased productivity allows me to continue to do interesting projects in the time and energy I have left after my jobs and disability take.
  - martin-t 9mo ago
    The current crop of genML criticism is a false dichotomy. People either focus on the models not meeting their standards (which is very subject matter dependent) or being exploitative. The thing is, both can be true at the same time and the models can in theory get good enough. Just like the water usage argument - I am not saying it's right or wrong, I am saying it can change if somebody comes up with more energy efficient models. We should focus on the fundamentals. And those are that LLMs are build on the work of millions (hundreds of millions?) of people without their consent and that their goal is to reduce or completely eliminate the economic (and therefore political) power of those people. I understand why you do it, I use stolen code too because ultimately nobody is likely to be punished and I'd just be disadvantaging myself for no reason. But we need to think about the endgame - not ours - theirs - because theirs dictates ours.

ozozozd 9mo ago

What a sober and accurate observation of the real capabilities of LLMs. And it’s nothing to sneeze at because it allows me to stay in the terminal rather than go back and forth between the terminal and Google.

martin-t 9mo ago
Have you heard about "run or raise" scripts/utilities? I have each program i use regularly bound to a keyboard shortcut and the search bar is ctrl+k in most browsers. If we're talking purely about the time saving of not having to open/focus another program and its search bar, than those costs can be negligible.

TonyStr 9mo ago

Yes, I think you are spot on. I've been toying with Claude Code recently to counter my own bias against agentic coding. It will confidently create a broken project, run it, read the error messages, fix it, run it, read the error messages and keep going until it runs. I used it to create a firefox addon, which meant that it invoked me very frequently to validate its output. This was much more tedious than letting it work on problems that it could validate with the console. It also kinda sucks at googling and looking up documentation. AI "reasoning" in it's current state is a hack meant to overcome the problem of contextual learning[0]. It somewhat works given enough time and good automatic tooling. When this problem is solved, I think we will see a significant boost in productivity from these tools. In it's current state, I'm not convinced that they are worth my time (and money). [0] - https://hy.tencent.com/research/100025?langVersion=en https://hy.tencent.com/research/100025?langVersion=en

logicprog 9mo ago

> I am sure if I used LLMs "properly" "agentically", then they would have triggered the tooling around them to build and execute the code, gotten the same results as me much faster, then equally authoritatively and confidently stated that I don't need to call Dispose. Yes, usually my agents directly read the source code of libraries that don't have lots of good documentation or information in their training data, and/or create test programs as minimal viable examples and compile and run them themselves to see what happens, it's quite useful. But you're right overall; LLMs placed inside agents are essentially providing a sort of highly steerable plausible prior for a genetic algorithm to automatically solve problems and do automation tasks. It's not as brute force as a classic genetic algorithm, but it can't always one-shot things, there is sometimes an element of guess-and-check. But IME at least that element is usually not more iterations than it takes me to figure something out (2-3), on average, although sometimes it needs more iterations than I would've on simple problems, and other times much less on harder ones, or vice versa.

martin-t 9mo ago
> usually my agents directly read the source code of libraries that don't have lots of good documentation That only works if the source is available. And since software licenses are meaningless now, we might see more and more closed source software.