3 ms·

Thank you kindly I’m working through thoughts on this as well and agree with your read on the incentives. There is an interesting set of conditions that happe

Thank you kindly I’m working through thoughts on this as well and agree with your read on the incentives. There is an interesting set of conditions that happens if/when models get so competent that they’re effectively indistinguishable from each other and inference becomes a true commodity. IRL impact will lag this ofc but it’s such a wild time to be alive.

abalashov 3mo ago
I do think it's easy, in this technology discussion bubble in which we dwell, to overestimate the centrality of LLMs to the arc of developments in our time. They'll be important, but I don't think they'll be _that_ important, because the rest of society and the economy don't move at the speed of SV. Instead, they'll be overtaken by other, more traditional categories of events, ruptures and dislocations. Moreover, folks will eventually realise that while they are very impressive derivative databases of knowledge, they're not at all "AI" -- well, not the "I" part, anyway -- as the concept is traditionally understood. There's not any "I". It emits convolutions of its training, and it does so very impressively, and that can even be harnessed by agents to connect them to levers, servos and richer information sources. It's nifty. But it's just not intelligence. It's more of a kind of queryable database than a robot. Once that realisation diffuses more widely, I think it'll turn out to be a more prosaic and underwhelming development than is presently hypothesised, either here or by the press. It doesn't mean many business and managerial class folks won't try to squeeze everything they can out of so-called AI, but the idea that this can effectuate truly widespread labour displacement will probably quiet down considerably. (The valuations that depend on this assumption may collapse more abruptly and less gracefully.) The challenge is staying solvent until then. :-)
- antonvs 3mo ago
  Functionally speaking, current models are intelligent by any reasonable definition of the term. You can test them on their understanding of complex domains such as software systems, and within that domain as an example, you can judge them on their ability to diagnose bugs, fix bugs, explain code to humans, design systems given vague specifications, and implement new systems. On all these tasks, current models are measurably superhuman compared to the average human software developer. You could make a better case that it’s humans that don’t possess true intelligence.
  - anon373839 3mo ago
    I don’t agree. I think they clearly lack general intelligence, which is why all the AI companies can think of is sourcing more and more niche domain data to plug more and more holes. When you get enough of that stuff, you can get a convincing illusion of general intelligence, but there is always another car wash test coming that shows it isn’t real.
    - antonvs 3mo ago
      What you’re doing is the classic No True Scotsman fallacy. “Isn’t real” is a tell for that. It’s the same as saying it’s not a “true” intelligence. Define what you mean by “true”, then. But for me, what I’m interested in is functional capability, not some mysterious ineffable quality that only humans can have. And in terms of functional capability, current models are certainly better at software development than you are. You’re just in denial.
      - anon373839 3mo ago
        Perhaps I was unclear: the part that isn't real is the generalization. The models appear to generalize because they're fitted to so many discrete tasks that it almost doesn't appear to matter. But then it leaks, and the failure modes reveal no coherent model or process that generated the failure. The labs only have one answer for this, which is more duct tape. > current models are certainly better at software development than you are. You’re just in denial. I have no ego in this. It could be true. Wikipedia is also "smarter" than I am; it "knows" so many more concepts than I could ever. But regardless, I think the state of slopcoded messes like Claude Code shows that the models are missing something.
        abalashov 3mo ago
        Although I agree with you, in fairness, there are some lively controversies in the world of cognitive science and philosophy of mind about whether this meaningfully differs from human thinking at sufficient scale. The general idea is that the building blocks of "coherent models" and "processes" within the churning of the human intellect are also, in so many words, prior art and existing concepts, and so, while the human mind is not a text model, a sufficiently large and sensorily multimodal neural net would not be too different. Neural nets are, after all, inspired by what we understand of human cognition -- they'd say.

abalashov 3mo ago

I do think it's easy, in this technology discussion bubble in which we dwell, to overestimate the centrality of LLMs to the arc of developments in our time. They'll be important, but I don't think they'll be _that_ important, because the rest of society and the economy don't move at the speed of SV. Instead, they'll be overtaken by other, more traditional categories of events, ruptures and dislocations. Moreover, folks will eventually realise that while they are very impressive derivative databases of knowledge, they're not at all "AI" -- well, not the "I" part, anyway -- as the concept is traditionally understood. There's not any "I". It emits convolutions of its training, and it does so very impressively, and that can even be harnessed by agents to connect them to levers, servos and richer information sources. It's nifty. But it's just not intelligence. It's more of a kind of queryable database than a robot. Once that realisation diffuses more widely, I think it'll turn out to be a more prosaic and underwhelming development than is presently hypothesised, either here or by the press. It doesn't mean many business and managerial class folks won't try to squeeze everything they can out of so-called AI, but the idea that this can effectuate truly widespread labour displacement will probably quiet down considerably. (The valuations that depend on this assumption may collapse more abruptly and less gracefully.) The challenge is staying solvent until then. :-)

antonvs 3mo ago
Functionally speaking, current models are intelligent by any reasonable definition of the term. You can test them on their understanding of complex domains such as software systems, and within that domain as an example, you can judge them on their ability to diagnose bugs, fix bugs, explain code to humans, design systems given vague specifications, and implement new systems. On all these tasks, current models are measurably superhuman compared to the average human software developer. You could make a better case that it’s humans that don’t possess true intelligence.
- anon373839 3mo ago
  I don’t agree. I think they clearly lack general intelligence, which is why all the AI companies can think of is sourcing more and more niche domain data to plug more and more holes. When you get enough of that stuff, you can get a convincing illusion of general intelligence, but there is always another car wash test coming that shows it isn’t real.
  - antonvs 3mo ago
    What you’re doing is the classic No True Scotsman fallacy. “Isn’t real” is a tell for that. It’s the same as saying it’s not a “true” intelligence. Define what you mean by “true”, then. But for me, what I’m interested in is functional capability, not some mysterious ineffable quality that only humans can have. And in terms of functional capability, current models are certainly better at software development than you are. You’re just in denial.
    - anon373839 3mo ago
      Perhaps I was unclear: the part that isn't real is the generalization. The models appear to generalize because they're fitted to so many discrete tasks that it almost doesn't appear to matter. But then it leaks, and the failure modes reveal no coherent model or process that generated the failure. The labs only have one answer for this, which is more duct tape. > current models are certainly better at software development than you are. You’re just in denial. I have no ego in this. It could be true. Wikipedia is also "smarter" than I am; it "knows" so many more concepts than I could ever. But regardless, I think the state of slopcoded messes like Claude Code shows that the models are missing something.
      - abalashov 3mo ago
        Although I agree with you, in fairness, there are some lively controversies in the world of cognitive science and philosophy of mind about whether this meaningfully differs from human thinking at sufficient scale. The general idea is that the building blocks of "coherent models" and "processes" within the churning of the human intellect are also, in so many words, prior art and existing concepts, and so, while the human mind is not a text model, a sufficiently large and sensorily multimodal neural net would not be too different. Neural nets are, after all, inspired by what we understand of human cognition -- they'd say.