19 ms·

The first cars broke down all the time. They had a limited range. There wasn't a vast supply of parts for them. There wasn't a vast industry of experts who coul

The first cars broke down all the time. They had a limited range. There wasn't a vast supply of parts for them. There wasn't a vast industry of experts who could work on them. There wasn't a vast network of fuel stations to provide energy for them. The horse was a proven method. What an LLM cannot do today is almost irrelevant in the tide of change upon the industry. The fact is, with improvements, it doesn't mean an LLM cannot do it tomorrow.

skydhash 1y ago
When the first cars broke down, people were not saying: One day, we’ll go to the moon with one of these. LLMs may get better, but it will not be what people are clamoring them to be.
- deleted 1y ago
  [deleted]
- serf 1y ago
  >When the first cars broke down, people were not saying: One day, we’ll go to the moon with one of these. maybe they should have; a lot of the engineering techniques and methodologies that produced the assembly line and the mass produced vehicle also lead the way into space exploration.
jedimastert 1y ago
> The first cars broke down all the time. They had a limited range. There wasn't a vast supply of parts for them. There wasn't a vast industry of experts who could work on them. I mean, there was and then there wasn't. All of those things are shrinking fast because we handed over control to people who care more about profits than customers because we got too comfy and too cheap, and now right to repair is screwed. Honestly, I see llm-driven development as a threat to open source and right to repair, among the litany of other things
Night_Thastus 1y ago
The difference is that the weaknesses of cars were problems of engineering, and some of infrastructure. Both aren't very hard to solve, though they take time. The fundamental way cars operated worked and just needed revision, sanding off rough edges. LLMs are not like this. The fundamental way they operate, the core of their design is faulty. They don't understand rules or knowledge. They can't, despite marketing, really reason. They can't learn with each interaction. They don't understand what they write. All they do is spit out the most likely text to follow some other text based on probability. For casual discussion about well-written topics, that's more than good enough. But for unique problems in a non-English language, it struggles. It always will. It doesn't matter how big you make the model. They're great for writing boilerplate that has been written a million times with different variations - which can save programmers a LOT of time. The moment you hand them anything more complex it's asking for disaster.
- programd 1y ago
  > [LLMs] spit out the most likely text to follow some other text based on probability. Modern coding AI models are not just probability crunching transformers. They haven't been just that for some time. In current coding models the transformer bit is just one part of what is really an expert system. The complete package includes things like highly curated training data, specialized tokenizers, pre and post training regimens, guardrails, optimized system prompts etc, all tuned to coding. Put it all together and you get one shot performance on generating the type of code that was unthinkable even a year ago. The point is that the entire expert system is getting better at a rapid pace and the probability bit is just one part of it. The complexity frontier for code generation keeps moving and there's still a lot of low hanging fruit to be had in pushing it forward. > They're great for writing boilerplate that has been written a million times with different variations That's >90% of all code in the wild. Probably more. We have three quarters of a century of code in our history so there is very little that's original anymore. Maybe original to the human coder fresh out of school, but the models have all this history to draw upon. So if the models produce the boilerplate reliably then human toil in writing if/then statements is at an end. Kind of like - barring the occasional mad genious [0] - the vast majority of coders don't write assembly to create a website anymore. [0] https://asm32.info/index.cgi?page=content/0_MiniMagAsm/index.txt https://asm32.info/index.cgi?page=content/0_MiniMagAsm/index...
tobr 1y ago
The article has a very nuanced point about why it’s not just a matter of today’s vs tomorrow’s LLMs. What’s lacking is a fundamental capacity to build mental models and learn new things specific to the problem at hand. Maybe this can be fixed in theory with some kind of on-the-fly finetuning, but it’s not just about more context.
- ako 1y ago
  You can give it some documents, or classroom textbooks, and it can turn those into rdf graphs, explaining what the main concepts are, and how they are related. This can then be used by an llm to solve other problems. It can also learn new things using trial and error with mcp tools. Once it has figured out some problem, you can ask it to summarize the insights for later use. What would define as an AI mental model?
  - tobr 1y ago
    I’m not an expert on this, so I’m not familiar with what RDF graphs are, but I feel like everything you’re describing happens textually, and used as context? That is, it’s not at all ”learning” the way it’s learning during training, but by writing things down to refer to them later? As you say - ”ask it to summarize the insights for later use” - this is fundamentally different from the types of ”insights” it can have during training. So, it can take notes about your code and refer back to them, but it only has meaningful ”knowledge” about code it came across in training. To me as a layman, this feels like a clear explanation of how these tools break down, why they start going in circles when you reach a certain complexity, why they make a mess of unusual requirements, and why they have such an incredible nuanced grasp of complex ideas that are widely publicized, while being unable to draw basic conclusions about specific constraints in your project.
    - ako 1y ago
      To me it feels very much like a brain: my brain often lacks knowledge, but i can use external documents to augment it. My brain also has limitations in what it can remember, I hardly remember anything I learned in high school or university on science, chemistry, math, so I need to write things down to bring back knowledge later. Text and words are the concepts we use to transfer knowledge in schools, across generations, etc. we describe concepts in words, so other people can learn these concepts. Without words and text we would be like animals unable to express and think about concepts
jerf 1y ago
AI != LLM. We can reasonably speak about certain fundamental limitations of LLMs without those being claims about what AI may ever do. I would agree they fundamentally lack models of the current task and that it is not very likely that continually growing the context will solve that problem, since it hasn't already. That doesn't mean there won't someday be an AI that has a model much as we humans do. But I'm fairly confident it won't be an LLM. It may have an LLM as a component but the AI component won't be primarily an LLM. It'll be something else.
- byteknight 1y ago
  I have to disagree. Anyone that says LLMs do not qualify as AI are the same people who will continue to move the goal posts for AGI. "Well it doesn't do this!". No one here is trying to replicate a human brain or condition in its entirety. They just want to replicate the thinking ability of one. LLMs represent the closest parallel we have experienced thus far to that goal. Saying that LLMs are not AI feel disingenuous at best and entirely purposely dishonest at the worst (perhaps perceived as staving off the impending demise of a profession). The sooner people stop worrying about a label for what you feel fits LLMs best, the sooner they can find the things they (LLMs) absolutely excel at and improve their (the user's) workflows. Stop fighting the future. Its not replacing right now. Later? Maybe. But right now the developers and users fully embracing it are experiencing productivity boosts unseen previously. Language is what people use it as.
  - sarchertech 1y ago
    > the developers and users fully embracing it are experiencing productivity boosts unseen previously This is the kind of thing that I disagree with. Over the last 75 years we’ve seen enormous productivity gains. You think that LLMs are a bigger productivity boost than moving from physically rewiring computers to using punch cards, from running programs as batch processes with printed output to getting immediate output, from programming in assembly to higher level languages, or even just moving from enterprise Java to Rails?
    - Espressosaurus 1y ago
aaroninsf 1y ago
My preferred formulation is Ximm's Law, "Every critique of AI assumes to some degree that contemporary implementations will not, or cannot, be improved upon. Lemma: any statement about AI which uses the word "never" to preclude some feature from future realization is false. Lemma: contemporary implementations have almost always already been improved upon, but are unevenly distributed."
- moregrist 1y ago
  Replace “AI” with “fusion” and you immediately see the problem: there’s no concept of timescale or cost. And with fusion, we already have a working prototype (the Sun). And if we could just scale our tech up enough, maybe we’d have usable fusion.
  - dpatterbee 1y ago
    Heck, replace "AI" with almost any noun and you can close your eyes to any and all criticism!
    - gjm11 1y ago
      Only to criticism of the form "X can never ...", and some such criticism richly deserves to be ignored. (Sometimes that sort of criticism is spot on. If someone says they've got a brilliant new design for a perpetual motion machine, go ahead and tell them it'll never work. But in the general case it's overconfident.)
- latexr 1y ago
  > Every critique of AI assumes to some degree that contemporary implementations will not, or cannot, be improved upon. That is too reductive and simply not true. Contemporary critiques of AI include that they waste precious resources (such as water and energy) and accelerate bad environmental and societal outcomes (such as climate change, the spread of misinformation, loss of expertise), among others. Critiques go far beyond “hur dur, LLM can’t code good”, and those problems are both serious and urgent. Keep sweeping critiques under the rug because “they’ll be solved in the next five years” (eternally away) and it may be too late. Critiques have to take into account the now and the very real repercussions already happening.
brandon272 1y ago
The question is, when is “tomorrow”? Dismissing a concern with “LLMs/AI can’t do it today but they will probably be able to do it tomorrow” isn’t all that useful or helpful when “tomorrow” in this context could just as easily be “two months from now” or “50 years from now”.
apwell23 1y ago
ugh.. no analogies pls
card_zero 1y ago
When monowheels were first invented, they were very difficult to steer due to the gyroscopic effects inherent to a large wheel model (LWM).
ajuc 1y ago
It also doesn't mean they can. LLMs may be the steam-powered planes of our times. A crucial ingredient might be missing.
ants_everywhere 1y ago
The anti-LLM chorus hates when you bring up the history of technological change
dml2135 1y ago
This is like saying that because of all the advancements that automobiles have made, teleportation is right around the corner.
shalmanese 1y ago
The analogy is very apt because the first cars: * are many times the size of the occupants, greatly constricting throughput. * are many times heavier than humans, requiring vastly more energy to move. * travel at speeds and weights that are danger to humans, thus requiring strictly segregated spaces. * are only used less than 5% of the day, requiring places to store them when unused. * require extremely wide turning radiuses when traveling at speed (there’s a viral photo showing the entire historical city of Florence fit inside a single US cloverleaf interchange) Not only have none of these flaws been fixed, many of them have gotten worse with advancing technology because they’re baked into the nature of cars. Anyone at the invention of automobiles with sufficient foresight could have seen the intersecting incentives that cars would wreak, same as how many of the future impacts of LLMs are foreseeable today, independent of technical progress.
- oblio 1y ago
  > Anyone at the invention of automobiles with sufficient foresight could have seen the intersecting incentives that cars would wreak, same as how many of the future impacts of LLMs are foreseeable today, independent of technical progress. Yeah, but where's the money to be made in not selling people stuff? https://imgur.com/few-shareholders-had-good-value-least-jpsPfCt https://imgur.com/few-shareholders-had-good-value-least-jpsP...
windward 1y ago
How do you differentiate between tech that's 'first cars' and tech that's 'first passenger supersonic aircraft'?

skydhash 1y ago

When the first cars broke down, people were not saying: One day, we’ll go to the moon with one of these. LLMs may get better, but it will not be what people are clamoring them to be.

deleted 1y ago
[deleted]
serf 1y ago
>When the first cars broke down, people were not saying: One day, we’ll go to the moon with one of these. maybe they should have; a lot of the engineering techniques and methodologies that produced the assembly line and the mass produced vehicle also lead the way into space exploration.

jedimastert 1y ago

> The first cars broke down all the time. They had a limited range. There wasn't a vast supply of parts for them. There wasn't a vast industry of experts who could work on them. I mean, there was and then there wasn't. All of those things are shrinking fast because we handed over control to people who care more about profits than customers because we got too comfy and too cheap, and now right to repair is screwed. Honestly, I see llm-driven development as a threat to open source and right to repair, among the litany of other things

Night_Thastus 1y ago

The difference is that the weaknesses of cars were problems of engineering, and some of infrastructure. Both aren't very hard to solve, though they take time. The fundamental way cars operated worked and just needed revision, sanding off rough edges. LLMs are not like this. The fundamental way they operate, the core of their design is faulty. They don't understand rules or knowledge. They can't, despite marketing, really reason. They can't learn with each interaction. They don't understand what they write. All they do is spit out the most likely text to follow some other text based on probability. For casual discussion about well-written topics, that's more than good enough. But for unique problems in a non-English language, it struggles. It always will. It doesn't matter how big you make the model. They're great for writing boilerplate that has been written a million times with different variations - which can save programmers a LOT of time. The moment you hand them anything more complex it's asking for disaster.

programd 1y ago
> [LLMs] spit out the most likely text to follow some other text based on probability. Modern coding AI models are not just probability crunching transformers. They haven't been just that for some time. In current coding models the transformer bit is just one part of what is really an expert system. The complete package includes things like highly curated training data, specialized tokenizers, pre and post training regimens, guardrails, optimized system prompts etc, all tuned to coding. Put it all together and you get one shot performance on generating the type of code that was unthinkable even a year ago. The point is that the entire expert system is getting better at a rapid pace and the probability bit is just one part of it. The complexity frontier for code generation keeps moving and there's still a lot of low hanging fruit to be had in pushing it forward. > They're great for writing boilerplate that has been written a million times with different variations That's >90% of all code in the wild. Probably more. We have three quarters of a century of code in our history so there is very little that's original anymore. Maybe original to the human coder fresh out of school, but the models have all this history to draw upon. So if the models produce the boilerplate reliably then human toil in writing if/then statements is at an end. Kind of like - barring the occasional mad genious [0] - the vast majority of coders don't write assembly to create a website anymore. [0] https://asm32.info/index.cgi?page=content/0_MiniMagAsm/index.txt https://asm32.info/index.cgi?page=content/0_MiniMagAsm/index...

tobr 1y ago

The article has a very nuanced point about why it’s not just a matter of today’s vs tomorrow’s LLMs. What’s lacking is a fundamental capacity to build mental models and learn new things specific to the problem at hand. Maybe this can be fixed in theory with some kind of on-the-fly finetuning, but it’s not just about more context.

ako 1y ago
You can give it some documents, or classroom textbooks, and it can turn those into rdf graphs, explaining what the main concepts are, and how they are related. This can then be used by an llm to solve other problems. It can also learn new things using trial and error with mcp tools. Once it has figured out some problem, you can ask it to summarize the insights for later use. What would define as an AI mental model?
- tobr 1y ago
  I’m not an expert on this, so I’m not familiar with what RDF graphs are, but I feel like everything you’re describing happens textually, and used as context? That is, it’s not at all ”learning” the way it’s learning during training, but by writing things down to refer to them later? As you say - ”ask it to summarize the insights for later use” - this is fundamentally different from the types of ”insights” it can have during training. So, it can take notes about your code and refer back to them, but it only has meaningful ”knowledge” about code it came across in training. To me as a layman, this feels like a clear explanation of how these tools break down, why they start going in circles when you reach a certain complexity, why they make a mess of unusual requirements, and why they have such an incredible nuanced grasp of complex ideas that are widely publicized, while being unable to draw basic conclusions about specific constraints in your project.
  - ako 1y ago
    To me it feels very much like a brain: my brain often lacks knowledge, but i can use external documents to augment it. My brain also has limitations in what it can remember, I hardly remember anything I learned in high school or university on science, chemistry, math, so I need to write things down to bring back knowledge later. Text and words are the concepts we use to transfer knowledge in schools, across generations, etc. we describe concepts in words, so other people can learn these concepts. Without words and text we would be like animals unable to express and think about concepts

jerf 1y ago

AI != LLM. We can reasonably speak about certain fundamental limitations of LLMs without those being claims about what AI may ever do. I would agree they fundamentally lack models of the current task and that it is not very likely that continually growing the context will solve that problem, since it hasn't already. That doesn't mean there won't someday be an AI that has a model much as we humans do. But I'm fairly confident it won't be an LLM. It may have an LLM as a component but the AI component won't be primarily an LLM. It'll be something else.

byteknight 1y ago
I have to disagree. Anyone that says LLMs do not qualify as AI are the same people who will continue to move the goal posts for AGI. "Well it doesn't do this!". No one here is trying to replicate a human brain or condition in its entirety. They just want to replicate the thinking ability of one. LLMs represent the closest parallel we have experienced thus far to that goal. Saying that LLMs are not AI feel disingenuous at best and entirely purposely dishonest at the worst (perhaps perceived as staving off the impending demise of a profession). The sooner people stop worrying about a label for what you feel fits LLMs best, the sooner they can find the things they (LLMs) absolutely excel at and improve their (the user's) workflows. Stop fighting the future. Its not replacing right now. Later? Maybe. But right now the developers and users fully embracing it are experiencing productivity boosts unseen previously. Language is what people use it as.
- sarchertech 1y ago
  > the developers and users fully embracing it are experiencing productivity boosts unseen previously This is the kind of thing that I disagree with. Over the last 75 years we’ve seen enormous productivity gains. You think that LLMs are a bigger productivity boost than moving from physically rewiring computers to using punch cards, from running programs as batch processes with printed output to getting immediate output, from programming in assembly to higher level languages, or even just moving from enterprise Java to Rails?
  - Espressosaurus 1y ago

aaroninsf 1y ago

My preferred formulation is Ximm's Law, "Every critique of AI assumes to some degree that contemporary implementations will not, or cannot, be improved upon. Lemma: any statement about AI which uses the word "never" to preclude some feature from future realization is false. Lemma: contemporary implementations have almost always already been improved upon, but are unevenly distributed."

moregrist 1y ago
Replace “AI” with “fusion” and you immediately see the problem: there’s no concept of timescale or cost. And with fusion, we already have a working prototype (the Sun). And if we could just scale our tech up enough, maybe we’d have usable fusion.
- dpatterbee 1y ago
  Heck, replace "AI" with almost any noun and you can close your eyes to any and all criticism!
  - gjm11 1y ago
    Only to criticism of the form "X can never ...", and some such criticism richly deserves to be ignored. (Sometimes that sort of criticism is spot on. If someone says they've got a brilliant new design for a perpetual motion machine, go ahead and tell them it'll never work. But in the general case it's overconfident.)
latexr 1y ago
> Every critique of AI assumes to some degree that contemporary implementations will not, or cannot, be improved upon. That is too reductive and simply not true. Contemporary critiques of AI include that they waste precious resources (such as water and energy) and accelerate bad environmental and societal outcomes (such as climate change, the spread of misinformation, loss of expertise), among others. Critiques go far beyond “hur dur, LLM can’t code good”, and those problems are both serious and urgent. Keep sweeping critiques under the rug because “they’ll be solved in the next five years” (eternally away) and it may be too late. Critiques have to take into account the now and the very real repercussions already happening.

brandon272 1y ago

The question is, when is “tomorrow”? Dismissing a concern with “LLMs/AI can’t do it today but they will probably be able to do it tomorrow” isn’t all that useful or helpful when “tomorrow” in this context could just as easily be “two months from now” or “50 years from now”.

apwell23 1y ago

ugh.. no analogies pls

card_zero 1y ago

When monowheels were first invented, they were very difficult to steer due to the gyroscopic effects inherent to a large wheel model (LWM).

ajuc 1y ago

It also doesn't mean they can. LLMs may be the steam-powered planes of our times. A crucial ingredient might be missing.

ants_everywhere 1y ago

The anti-LLM chorus hates when you bring up the history of technological change

dml2135 1y ago

This is like saying that because of all the advancements that automobiles have made, teleportation is right around the corner.

shalmanese 1y ago

The analogy is very apt because the first cars: * are many times the size of the occupants, greatly constricting throughput. * are many times heavier than humans, requiring vastly more energy to move. * travel at speeds and weights that are danger to humans, thus requiring strictly segregated spaces. * are only used less than 5% of the day, requiring places to store them when unused. * require extremely wide turning radiuses when traveling at speed (there’s a viral photo showing the entire historical city of Florence fit inside a single US cloverleaf interchange) Not only have none of these flaws been fixed, many of them have gotten worse with advancing technology because they’re baked into the nature of cars. Anyone at the invention of automobiles with sufficient foresight could have seen the intersecting incentives that cars would wreak, same as how many of the future impacts of LLMs are foreseeable today, independent of technical progress.

oblio 1y ago
> Anyone at the invention of automobiles with sufficient foresight could have seen the intersecting incentives that cars would wreak, same as how many of the future impacts of LLMs are foreseeable today, independent of technical progress. Yeah, but where's the money to be made in not selling people stuff? https://imgur.com/few-shareholders-had-good-value-least-jpsPfCt https://imgur.com/few-shareholders-had-good-value-least-jpsP...

windward 1y ago

How do you differentiate between tech that's 'first cars' and tech that's 'first passenger supersonic aircraft'?