4 ms·

I was slightly surprised that my colleagues, who are extremely invested in capabilities of LLMs, didn’t show any interest in Karpathy’s communication on the sub

by kubb 1y ago

I was slightly surprised that my colleagues, who are extremely invested in capabilities of LLMs, didn’t show any interest in Karpathy’s communication on the subject when I recommended it to them. Later I understood that they don’t need to understand LLMs, and they don’t care how they work. Rather they need to believe and buy into them. They’re more interested in science fiction discussions — how would we organize a society where all work is done by intelligent machines — than what kinds of tasks are LLMs good at today and why.

teiferer 1y ago
Which is terrible. That's the root of all the BS around LLMs. People lacking understanding of what they are and ascribing capabilities which LLMs just don't have, by design. Even HN discussions are full of that. Even though this page literally has "hacker" in its name.
- kubb 1y ago
  I’m trying not to be disappointed by people, I’d rather understand what’s going on in their minds, and how to navigate that.
  - deleted 1y ago
    [deleted]
- tim333 1y ago
  I see your point but on the other hand a lot of conversations go: A: what will we do when AI do all the jobs, B: that's silly LLMs can't do the jobs. The thing is A didn't say LLM, they said AI as in whatever that will be a short while into the future. Which is changing rapidly because thousands of bright people are being paid to change it.
  - HarHarVeryFunny 1y ago
    The trouble is that "AI" is also very much a leaky abstraction, which makes it tempting to see all the "AI" advances of recent years, then correctly predict that these "AI" advances will continue, but then jump to all sorts of wrong conclusions about what those advances will be. For example, things like "AI" image and video generation are amazing, as are things like AlphaGo and AlphaFold, but none of these have anything to do with LLMs, and the only technology they share with LLMs is machine learning and neural nets. If you lump these together with LLMs, calling them all "AI", then you'll come to the wrong conclusion that all of these non-LLM advances indicate that "AI" is rapidly advancing and therefore LLMs (also being "AI") will do too ... Even if you leave aside things like AlphaGo, and just focus on LLMs, and other future technology that may take all our jobs, then using terms like "AI" and "AGI" are still confusing and misleading. It's easy to fall into the mindset that "AGI" is just better "AI", and that since LLMs are "AI", AGI is just better LLMs, and is around the corner because "AI" is advancing rapidly ... In reality LLMs are, like AlphaFold, something highly specific - they are auto-regressive next-word predictor language models (just as a statement of fact, and how they are trained, not a put-down), based on the Transformer architecture. The technology that could replace humans for most jobs in the future isn't going to be a better language model - a better auto-regressive next-word predictor - but will need to be something much more brain like. The architecture itself doesn't have to be brain-like, but in order to deliver brain-like functionality it will probably need to include another half-dozen "Transformer-level" architectural/algorithmic breakthroughs including things like continual learning, which will likely turn the whole current LLM training and deployment paradigm on it's head. Again, just focusing on LLMs, and LLM-based agents, regarding them as a black-box technology, it's easy to be misled into thinking that advances in capability are broadly advancing, and will rise all ships, when in reality progress is much more narrow. Headlines about LLMs achievement in math and competitive programming, touted as evidence of reasoning, do NOT imply that LLM reasoning is broadly advancing, but you need to get under the hood and understand RL training goals to realize why that is not necessarily the case. The correctness of most business and real-world reasoning is not as easy to check as is marking a math problem as correct or not, yet that capability is what RL training depends on. I could go on .. LLM-based agents are also blurring the lines of what "AI" can do, and again if treated as a black box will also misinform as to what is actually progressing and what is not. Thousands of bright people are indeed working on improving LLM-adjacent low-hanging fruit like this, but it'd be illogical to conclude that this is somehow helping to create next-generation brain-like architectures that will take away our jobs.
  - teiferer 1y ago
    > a short while into the future And what gives you that confidence? A few AI nerds already claimed that in the 80s. We're currently exploring what LLMs can do. There is no indication that any further fundamental breakthrough is around the corner. Everybody is currently squeezing the same stone.
Al-Khwarizmi 1y ago
What's wrong or odd about that? You can like a technology as a user and not want to delve into how it works (sentence written by a human despite use of "delve"). Everyone should have some notions on what LLMs can or cannot do, in order to use them successfully and not be misguided by their limitations, but we don't need everyone to understand what backpropagation is, just as most of us use cars without knowing much about how an internal combustion engine works. And the issue you mention in the last paragraph is very relevant, since the scenario is plausible, so it is something we definitely should be discussing.
- Marazan 1y ago
  Because if you don't understand how a tool works you can't use the tool to it's full potential. Imagine if you were using single layer perceptrons without understanding seperability and going "just a few more tweaks and it will approximate XOR!"
  - kubb 1y ago
    You hit the nail on the head, in my opinion. There are things that you just can’t expect from current LLMs that people routinely expect from them. They start out projects with those expectations. And that’s fine. But they don’t always learn from the outcomes of those projects.
  - Al-Khwarizmi 1y ago
    I don't think that's a good analogy, becuase if you're trying to train a single layer perceptron to approximate XOR you're not the end user.
    - Marazan 1y ago
      The analogy is if you don't understand the limitations of the tool you may try and make it do something it is bad at and never understand why it will never do the thing you want despite looking like it potentially coild
    - vajrabum 1y ago
arisAlexis 1y ago
Obviously they are more focused on making something that works
- spwa4 1y ago
  Wow. Definitely NOT management material then.
CuriouslyC 1y ago
I think there are a lot of people who just don't care about stuff like nanochat because it's exclusively pedagogical, and a lot of people want to learn by building something cool, not taking a ride on a kiddie bike with training wheels.
- HarHarVeryFunny 1y ago
  That's fine as far as it goes, but there is a middle ground ... Feynman was right that "If you can't build it, you don't understand it", but of course not everyone needs or wants to fully understand how an LLM works. However, regarding an LLM as a magic black box seems a bit extreme if you are a technologist and hope to understand where the technology is heading. I guess we are in an era of vibe-coded disposable "fast tech" (cf fast fashion), so maybe it only matters what can it do today, if playing with or applying towards this end it is all you care about, but this seems a rather blinkered view.
  - vrighter 1y ago
    The problem is that not even the ones building them can understand it. Otherwise they wouldn't be breaking their "I promise AGI by the end of next year" promises for the Nth time. That or they are flat out lying. My money's on the latter.
android521 1y ago
Do you go deep into molecular biology to see how it works , it is much more interesting and important
danielbln 1y ago
I'm personally very interested in how LLMs work under the hood, but I don't think everyone who uses them as tools needs that. I don't know the wiring inside my drill, but I know how to put a hole in my wall and not my hand regardless.
- vrighter 1y ago
  The thing is that if you actually learn how they work, they lose all of their magic. This has happened to anyone I know who bothered studying them and is not selling them. So I'd rather people learned. Knowing how a drill works doesn't make you any less likely to use a drill.
amelius 1y ago
But the question is if you have a better understanding of LLMs from a user's perspective, or they.
tanepiper 1y ago
If everyone had to understand how carburettors, engines and break systems work; to be able to drive a car - rather than just learn to drive and get from A to B - I'm guessing there would be a lot less cars on the road. (Thinking about it, would that necessarily be a bad thing...)
- whizzter 1y ago
  The problem is that we have a huge swathe of "mechanics" that basically don't know much more than how to open a paintcan and paint a pig despite promising to deliver finely tuned supercars with their magic car making machine.
- deleted 1y ago
  [deleted]
miki123211 1y ago
Not everybody who drives a car (even as a professional driver) knows how to make one. If you live in a world of horse carriages, you can be thinking about what the world of cars is going to be like, even if you don't fully understand what fuel mix is the most efficient or what material one should use for a piston in a four-stroke.

by kubb 1y ago

teiferer 1y ago

Which is terrible. That's the root of all the BS around LLMs. People lacking understanding of what they are and ascribing capabilities which LLMs just don't have, by design. Even HN discussions are full of that. Even though this page literally has "hacker" in its name.

kubb 1y ago
I’m trying not to be disappointed by people, I’d rather understand what’s going on in their minds, and how to navigate that.
- deleted 1y ago
  [deleted]
tim333 1y ago
I see your point but on the other hand a lot of conversations go: A: what will we do when AI do all the jobs, B: that's silly LLMs can't do the jobs. The thing is A didn't say LLM, they said AI as in whatever that will be a short while into the future. Which is changing rapidly because thousands of bright people are being paid to change it.
- HarHarVeryFunny 1y ago
  The trouble is that "AI" is also very much a leaky abstraction, which makes it tempting to see all the "AI" advances of recent years, then correctly predict that these "AI" advances will continue, but then jump to all sorts of wrong conclusions about what those advances will be. For example, things like "AI" image and video generation are amazing, as are things like AlphaGo and AlphaFold, but none of these have anything to do with LLMs, and the only technology they share with LLMs is machine learning and neural nets. If you lump these together with LLMs, calling them all "AI", then you'll come to the wrong conclusion that all of these non-LLM advances indicate that "AI" is rapidly advancing and therefore LLMs (also being "AI") will do too ... Even if you leave aside things like AlphaGo, and just focus on LLMs, and other future technology that may take all our jobs, then using terms like "AI" and "AGI" are still confusing and misleading. It's easy to fall into the mindset that "AGI" is just better "AI", and that since LLMs are "AI", AGI is just better LLMs, and is around the corner because "AI" is advancing rapidly ... In reality LLMs are, like AlphaFold, something highly specific - they are auto-regressive next-word predictor language models (just as a statement of fact, and how they are trained, not a put-down), based on the Transformer architecture. The technology that could replace humans for most jobs in the future isn't going to be a better language model - a better auto-regressive next-word predictor - but will need to be something much more brain like. The architecture itself doesn't have to be brain-like, but in order to deliver brain-like functionality it will probably need to include another half-dozen "Transformer-level" architectural/algorithmic breakthroughs including things like continual learning, which will likely turn the whole current LLM training and deployment paradigm on it's head. Again, just focusing on LLMs, and LLM-based agents, regarding them as a black-box technology, it's easy to be misled into thinking that advances in capability are broadly advancing, and will rise all ships, when in reality progress is much more narrow. Headlines about LLMs achievement in math and competitive programming, touted as evidence of reasoning, do NOT imply that LLM reasoning is broadly advancing, but you need to get under the hood and understand RL training goals to realize why that is not necessarily the case. The correctness of most business and real-world reasoning is not as easy to check as is marking a math problem as correct or not, yet that capability is what RL training depends on. I could go on .. LLM-based agents are also blurring the lines of what "AI" can do, and again if treated as a black box will also misinform as to what is actually progressing and what is not. Thousands of bright people are indeed working on improving LLM-adjacent low-hanging fruit like this, but it'd be illogical to conclude that this is somehow helping to create next-generation brain-like architectures that will take away our jobs.
- teiferer 1y ago
  > a short while into the future And what gives you that confidence? A few AI nerds already claimed that in the 80s. We're currently exploring what LLMs can do. There is no indication that any further fundamental breakthrough is around the corner. Everybody is currently squeezing the same stone.

Al-Khwarizmi 1y ago

What's wrong or odd about that? You can like a technology as a user and not want to delve into how it works (sentence written by a human despite use of "delve"). Everyone should have some notions on what LLMs can or cannot do, in order to use them successfully and not be misguided by their limitations, but we don't need everyone to understand what backpropagation is, just as most of us use cars without knowing much about how an internal combustion engine works. And the issue you mention in the last paragraph is very relevant, since the scenario is plausible, so it is something we definitely should be discussing.

Marazan 1y ago
Because if you don't understand how a tool works you can't use the tool to it's full potential. Imagine if you were using single layer perceptrons without understanding seperability and going "just a few more tweaks and it will approximate XOR!"
- kubb 1y ago
  You hit the nail on the head, in my opinion. There are things that you just can’t expect from current LLMs that people routinely expect from them. They start out projects with those expectations. And that’s fine. But they don’t always learn from the outcomes of those projects.
- Al-Khwarizmi 1y ago
  I don't think that's a good analogy, becuase if you're trying to train a single layer perceptron to approximate XOR you're not the end user.
  - Marazan 1y ago
    The analogy is if you don't understand the limitations of the tool you may try and make it do something it is bad at and never understand why it will never do the thing you want despite looking like it potentially coild
  - vajrabum 1y ago

arisAlexis 1y ago

Obviously they are more focused on making something that works

spwa4 1y ago
Wow. Definitely NOT management material then.

CuriouslyC 1y ago

I think there are a lot of people who just don't care about stuff like nanochat because it's exclusively pedagogical, and a lot of people want to learn by building something cool, not taking a ride on a kiddie bike with training wheels.

HarHarVeryFunny 1y ago
That's fine as far as it goes, but there is a middle ground ... Feynman was right that "If you can't build it, you don't understand it", but of course not everyone needs or wants to fully understand how an LLM works. However, regarding an LLM as a magic black box seems a bit extreme if you are a technologist and hope to understand where the technology is heading. I guess we are in an era of vibe-coded disposable "fast tech" (cf fast fashion), so maybe it only matters what can it do today, if playing with or applying towards this end it is all you care about, but this seems a rather blinkered view.
- vrighter 1y ago
  The problem is that not even the ones building them can understand it. Otherwise they wouldn't be breaking their "I promise AGI by the end of next year" promises for the Nth time. That or they are flat out lying. My money's on the latter.

android521 1y ago

Do you go deep into molecular biology to see how it works , it is much more interesting and important

danielbln 1y ago

I'm personally very interested in how LLMs work under the hood, but I don't think everyone who uses them as tools needs that. I don't know the wiring inside my drill, but I know how to put a hole in my wall and not my hand regardless.

vrighter 1y ago
The thing is that if you actually learn how they work, they lose all of their magic. This has happened to anyone I know who bothered studying them and is not selling them. So I'd rather people learned. Knowing how a drill works doesn't make you any less likely to use a drill.

amelius 1y ago

But the question is if you have a better understanding of LLMs from a user's perspective, or they.

tanepiper 1y ago

If everyone had to understand how carburettors, engines and break systems work; to be able to drive a car - rather than just learn to drive and get from A to B - I'm guessing there would be a lot less cars on the road. (Thinking about it, would that necessarily be a bad thing...)

whizzter 1y ago
The problem is that we have a huge swathe of "mechanics" that basically don't know much more than how to open a paintcan and paint a pig despite promising to deliver finely tuned supercars with their magic car making machine.
deleted 1y ago
[deleted]

miki123211 1y ago

Not everybody who drives a car (even as a professional driver) knows how to make one. If you live in a world of horse carriages, you can be thinking about what the world of cars is going to be like, even if you don't fully understand what fuel mix is the most efficient or what material one should use for a piston in a four-stroke.