4 ms·

There’s an exercise that some people do when learning programming, which is to write down the steps to make a sandwich. Then the teacher follows the exact instr

There’s an exercise that some people do when learning programming, which is to write down the steps to make a sandwich. Then the teacher follows the exact instructions to make a sandwich and most people don’t put enough detail for a computer to follow (I.e. open the fridge etc) and the teacher will run around bumping into things. That used to be a teaching exercise to show people the amount of precision required when telling a machine what to do. Now with LLMs, I think one of the great leaps is the idea that it’s no longer necessary to be “pedantic” when giving computers instructions because LLMs have somehow learned to fill in the blanks with a similar shared “understanding” of the world that we have (I.e. cheese is stored in the fridge so you have to go open the fridge to fetch the cheese for the sandwich).

hattmall 3y ago
I don't get the "magic" people are seeing. It makes sense. >LLMs have somehow learned to fill in the blanks It's not somehow, it's because they have read a ton of books, documents, etc and can make enough links between cheese and refrigerator and follow that back to know that a refrigerator needs to be opened. I have seen a lot of very clever AI examples using the latest tools, but I haven't seen anything that seems difficult to deconstruct.
- lee101 3y ago
  [dead]
deleted 3y ago
[deleted]
prox 3y ago
Don’t forget they hold vector spaces. So fridge and cheese score high together for cohesiveness, but fridge and Antarctica less so, but both have something to do with cold. Together with all the training on texts creates a good ability to make inferences and “conclusions”. It has a net of lines of meanings of all concepts we fed it that give it the ability that it has, without actually understanding.
- TeMPOraL 3y ago
  > It has a net of lines of meanings of all concepts we fed it that give it the ability that it has, without actually understanding. I'm increasingly convinced this is what understanding fundamentally is.
  - prox 3y ago
    Yeah, I guess you could see it that way. Object <> Symbolism (aka words, thoughts, concepts, art) <> Meaning. Meaning is knowing how an object relates to others. Language is a kind of information web already, where each word is a hyperlink into meaning.
    - TeMPOraL 3y ago
      This is a new idea that I had (or at least consciously noticed) for the first time a few days ago, but - I really don't think the meaning is in words. The words/terms themselves are more like information-free[0] points. The meaning is entirely determined by links. This works, because the links eventually lead you to ground truth - sensory inputs.
  - eternalban 3y ago
    That conflates perception with perceiver. LLMs have only internalized [/encoded] our perceptions and expressions. From a model of the mind pov, the 'self' that we sense has an internal LLM-like tool. And it is that self that understands and not the tool.
    - TeMPOraL 3y ago
      > That conflates perception with perceiver. I'm not sure I understand. Can you elaborate? > From a model of the mind pov, the 'self' that we sense has an internal LLM-like tool. And it is that self that understands and not the tool. I'm starting to think it's the other way around. I think it's somewhat widely accepted that our brains do most of the "thinking" and "understanding" unconsciously - our conscious self is more of an observer / moderator, occasionally hand-holding the thought process when the topic of interest is hard, and one isn't yet proficient[0] in it. Keeping that in mind, if you - like me - feel that LLMs are best compared to our "inner voice", i.e. the bit on the boundary between conscious and unconscious that uses language as an interface to the former, then it's not unreasonable to expect that LLMs may, in fact, understand things. Not emulate, but actually understand. The whole deal with a hundred thousand dimensional latent space? I have a growing suspicion that this is exactly the fundamental principle behind how understanding, thinking in concepts, and thinking in general works for humans too. Sure, we have multiple senses feeding into our "thinking" bit, but that doesn't change much. At a conceptual, handwavy level (I don't know the actual architecture and math details well enough to offer more concrete explanations/stories), I feel there are too many coincidences to ignore. Is this coincidence that someone trained an LLM and an image network, and found their independently learned latent spaces map to each other with a simple transforms? Maybe[1], but this also makes sense - both network segmented data about the same view of reality humans have. There is no reason for LLMs to have an entirely different way of representing "understanding" than img2txt or txt2img networks. Assuming the above is true, is this coincidence that it offers a decent explanation for how humans developed language? You start with a image/sound/touch/other senses acquisition and association system forming a basic brain. Predicting next sensations, driving actions. As it evolves in size and complexity, dimensionality of its representation space grows, and at some point, the associations cluster in something of a world model. Let evolution iterate some (couple hundred thousand years) more, and you end up with brains that can build more complex world model, working with more complex associations (e.g. vibration -> sound -> tone -> grunt -> phrase/song). At this level, language seems like an obvious thing - it's taking complex associations of basic sensory input, and associating them wholesale with different areas of the latent space, so that e.g. a specific grunt now associates with danger, a different one with safety, etc. and once you have brains being able to do that naturally, it's pretty much straight line to a proper language. Yes, this probably comes as a lot of hand-waving; I don't have the underlying insights properly sorted yet. But a core observation I want to communicate, and recommend people to ponder on, is continuity. This process gains capabilities in a continuous fashion, as it scales - which is exactly a kind of system you'd expect evolution to lock on to. -- [0] - What is "proficiency" anyway? To me, being proficient in a field of interest is mostly about... shifting understanding of that field to unconscious level as much as possible. [1] - This was one paper I am aware of; they probably didn't do good enough control, so it might turn out to be happenstance.
crooked-v 3y ago
Even then, you can see some of the "pedantic" cases when it comes to actually understanding the nature of the the connections between those concepts. For example, it's very easy to get it to reverse shorter/taller or younger/older of clearly defined relationships.
thomastjeffery 3y ago
> Now with LLMs, I think one of the great leaps is the idea that it’s no longer necessary to be “pedantic” when giving computers instructions Yes, but they also can't. They can't be pedantic or follow explicit instructions. That's the other side of the coin that isn't being presented. They can present the right elements of the story in the the right places, but they can't perform it.
satvikpendem 3y ago
It depends on the task. For certain programs you absolutely need to be pedantic in describing what needs to happen. There is a reason we don't program in natural language and that won't change with LLMs.

hattmall 3y ago

I don't get the "magic" people are seeing. It makes sense. >LLMs have somehow learned to fill in the blanks It's not somehow, it's because they have read a ton of books, documents, etc and can make enough links between cheese and refrigerator and follow that back to know that a refrigerator needs to be opened. I have seen a lot of very clever AI examples using the latest tools, but I haven't seen anything that seems difficult to deconstruct.

lee101 3y ago
[dead]

deleted 3y ago

[deleted]

prox 3y ago

Don’t forget they hold vector spaces. So fridge and cheese score high together for cohesiveness, but fridge and Antarctica less so, but both have something to do with cold. Together with all the training on texts creates a good ability to make inferences and “conclusions”. It has a net of lines of meanings of all concepts we fed it that give it the ability that it has, without actually understanding.

TeMPOraL 3y ago
> It has a net of lines of meanings of all concepts we fed it that give it the ability that it has, without actually understanding. I'm increasingly convinced this is what understanding fundamentally is.
- prox 3y ago
  Yeah, I guess you could see it that way. Object <> Symbolism (aka words, thoughts, concepts, art) <> Meaning. Meaning is knowing how an object relates to others. Language is a kind of information web already, where each word is a hyperlink into meaning.
  - TeMPOraL 3y ago
    This is a new idea that I had (or at least consciously noticed) for the first time a few days ago, but - I really don't think the meaning is in words. The words/terms themselves are more like information-free[0] points. The meaning is entirely determined by links. This works, because the links eventually lead you to ground truth - sensory inputs.
- eternalban 3y ago
  That conflates perception with perceiver. LLMs have only internalized [/encoded] our perceptions and expressions. From a model of the mind pov, the 'self' that we sense has an internal LLM-like tool. And it is that self that understands and not the tool.
  - TeMPOraL 3y ago
    > That conflates perception with perceiver. I'm not sure I understand. Can you elaborate? > From a model of the mind pov, the 'self' that we sense has an internal LLM-like tool. And it is that self that understands and not the tool. I'm starting to think it's the other way around. I think it's somewhat widely accepted that our brains do most of the "thinking" and "understanding" unconsciously - our conscious self is more of an observer / moderator, occasionally hand-holding the thought process when the topic of interest is hard, and one isn't yet proficient[0] in it. Keeping that in mind, if you - like me - feel that LLMs are best compared to our "inner voice", i.e. the bit on the boundary between conscious and unconscious that uses language as an interface to the former, then it's not unreasonable to expect that LLMs may, in fact, understand things. Not emulate, but actually understand. The whole deal with a hundred thousand dimensional latent space? I have a growing suspicion that this is exactly the fundamental principle behind how understanding, thinking in concepts, and thinking in general works for humans too. Sure, we have multiple senses feeding into our "thinking" bit, but that doesn't change much. At a conceptual, handwavy level (I don't know the actual architecture and math details well enough to offer more concrete explanations/stories), I feel there are too many coincidences to ignore. Is this coincidence that someone trained an LLM and an image network, and found their independently learned latent spaces map to each other with a simple transforms? Maybe[1], but this also makes sense - both network segmented data about the same view of reality humans have. There is no reason for LLMs to have an entirely different way of representing "understanding" than img2txt or txt2img networks. Assuming the above is true, is this coincidence that it offers a decent explanation for how humans developed language? You start with a image/sound/touch/other senses acquisition and association system forming a basic brain. Predicting next sensations, driving actions. As it evolves in size and complexity, dimensionality of its representation space grows, and at some point, the associations cluster in something of a world model. Let evolution iterate some (couple hundred thousand years) more, and you end up with brains that can build more complex world model, working with more complex associations (e.g. vibration -> sound -> tone -> grunt -> phrase/song). At this level, language seems like an obvious thing - it's taking complex associations of basic sensory input, and associating them wholesale with different areas of the latent space, so that e.g. a specific grunt now associates with danger, a different one with safety, etc. and once you have brains being able to do that naturally, it's pretty much straight line to a proper language. Yes, this probably comes as a lot of hand-waving; I don't have the underlying insights properly sorted yet. But a core observation I want to communicate, and recommend people to ponder on, is continuity. This process gains capabilities in a continuous fashion, as it scales - which is exactly a kind of system you'd expect evolution to lock on to. -- [0] - What is "proficiency" anyway? To me, being proficient in a field of interest is mostly about... shifting understanding of that field to unconscious level as much as possible. [1] - This was one paper I am aware of; they probably didn't do good enough control, so it might turn out to be happenstance.

crooked-v 3y ago

Even then, you can see some of the "pedantic" cases when it comes to actually understanding the nature of the the connections between those concepts. For example, it's very easy to get it to reverse shorter/taller or younger/older of clearly defined relationships.

thomastjeffery 3y ago

> Now with LLMs, I think one of the great leaps is the idea that it’s no longer necessary to be “pedantic” when giving computers instructions Yes, but they also can't. They can't be pedantic or follow explicit instructions. That's the other side of the coin that isn't being presented. They can present the right elements of the story in the the right places, but they can't perform it.

satvikpendem 3y ago

It depends on the task. For certain programs you absolutely need to be pedantic in describing what needs to happen. There is a reason we don't program in natural language and that won't change with LLMs.