6 ms·

I've been forced to work with a team of very very smart Phds from ivy league universities to help put together an explanation deck for the c-suite. They've info

I've been forced to work with a team of very very smart Phds from ivy league universities to help put together an explanation deck for the c-suite. They've informally told me, the neural nets AI tech is beyond human understanding. Everybody can only explain very small pieces of it,and no one knows how the pieces work together.

justanotherjoe 3y ago
It is true and it's worth reminding. Science is progressed by interpretation of data. And we have easy access to a behemoth that needs interpreting. AI winter shouldn't come anytime soon.
- layer8 3y ago
  …assuming we make significant progress in explaining the data. Science means coming up with theories based on observations, and then testing the theory by verifying the further predictions made by those theories by experiment. It remains to be seen how much success there will be in that regard for LLMs.
layer8 3y ago
Yes. In a recent HN submission, ChatGPT was listed as a “scientific advance” of the past year. While it’s certainly some kind of advance, to me it seems to be more on the engineering side than on the scientific understanding side.
- Sharlin 3y ago
  Definitely engineering. It’s not entirely wrong to say that the two reasons it took us until 2022 to make a ChatGPT are 1) the computing power needed and 2) the size of the training corpus needed. The same goes for other generative AI – it took a corpus of a couple billion images to train a Stable Diffusion model.
  - amelius 3y ago
    You may argue that it took a leap of insight to get to transformer models, though.
    - charcircuit 3y ago
      That was not an innovation of ChatGPT.
      - mannykannot 3y ago
        It's pretty clear that ChatGPT is being used here as a synecdoche for recent LLMs, and transformer LLMs in particular.
        idontpost 3y ago
        [dead]
        charcircuit 3y ago
        "Transformer is All You Need" is from 6 years ago. It isn't an advancement that happened last year.
- mannykannot 3y ago
  If we discovered a species of parrot that could learn to use language in the manner of recent LLMs, that would count as a scientific advance (though not a breakthrough, at least until we achieved a good grasp on how it is possible.) Science advances firstly by finding something in want of an explanation, and then by coming up with one.
  - layer8 3y ago
    I don’t follow. IMO coming up with a working explanation is a necessary part of scientific advance. And that’s what we’re currently missing with LLMs, and with your parrot example.
    - mannykannot 3y ago
      Case in point: when Zwicky discovered that galaxies were rotating faster than could be explained in terms of what we know, that was an advance - an increase in our scientific knowledge. when we come up with a satisfactory explanation of why that is the case, then we will also have an advance - an increase in our scientific understanding. You can't get to the latter until you have advanced to the former, and we will probably need further advances of our scientific knowledge before we can understand the phenomenon Zwicky identified.
      - amelius 3y ago
        I get where you're coming from but we might also be like ants walking over an iPhone, wondering where the vibration is coming from. They might eventually figure it out, but if so it will be after an extremely long time, and they probably should better focus on other things at this very moment if they seek enlightenment.
        pixl97 3y ago
        Luckily there are lots of humans and we can focus on a lot of different things.
charcircuit 3y ago
It's not beyond human understanding. Unless you mean that one must know everything from every research paper released. At its core you are just finding a well performing model using gradient descent. Gradient descent is not beyond human understanding.
- famouswaffles 3y ago
  Gradient descent in isolation is obviously not what they are alluding to. What the models are doing inside the box and what any of those millions or billions of weights mean or do is beyond human understanding.
  - gen220 3y ago
    I don't think it is, as somebody who's spent maybe 100 combined hours reading AI papers mostly focused around NLP and image classification. You have a dataset, symbolically represented in 1s and 0s. You have an objective function (e.g. classify the object as belonging to one of N categories). The purpose of the collective neurons in the network is to "encode" the input space in a way that satisfies the objective function. In the same way that we "encode" higher-level concepts into shorthand representations. Gradient descent is the optimization function we use to develop this encoding. Beyond this, there are all kinds of tricks people have developed (interesting activation functions for neurons, grouping + segregating neurons, introducing a dimension of recurrence/time, dataset pre-processing, using bigger datasets, having another model generate data that's deliberately challenging for the first model) to try to converge to a more robust/accurate encoding, or to try to converge to a decent encoding at a faster rate. There is no magic here at the lowest level – you can interrogate the math at each step and it'll make sense. The "magic" is that we have zero epistemology to explain why tricks work, other than "look, ma test results". We know certain techniques work, and we have post-hoc intuitive explanations, but we're mostly fumbling our way "forwards" via trial and error. This is "science" in the 17th century definition of the term, where we're mixing chemicals together and seeing what happens. Maybe we'll have a good theoretical explanation for our experimental results 100 years from now, if we're still around.
    - 3y ago
joe_the_user 3y ago
To explain thing: neural networks are referred to as AI today (while for a long time they were just "machine learning") but there's a substantial consensus they won't be AGI (Artificial General Intelligence, "human like intelligence" etc) and a nearly universal belief they aren't AGI now. That scientists don't understand their internal processes doesn't change this and isn't necessarily related, directly, to humans not knowing how to create AGI.

justanotherjoe 3y ago

It is true and it's worth reminding. Science is progressed by interpretation of data. And we have easy access to a behemoth that needs interpreting. AI winter shouldn't come anytime soon.

layer8 3y ago
…assuming we make significant progress in explaining the data. Science means coming up with theories based on observations, and then testing the theory by verifying the further predictions made by those theories by experiment. It remains to be seen how much success there will be in that regard for LLMs.

layer8 3y ago

Yes. In a recent HN submission, ChatGPT was listed as a “scientific advance” of the past year. While it’s certainly some kind of advance, to me it seems to be more on the engineering side than on the scientific understanding side.

Sharlin 3y ago
Definitely engineering. It’s not entirely wrong to say that the two reasons it took us until 2022 to make a ChatGPT are 1) the computing power needed and 2) the size of the training corpus needed. The same goes for other generative AI – it took a corpus of a couple billion images to train a Stable Diffusion model.
- amelius 3y ago
  You may argue that it took a leap of insight to get to transformer models, though.
  - charcircuit 3y ago
    That was not an innovation of ChatGPT.
    - mannykannot 3y ago
      It's pretty clear that ChatGPT is being used here as a synecdoche for recent LLMs, and transformer LLMs in particular.
      - idontpost 3y ago
        [dead]
      - charcircuit 3y ago
        "Transformer is All You Need" is from 6 years ago. It isn't an advancement that happened last year.
mannykannot 3y ago
If we discovered a species of parrot that could learn to use language in the manner of recent LLMs, that would count as a scientific advance (though not a breakthrough, at least until we achieved a good grasp on how it is possible.) Science advances firstly by finding something in want of an explanation, and then by coming up with one.
- layer8 3y ago
  I don’t follow. IMO coming up with a working explanation is a necessary part of scientific advance. And that’s what we’re currently missing with LLMs, and with your parrot example.
  - mannykannot 3y ago
    Case in point: when Zwicky discovered that galaxies were rotating faster than could be explained in terms of what we know, that was an advance - an increase in our scientific knowledge. when we come up with a satisfactory explanation of why that is the case, then we will also have an advance - an increase in our scientific understanding. You can't get to the latter until you have advanced to the former, and we will probably need further advances of our scientific knowledge before we can understand the phenomenon Zwicky identified.
    - amelius 3y ago
      I get where you're coming from but we might also be like ants walking over an iPhone, wondering where the vibration is coming from. They might eventually figure it out, but if so it will be after an extremely long time, and they probably should better focus on other things at this very moment if they seek enlightenment.
      - pixl97 3y ago
        Luckily there are lots of humans and we can focus on a lot of different things.

charcircuit 3y ago

It's not beyond human understanding. Unless you mean that one must know everything from every research paper released. At its core you are just finding a well performing model using gradient descent. Gradient descent is not beyond human understanding.

famouswaffles 3y ago
Gradient descent in isolation is obviously not what they are alluding to. What the models are doing inside the box and what any of those millions or billions of weights mean or do is beyond human understanding.
- gen220 3y ago
  I don't think it is, as somebody who's spent maybe 100 combined hours reading AI papers mostly focused around NLP and image classification. You have a dataset, symbolically represented in 1s and 0s. You have an objective function (e.g. classify the object as belonging to one of N categories). The purpose of the collective neurons in the network is to "encode" the input space in a way that satisfies the objective function. In the same way that we "encode" higher-level concepts into shorthand representations. Gradient descent is the optimization function we use to develop this encoding. Beyond this, there are all kinds of tricks people have developed (interesting activation functions for neurons, grouping + segregating neurons, introducing a dimension of recurrence/time, dataset pre-processing, using bigger datasets, having another model generate data that's deliberately challenging for the first model) to try to converge to a more robust/accurate encoding, or to try to converge to a decent encoding at a faster rate. There is no magic here at the lowest level – you can interrogate the math at each step and it'll make sense. The "magic" is that we have zero epistemology to explain why tricks work, other than "look, ma test results". We know certain techniques work, and we have post-hoc intuitive explanations, but we're mostly fumbling our way "forwards" via trial and error. This is "science" in the 17th century definition of the term, where we're mixing chemicals together and seeing what happens. Maybe we'll have a good theoretical explanation for our experimental results 100 years from now, if we're still around.
  - 3y ago

joe_the_user 3y ago

To explain thing: neural networks are referred to as AI today (while for a long time they were just "machine learning") but there's a substantial consensus they won't be AGI (Artificial General Intelligence, "human like intelligence" etc) and a nearly universal belief they aren't AGI now. That scientists don't understand their internal processes doesn't change this and isn't necessarily related, directly, to humans not knowing how to create AGI.