13 ms·

Andrej Karpathy, the director of AI at tesla, published an AMAZING video about how the current models are built which I think everyone in the tech space should

Andrej Karpathy, the director of AI at tesla, published an AMAZING video about how the current models are built which I think everyone in the tech space should watch. It is intuitive, easy to follow and quite frankly the best video I have seen on the topic. Here is the link: https://www.youtube.com/watch?v=kCc8FmEb1nY&list=FL2tbfd7UpJdKc76jlXrKLEg&index=1&t=2s https://www.youtube.com/watch?v=kCc8FmEb1nY&list=FL2tbfd7UpJ... If you conclude, after watching this, that this is all intelligence is - namely cascading optimization and data structure problems chained in a row, then you and a fair number of people here wont ever find common ground. That is not to say that you're wrong, it just seems as if there is a missing component here that we haven't yet discovered.

spuz 3y ago
> If you conclude, after watching this, that this is all intelligence is - namely cascading optimization and data structure problems chained in a row, then you and a fair number of people here wont ever find common ground. They won't find common ground with who?
Lerc 3y ago
I'm ok with believing that all it takes to make intelligence (probably not the only way) is a sufficiently large neural net with the right architecture and weights. I think it is easy to distracted by the specific mechanisms by which these models work, but most of the technological detail is because we want something to happen on systems at the scale of what we can actually build. We simply can't build a human brain scale neural net yet. We build what we can and besides maybe with all this research we will figure something significant about what intelligence actually is. The notion "This can't be all thought is" is as old as the idea of AI. I think it informed Turing when he proposed the Imitation Game. The insight is that people would be resistant to the idea of a bunch of simple things stuck together becoming thinking until they were faced with something that behaves sufficiently indistinguishable from themselves that to doubt that they were thinking would be akin to doubting everyone you meet. In the end some people won't even accept an AI that does everything a human does as actually thinking, but then again some people are actually solipsistic.
- bondarchuk 3y ago
  >The notion "This can't be all thought is" is as old as the idea of AI. Older still: >It must be confessed, moreover, that perception, and that which depends on it, are inexplicable by mechanical causes, that is, by figures and motions, And, supposing that there were a mechanism so constructed as to think, feel and have perception, we might enter it as into a mill. And this granted, we should only find on visiting it, pieces which push one against another, but never anything by which to explain a perception. This must be sought, therefore, in the simple substance, and not in the composite or in the machine. - Leibniz, 1714
  - Lerc 3y ago
    I would argue that that quote contains the idea of AI
    - bondarchuk 3y ago
      Fair enough.
calf 3y ago
I watched his first video in the series, but immediately I don't know what a loss function is. Can you recommend a book/free course that goes over the more basic concepts?

spuz 3y ago

> If you conclude, after watching this, that this is all intelligence is - namely cascading optimization and data structure problems chained in a row, then you and a fair number of people here wont ever find common ground. They won't find common ground with who?

Lerc 3y ago

I'm ok with believing that all it takes to make intelligence (probably not the only way) is a sufficiently large neural net with the right architecture and weights. I think it is easy to distracted by the specific mechanisms by which these models work, but most of the technological detail is because we want something to happen on systems at the scale of what we can actually build. We simply can't build a human brain scale neural net yet. We build what we can and besides maybe with all this research we will figure something significant about what intelligence actually is. The notion "This can't be all thought is" is as old as the idea of AI. I think it informed Turing when he proposed the Imitation Game. The insight is that people would be resistant to the idea of a bunch of simple things stuck together becoming thinking until they were faced with something that behaves sufficiently indistinguishable from themselves that to doubt that they were thinking would be akin to doubting everyone you meet. In the end some people won't even accept an AI that does everything a human does as actually thinking, but then again some people are actually solipsistic.

bondarchuk 3y ago
>The notion "This can't be all thought is" is as old as the idea of AI. Older still: >It must be confessed, moreover, that perception, and that which depends on it, are inexplicable by mechanical causes, that is, by figures and motions, And, supposing that there were a mechanism so constructed as to think, feel and have perception, we might enter it as into a mill. And this granted, we should only find on visiting it, pieces which push one against another, but never anything by which to explain a perception. This must be sought, therefore, in the simple substance, and not in the composite or in the machine. - Leibniz, 1714
- Lerc 3y ago
  I would argue that that quote contains the idea of AI
  - bondarchuk 3y ago
    Fair enough.

calf 3y ago

I watched his first video in the series, but immediately I don't know what a loss function is. Can you recommend a book/free course that goes over the more basic concepts?