4 ms·

Seems coincide with this that was on HN: http://www.incompleteideas.net/IncIdeas/BitterLesson.html http://www.incompleteideas.net/IncIdeas/BitterLesson.html

galkk 3y ago
I love story in spite of the article above. Speech synthesis also was attempted as modeling of human biology: computer modeling of throats, vocal cords, how the air is going through mouth. In the end computational power also won. No need of all of that.
- nomel 3y ago
  And then there's the Voder (1939): https://www.youtube.com/watch?v=TsdOej_nC1M https://www.youtube.com/watch?v=TsdOej_nC1M
- acer4666 3y ago
  The article is talking about deep learning winning, ie neural networks. Surely modelling of human biology is part of that?
  - nomel 3y ago
    Maybe emergent, to some extent, but not explicit.
  - bigyikes 3y ago
    Neural nets do not model human biology, they are models which are inspired by human biology.
- totetsu 3y ago
  Maybe we didn’t need to, but I’m glad someone did https://m.youtube.com/watch?v=qobhDJ_vEOc https://m.youtube.com/watch?v=qobhDJ_vEOc
bbor 3y ago
> The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin. It’s nice when an author includes a sentence up top that betrays their standpoint so that I can stop reading. I’m sure this person is very nice and has lots of stuff to say, but this is the same old Scruffy v. Neat fight, except now the former side thinks that they’re empirically completely right. Which doesn’t even make sense, they’re not mutually exclusive claims, and to say that the result of 70 years of expert systems is any kind of failure is just revisionist. For the same reason, I don’t read many papers about Realism vs Idealism, Nature vs Nurture, etc
- SonOfLilit 3y ago
  I recommend that you continue. It's a very short and highly influential piece by a guy at the top of his field, you'll take a thing or two from the one-minute read even if you agree with nothing he says.
  - bbor 3y ago
    Why thanks for the suggestion, that’s very kind. I just read it - in fact I think I read this early on in my research on scruffies v neats. My takeaway now is the same, sadly. It’s not so much that I disagree with his premises that I find his whole attitude and conclusion to be a preposterous artifact of ego inflation after helping found a line of research that was much more productive than people thought it would be. I get it, that’s very exciting, but I need way more evidence than that to completely abandon self-conscious structured reasoning in my conception of a good AGI, much less the human mind. Like this: This is a big lesson. As a field, we still have not thoroughly learned it, as we are continuing to make the same kind of mistakes. This is just arrogance. You don’t see this among philosophers or social scientists, who recognize that rhetoric is more than the cherry on top of science, and that this sort of confidence is dangerous. To see “designing things by hand” as a categorical “mistake” is just… that’s a hot take. But either way we’re all on the same side. Despite my harsh words I’m glad he helped pave the way LLMs, which are the biggest unexpected breakthrough in our lifetimes IMO. Which understandably makes people confident

galkk 3y ago

I love story in spite of the article above. Speech synthesis also was attempted as modeling of human biology: computer modeling of throats, vocal cords, how the air is going through mouth. In the end computational power also won. No need of all of that.

nomel 3y ago
And then there's the Voder (1939): https://www.youtube.com/watch?v=TsdOej_nC1M https://www.youtube.com/watch?v=TsdOej_nC1M
acer4666 3y ago
The article is talking about deep learning winning, ie neural networks. Surely modelling of human biology is part of that?
- nomel 3y ago
  Maybe emergent, to some extent, but not explicit.
- bigyikes 3y ago
  Neural nets do not model human biology, they are models which are inspired by human biology.
totetsu 3y ago
Maybe we didn’t need to, but I’m glad someone did https://m.youtube.com/watch?v=qobhDJ_vEOc https://m.youtube.com/watch?v=qobhDJ_vEOc

bbor 3y ago

> The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin. It’s nice when an author includes a sentence up top that betrays their standpoint so that I can stop reading. I’m sure this person is very nice and has lots of stuff to say, but this is the same old Scruffy v. Neat fight, except now the former side thinks that they’re empirically completely right. Which doesn’t even make sense, they’re not mutually exclusive claims, and to say that the result of 70 years of expert systems is any kind of failure is just revisionist. For the same reason, I don’t read many papers about Realism vs Idealism, Nature vs Nurture, etc

SonOfLilit 3y ago
I recommend that you continue. It's a very short and highly influential piece by a guy at the top of his field, you'll take a thing or two from the one-minute read even if you agree with nothing he says.
- bbor 3y ago
  Why thanks for the suggestion, that’s very kind. I just read it - in fact I think I read this early on in my research on scruffies v neats. My takeaway now is the same, sadly. It’s not so much that I disagree with his premises that I find his whole attitude and conclusion to be a preposterous artifact of ego inflation after helping found a line of research that was much more productive than people thought it would be. I get it, that’s very exciting, but I need way more evidence than that to completely abandon self-conscious structured reasoning in my conception of a good AGI, much less the human mind. Like this: This is a big lesson. As a field, we still have not thoroughly learned it, as we are continuing to make the same kind of mistakes. This is just arrogance. You don’t see this among philosophers or social scientists, who recognize that rhetoric is more than the cherry on top of science, and that this sort of confidence is dangerous. To see “designing things by hand” as a categorical “mistake” is just… that’s a hot take. But either way we’re all on the same side. Despite my harsh words I’m glad he helped pave the way LLMs, which are the biggest unexpected breakthrough in our lifetimes IMO. Which understandably makes people confident