3 ms·

"The result is an artificial idiot savant that can excel at well-bounded tasks, but can get things very wrong if faced with unexpected input." I think this get

"The result is an artificial idiot savant that can excel at well-bounded tasks, but can get things very wrong if faced with unexpected input." I think this gets to the core of what is still a limitation of current technologies. Venturing into the unknown is still a deeply relevant task that seems unlikely to be replaced by computers anytime soon.

zby 6y ago
This is the same thing people have been writing about computers since 1950s. At that level of abstraction the progress is invisible - and yet: https://slatestarcodex.com/2020/06/10/the-obligatory-gpt-3-post/ https://slatestarcodex.com/2020/06/10/the-obligatory-gpt-3-p... https://slatestarcodex.com/2020/01/06/a-very-unlikely-chess-game/ https://slatestarcodex.com/2020/01/06/a-very-unlikely-chess-...
londons_explore 6y ago
> can get things very wrong if faced with unexpected input Alice: Bob, Can you translate "Eat my shorts" into latin for me? Bob: No. I don't speak latin. Alice: Go on - try anyway. Bob: "Eatus mine shortus" Alice: Wrong! The answer is "Vescere bracis meis". You're totally wrong Bob! I was expecting more of you!
- Radim 6y ago
  You missed the preface: Bob's manager: Here's Bob. You paid a gazillion dollars for him, and he can translate whatever you want. Any language, whenever – the true revolution everyone's been talking about. Your life and our economy will never be the same. Alice: Bob, Can you translate "Eat my shorts" into latin for me? …
- Olreich 6y ago
  If models actually were able to tell you what they know and don’t know, then sure. But instead they just give you an output for any input you give them, whether they have a clue or not.
  - 0xBABAD00C 6y ago
    This is such a simplistic view of a vast and evolving field of scientific research, it's too cartoonish of an argument to even warrant a real response. Which is why the conversation around ML gets negatively selected against actual researchers and practitioners, who get headaches from opinions like the one above.
    - memexy 6y ago
      What's wrong with their view. They're right that such models will always give an answer whether it makes sense or not. Has anyone tried asking Bob to solve sudoku? What happens? Does Bob fill in the blank spots according to sudoku rules and is the final solution an actual solution for the given board or does it end up being a random list of numbers?
    - imtringued 6y ago
      Honestly last time I tried to get into machine learning the huge limitations made me lose interest. The idea that we can just pretend that these limitations don't exist is absurd. ML lets us do a lot of very interesting things but it's just a stepping stone. It's something we are stuck with rather than something that we want to keep.
- amayne 6y ago
  For fun I put this into OpenAI's new API: Alice: Bob, Can you translate "Eat my shorts" into latin for me? This is what it replied without any training: Bob: As it happens, yes, I can. Alice: Great. Bob: Ego sum vestrum braccas comedisse shortus. Alice: Exactly what I was looking for. Google translate says "Ego sum vestrum braccas comedisse shortus" means " I have eaten your pants shorts."
SomewhatLikely 6y ago
This is changing. In natural language processing just in the past couple weeks OpenAI wrote about their GPT3 model which can learn some tasks remarkably quickly, after only 1 or 2 examples. That model has extreme compute requirements, but it shows strong progress on performing never before seen tasks. There is still some steam left in the current deep learning boom. https://arxiv.org/abs/2005.14165 https://arxiv.org/abs/2005.14165
- ausbah 6y ago
  I heard someone theorize that GPT3 can perform so well simply because it can overfit to every domain

zby 6y ago

This is the same thing people have been writing about computers since 1950s. At that level of abstraction the progress is invisible - and yet: https://slatestarcodex.com/2020/06/10/the-obligatory-gpt-3-post/ https://slatestarcodex.com/2020/06/10/the-obligatory-gpt-3-p... https://slatestarcodex.com/2020/01/06/a-very-unlikely-chess-game/ https://slatestarcodex.com/2020/01/06/a-very-unlikely-chess-...

londons_explore 6y ago

> can get things very wrong if faced with unexpected input Alice: Bob, Can you translate "Eat my shorts" into latin for me? Bob: No. I don't speak latin. Alice: Go on - try anyway. Bob: "Eatus mine shortus" Alice: Wrong! The answer is "Vescere bracis meis". You're totally wrong Bob! I was expecting more of you!

Radim 6y ago
You missed the preface: Bob's manager: Here's Bob. You paid a gazillion dollars for him, and he can translate whatever you want. Any language, whenever – the true revolution everyone's been talking about. Your life and our economy will never be the same. Alice: Bob, Can you translate "Eat my shorts" into latin for me? …
Olreich 6y ago
If models actually were able to tell you what they know and don’t know, then sure. But instead they just give you an output for any input you give them, whether they have a clue or not.
- 0xBABAD00C 6y ago
  This is such a simplistic view of a vast and evolving field of scientific research, it's too cartoonish of an argument to even warrant a real response. Which is why the conversation around ML gets negatively selected against actual researchers and practitioners, who get headaches from opinions like the one above.
  - memexy 6y ago
    What's wrong with their view. They're right that such models will always give an answer whether it makes sense or not. Has anyone tried asking Bob to solve sudoku? What happens? Does Bob fill in the blank spots according to sudoku rules and is the final solution an actual solution for the given board or does it end up being a random list of numbers?
  - imtringued 6y ago
    Honestly last time I tried to get into machine learning the huge limitations made me lose interest. The idea that we can just pretend that these limitations don't exist is absurd. ML lets us do a lot of very interesting things but it's just a stepping stone. It's something we are stuck with rather than something that we want to keep.
amayne 6y ago
For fun I put this into OpenAI's new API: Alice: Bob, Can you translate "Eat my shorts" into latin for me? This is what it replied without any training: Bob: As it happens, yes, I can. Alice: Great. Bob: Ego sum vestrum braccas comedisse shortus. Alice: Exactly what I was looking for. Google translate says "Ego sum vestrum braccas comedisse shortus" means " I have eaten your pants shorts."

SomewhatLikely 6y ago

This is changing. In natural language processing just in the past couple weeks OpenAI wrote about their GPT3 model which can learn some tasks remarkably quickly, after only 1 or 2 examples. That model has extreme compute requirements, but it shows strong progress on performing never before seen tasks. There is still some steam left in the current deep learning boom. https://arxiv.org/abs/2005.14165 https://arxiv.org/abs/2005.14165

ausbah 6y ago
I heard someone theorize that GPT3 can perform so well simply because it can overfit to every domain