4 ms·

"So I asked Ilya Sutskever, OpenAI’s chief scientist, for a reading list. He gave me a list of like 40 research papers and said, ‘If you really learn all of the

by jasmer 4y ago

"So I asked Ilya Sutskever, OpenAI’s chief scientist, for a reading list. He gave me a list of like 40 research papers and said, ‘If you really learn all of these, you’ll know 90% of what matters today.’ And I did. I plowed through all those things and it all started sorting out in my head." Wow, that's going to be one of the more glib things I've read in a while. This is a bit of a Tom Cruise moment. I mean, I get it on some level but I suggest it's going to take a bit for someone to 'catch up' to cutting edge AI. Like more than a 'week of reading papers he doesn't understand'. Defeating the Rust borrow checker takes longer than that!

Yoric 4y ago
> Defeating the Rust borrow checker takes longer than that! Fighting the Rust borrow checker to hate leads and hate to the Dark Side leads. Harmony with the Rust borrow checker is what you must achieve, padawan.
- int_19h 4y ago
  Or, come to the Dark Side. We have comptime!
  - Yoric 4y ago
    Tempting, the Dark Side (Zide?) is.
  - ngrilly 4y ago
    The Dark Zige.
Jensson 4y ago
The models aren't particularly hard to implement, so a person who has spent his life and is one of the best game engine programmers in the world I don't think it would take that long to implement the cutting edge of todays machine learning models. Compared to game graphics shaders and the complex interactions you have to work with to get those performant and work correctly and display all the gameplay that happens the technical parts of machine learning are very simple. I've worked building low level machine learning stuff at Google, it isn't that hard to do at all. The hard part is improving these models, not building them when you already know what to build.
- gary_0 4y ago
  > The hard part is ... not building them when you already know what to build. Indeed. A somewhat sharp schoolchild could build a light bulb or an electric motor/generator, and understand the basic underlying principles, in a pretty short time. But how many decades did it take the first researchers and inventors of those things to get to that same point? Heck, there was a high school kid who built a primitive semiconductor fab in their garage.[0] But for novel advancements, even getting to the point where you have an idea of what isn't impossible is half the battle. [0] https://news.ycombinator.com/item?id=28176255 https://news.ycombinator.com/item?id=28176255
spaceman_2020 4y ago
It would have been a Tom Cruise moment if he’d said, “I learned CSS over the weekend and got it right the first time”!
nwoli 4y ago
He got those papers 4 years ago and have implemented them in the meantime. That doesn’t sound infeasible at all
lelanthran 4y ago
> I mean, I get it on some level but I suggest it's going to take a bit for someone to 'catch up' to cutting edge AI. Agreed, in general, but in the specifics we are talking about someone who has spent decades solving really difficult math problems in a creative and novel manner. Who's to say he won't find some novel edge of the AI discipline to which he can apply a creative and never-seen-before solution? I mean, we're not talking about a general "somebody" here, he's got a record of accomplishing things that other people never managed to accomplish.
qualudeheart 4y ago
Does Tom Cruise know how to code?
jstx1 4y ago
Carmack talked about implementing most of these things in C by himself, being used to working 10-12 hours a day for 6 days a week, and writing video codecs in Rust for him was on the level "a little bit beyond hello world". I don't know how much of that is really true but given his background in creating games, I tend to believe him, and his work ethic seems really impressive.
bitshiftfaced 4y ago
You say "glib", but I hear someone who sounds inspired. You combine Carmack plus "inspired", throw $20 million dollars at it and stick it in a room for 10 years, yeah I'd say he characterized it accurately by giving himself a "non-zero chance."
modeless 4y ago
Have you read these papers? It's absolutely feasible to get the gist of modern AI in a week of 16 hour days, as Carmack likes to do, for someone who is already world class at CS and has a mastery of college level math. Modern AI is very simple at its core! As Carmack mentions in the article, cutting edge models are implemented in a couple thousand lines of code, and the core algorithms are even less. Rust's borrow checker is more complex.
- Tepix 4y ago
  I'm pretty sure it took at least a couple of months to fundamentally understand 40 papers.

by jasmer 4y ago

Yoric 4y ago

> Defeating the Rust borrow checker takes longer than that! Fighting the Rust borrow checker to hate leads and hate to the Dark Side leads. Harmony with the Rust borrow checker is what you must achieve, padawan.

int_19h 4y ago
Or, come to the Dark Side. We have comptime!
- Yoric 4y ago
  Tempting, the Dark Side (Zide?) is.
- ngrilly 4y ago
  The Dark Zige.

Jensson 4y ago

The models aren't particularly hard to implement, so a person who has spent his life and is one of the best game engine programmers in the world I don't think it would take that long to implement the cutting edge of todays machine learning models. Compared to game graphics shaders and the complex interactions you have to work with to get those performant and work correctly and display all the gameplay that happens the technical parts of machine learning are very simple. I've worked building low level machine learning stuff at Google, it isn't that hard to do at all. The hard part is improving these models, not building them when you already know what to build.

gary_0 4y ago
> The hard part is ... not building them when you already know what to build. Indeed. A somewhat sharp schoolchild could build a light bulb or an electric motor/generator, and understand the basic underlying principles, in a pretty short time. But how many decades did it take the first researchers and inventors of those things to get to that same point? Heck, there was a high school kid who built a primitive semiconductor fab in their garage.[0] But for novel advancements, even getting to the point where you have an idea of what isn't impossible is half the battle. [0] https://news.ycombinator.com/item?id=28176255 https://news.ycombinator.com/item?id=28176255

spaceman_2020 4y ago

It would have been a Tom Cruise moment if he’d said, “I learned CSS over the weekend and got it right the first time”!

nwoli 4y ago

He got those papers 4 years ago and have implemented them in the meantime. That doesn’t sound infeasible at all

lelanthran 4y ago

> I mean, I get it on some level but I suggest it's going to take a bit for someone to 'catch up' to cutting edge AI. Agreed, in general, but in the specifics we are talking about someone who has spent decades solving really difficult math problems in a creative and novel manner. Who's to say he won't find some novel edge of the AI discipline to which he can apply a creative and never-seen-before solution? I mean, we're not talking about a general "somebody" here, he's got a record of accomplishing things that other people never managed to accomplish.

qualudeheart 4y ago

Does Tom Cruise know how to code?

jstx1 4y ago

Carmack talked about implementing most of these things in C by himself, being used to working 10-12 hours a day for 6 days a week, and writing video codecs in Rust for him was on the level "a little bit beyond hello world". I don't know how much of that is really true but given his background in creating games, I tend to believe him, and his work ethic seems really impressive.

bitshiftfaced 4y ago

You say "glib", but I hear someone who sounds inspired. You combine Carmack plus "inspired", throw $20 million dollars at it and stick it in a room for 10 years, yeah I'd say he characterized it accurately by giving himself a "non-zero chance."

modeless 4y ago

Have you read these papers? It's absolutely feasible to get the gist of modern AI in a week of 16 hour days, as Carmack likes to do, for someone who is already world class at CS and has a mastery of college level math. Modern AI is very simple at its core! As Carmack mentions in the article, cutting edge models are implemented in a couple thousand lines of code, and the core algorithms are even less. Rust's borrow checker is more complex.

Tepix 4y ago
I'm pretty sure it took at least a couple of months to fundamentally understand 40 papers.