6 ms·

Ilya knows how ChatGPT works. Any company that hires him will be able to catch up with ChatGPT.

threeseed 2y ago
You will still need the compute resources of Microsoft, Meta etc. And they have their own people who equally know how LLMs work. Even raising funds is not a certainty given that VCs are becoming more cautious with AI as they realise it's now a platform fight between the mega corporations.
- freshpretzels 2y ago
  > You will still need the compute resources of Microsoft, Meta etc. 2-3 years ago sure, but now? Maybe he knows how to reduce that need by order of magnitude.
- vanjajaja1 2y ago
  yep, and the puzzle isnt 'make chatgtp again' its 'catch up to chatgpt' which is a whole other puzzle
- zeroCalories 2y ago
  Plenty of companies have compute, but everyone is barely catching up to GPT-4 over a year after release. I'm sure places like Meta would love to unlock the details of what makes it so good.
  - CuriouslyC 2y ago
    There's no mystery, they just have a lot of actual user data, so they've been able to refine the question answering behavior. They've also baked some common problem solving strategies like chain of thought into the model via training.
- ignoramous 2y ago
  > Even raising funds is not a certainty given that VCs are becoming more cautious You'd think any investor would be utterly stupid to not make an exception for Ilya, regardless.
  - callalex 2y ago
    That’s not such a given. There are a toooon of deep pockets in this space already and absolutely zero path to profitable products for any of them.
mvkel 2y ago
Catching up right now is not a matter of tech innovation, but raw energy and compute. Of course, the next -revolution- in AI could very well come from Ilya. But why would he bestow that honor to anyone? He can self fund it if he wants. It's an R&D project, not a scaling problem.
- int_19h 2y ago
  If that is the case, why hasn't Google caught up yet?
  - Barrin92 2y ago
    probably because they have identified (correctly) that slowly integrating AI into their existing products that make them hundreds of billions is smarter than just burning that money to get upvoted on HN. Tall trees catch too much wind, ancient Chinese proverb. If you can afford to being second is usually less painful than being first.
    - A4ET8a8uTh0 2y ago
      There are clear advantages and disadvantages to that approach, but assuming this is the correct choice ( seems plausible, but I am not automatically convinced ), is that the actual reason for being as behind as they are? I would argue that is not the case. I won't re-list some of the reasons other posters mentioned, which, based on past year, appear more likely ( decisions hamstrung by corporate committees, data governance bureaucracy, and last, but not least, ideology focus ). Leadership that is actually focused on 'delivering value to the shareholder' or not being worried about first mover advantage seems only a part of it. edit: added first mover wording
  - gnarbarian 2y ago
    Google is ideologically compromised which ties them into knots and creates endless internal struggles. this led to the release of Gemini which was absurdly biased when viewed by the typical American.
  - nunez 2y ago
xyzzy123 2y ago
If his concern was irresponsible AI proliferation, too much commercial focus and wanting to move more carefully, accelerating competition doesn't seem like it would align with his goals.
tootie 2y ago
If he really is quitting over ethics he isn't going to Google or Meta. He'll probably go to Anthropic or academia. But who knows.
nothrowaways 2y ago
Everyone knows how chatGPT works.
gordon_freeman 2y ago
first Andrej and now Ilya. Talent exodus at OpenAI?
khazhoux 2y ago
I don't think it's good to lionize people like that, like some kind of tech Übermensch. He neither knows everything about ChatGPT, nor is he the only person there to know a whole lot about it.
menacingly 2y ago
I think what is probably very stressful about this space is virtually everyone knows how ChatGPT works. It is not a theoretical leap. It's actually fairly predictable how this shakes out, and OpenAI is pretty vulnerable. an LLM is a curiosity without user data, anyone with a big silo of data can put out something years behind frontier and still instantly see huge usage. No one wants to go to AI, they want AI to come to them, unless OpenAI can stake a claim in a super novel way they're the Dropbox. It's not like someone is going to use insider OpenAI knowledge to build an LLM so advanced you switch email, phone, or ERP providers
- ramraj07 2y ago
  No they don’t; if they did Google and Meta would have put out offerings that objectively beat OpenAI. However (barring temporary lapses) they’ve stayed ahead of the curve. Someone who constantly thinks low of their competition or people they hate is bound to fail.
  - menacingly 2y ago
    First, "objectively beat" doesn't matter. Comparative LLM performance means more or less nothing without application. Let's say Google's LLM has slightly poorer reasoning, and you have to be really clear when you tell it to delete your old emails. What are you going to do, go to ChatGPT and have it very eloquently walk you through how to manually delete your emails? But the idea that Google, the terrifying nation-state, who covetously gobbled up most of the bright minds of a generation, just couldn't fathom RLHF, means you think there is some inherent magic at OpenAI. I use OpenAI's models a lot, obviously they're great, but whole-thread-as-context as a product is not a product, and Google has people plural who could execute that from first principles.
- visarga 2y ago
  Last time such a departure led to the creation of Anthropic (Dario Amodei), they even equalled GPT-4's performance
  - menacingly 2y ago

You will still need the compute resources of Microsoft, Meta etc. And they have their own people who equally know how LLMs work. Even raising funds is not a certainty given that VCs are becoming more cautious with AI as they realise it's now a platform fight between the mega corporations.

freshpretzels 2y ago
> You will still need the compute resources of Microsoft, Meta etc. 2-3 years ago sure, but now? Maybe he knows how to reduce that need by order of magnitude.
vanjajaja1 2y ago
yep, and the puzzle isnt 'make chatgtp again' its 'catch up to chatgpt' which is a whole other puzzle
zeroCalories 2y ago
Plenty of companies have compute, but everyone is barely catching up to GPT-4 over a year after release. I'm sure places like Meta would love to unlock the details of what makes it so good.
- CuriouslyC 2y ago
  There's no mystery, they just have a lot of actual user data, so they've been able to refine the question answering behavior. They've also baked some common problem solving strategies like chain of thought into the model via training.
ignoramous 2y ago
> Even raising funds is not a certainty given that VCs are becoming more cautious You'd think any investor would be utterly stupid to not make an exception for Ilya, regardless.
- callalex 2y ago
  That’s not such a given. There are a toooon of deep pockets in this space already and absolutely zero path to profitable products for any of them.

mvkel 2y ago

Catching up right now is not a matter of tech innovation, but raw energy and compute. Of course, the next -revolution- in AI could very well come from Ilya. But why would he bestow that honor to anyone? He can self fund it if he wants. It's an R&D project, not a scaling problem.

int_19h 2y ago
If that is the case, why hasn't Google caught up yet?
- Barrin92 2y ago
  probably because they have identified (correctly) that slowly integrating AI into their existing products that make them hundreds of billions is smarter than just burning that money to get upvoted on HN. Tall trees catch too much wind, ancient Chinese proverb. If you can afford to being second is usually less painful than being first.
  - A4ET8a8uTh0 2y ago
    There are clear advantages and disadvantages to that approach, but assuming this is the correct choice ( seems plausible, but I am not automatically convinced ), is that the actual reason for being as behind as they are? I would argue that is not the case. I won't re-list some of the reasons other posters mentioned, which, based on past year, appear more likely ( decisions hamstrung by corporate committees, data governance bureaucracy, and last, but not least, ideology focus ). Leadership that is actually focused on 'delivering value to the shareholder' or not being worried about first mover advantage seems only a part of it. edit: added first mover wording
- gnarbarian 2y ago
  Google is ideologically compromised which ties them into knots and creates endless internal struggles. this led to the release of Gemini which was absurdly biased when viewed by the typical American.
- nunez 2y ago

xyzzy123 2y ago

If his concern was irresponsible AI proliferation, too much commercial focus and wanting to move more carefully, accelerating competition doesn't seem like it would align with his goals.

tootie 2y ago

If he really is quitting over ethics he isn't going to Google or Meta. He'll probably go to Anthropic or academia. But who knows.

nothrowaways 2y ago

Everyone knows how chatGPT works.

gordon_freeman 2y ago

first Andrej and now Ilya. Talent exodus at OpenAI?

khazhoux 2y ago

I don't think it's good to lionize people like that, like some kind of tech Übermensch. He neither knows everything about ChatGPT, nor is he the only person there to know a whole lot about it.

menacingly 2y ago

I think what is probably very stressful about this space is virtually everyone knows how ChatGPT works. It is not a theoretical leap. It's actually fairly predictable how this shakes out, and OpenAI is pretty vulnerable. an LLM is a curiosity without user data, anyone with a big silo of data can put out something years behind frontier and still instantly see huge usage. No one wants to go to AI, they want AI to come to them, unless OpenAI can stake a claim in a super novel way they're the Dropbox. It's not like someone is going to use insider OpenAI knowledge to build an LLM so advanced you switch email, phone, or ERP providers

ramraj07 2y ago
No they don’t; if they did Google and Meta would have put out offerings that objectively beat OpenAI. However (barring temporary lapses) they’ve stayed ahead of the curve. Someone who constantly thinks low of their competition or people they hate is bound to fail.
- menacingly 2y ago
  First, "objectively beat" doesn't matter. Comparative LLM performance means more or less nothing without application. Let's say Google's LLM has slightly poorer reasoning, and you have to be really clear when you tell it to delete your old emails. What are you going to do, go to ChatGPT and have it very eloquently walk you through how to manually delete your emails? But the idea that Google, the terrifying nation-state, who covetously gobbled up most of the bright minds of a generation, just couldn't fathom RLHF, means you think there is some inherent magic at OpenAI. I use OpenAI's models a lot, obviously they're great, but whole-thread-as-context as a product is not a product, and Google has people plural who could execute that from first principles.
visarga 2y ago
Last time such a departure led to the creation of Anthropic (Dario Amodei), they even equalled GPT-4's performance
- menacingly 2y ago