4 ms·

Well, my take as a non-AI expert is that it was this paper (transformers) in combination with RLHF that made ChatGPT possible (with the former coming out of Goo

Well, my take as a non-AI expert is that it was this paper (transformers) in combination with RLHF that made ChatGPT possible (with the former coming out of Google and the latter, as far as I can tell a bunch of places including DeepMind, OpenAI, etc). That said, still mind blowing how far ahead Google was (is?) on research, data, compute, and how hard it is to actually novel things with that tech (vs OpenAI which... Just ships things).

vagrantJin 4y ago
> vs OpenAI which... Just ships things). Sam Altman practicing what he preached at YC for so many years. There's something wholly respectable about someone whose words closely match their actions. And haven't we all been better for it?
- machina_ex_deus 4y ago
  A reminder that there's nothing open about OpenAI. Stable diffusion is open. OpenAI started spewing morality nonsense once they got something they could capitalize on. From supposedly non profit when there was no profit to be made to fake moral high ground, to becoming completely closed. Hardly "practice what you preach". They may have delivered technologically, but they have completely backtracked on their openess promises. I ran stable diffusion at home. Never could do the same with anything out of "Open"AI
  - simonw 4y ago
    "nothing open" doesn't fully hold up there: Whisper is a (somewhat ground breaking) model released by OpenAI that you can run on your own hardware.
  - mensetmanusman 4y ago
    They are open about their methods and goals, not open source.
  - vagrantJin 4y ago
    > Hardly "practice what you preach". My comment was about Sam Altman doing what he was telling startups to do for many years at YC, ie - ship product and iterat
bigcloud1299 4y ago
Would you mind posting some of the papers (links)? I would love to read them. Thank you.
jimmySixDOF 4y ago
I heard the Kitchen thing release of their lamda work this week is shockingly bad by comparison. Not good news for those hoping Google had been keeping a pound for pound competitor to OpenAI in their back pocket.
cma 4y ago
Doesn't Google ship transformer stuff for translation? And some use in the search natural language summary/answer box, or is that hardcoded?

vagrantJin 4y ago

> vs OpenAI which... Just ships things). Sam Altman practicing what he preached at YC for so many years. There's something wholly respectable about someone whose words closely match their actions. And haven't we all been better for it?

machina_ex_deus 4y ago
A reminder that there's nothing open about OpenAI. Stable diffusion is open. OpenAI started spewing morality nonsense once they got something they could capitalize on. From supposedly non profit when there was no profit to be made to fake moral high ground, to becoming completely closed. Hardly "practice what you preach". They may have delivered technologically, but they have completely backtracked on their openess promises. I ran stable diffusion at home. Never could do the same with anything out of "Open"AI
- simonw 4y ago
  "nothing open" doesn't fully hold up there: Whisper is a (somewhat ground breaking) model released by OpenAI that you can run on your own hardware.
- mensetmanusman 4y ago
  They are open about their methods and goals, not open source.
- vagrantJin 4y ago
  > Hardly "practice what you preach". My comment was about Sam Altman doing what he was telling startups to do for many years at YC, ie - ship product and iterat

bigcloud1299 4y ago

Would you mind posting some of the papers (links)? I would love to read them. Thank you.

jimmySixDOF 4y ago

I heard the Kitchen thing release of their lamda work this week is shockingly bad by comparison. Not good news for those hoping Google had been keeping a pound for pound competitor to OpenAI in their back pocket.

cma 4y ago

Doesn't Google ship transformer stuff for translation? And some use in the search natural language summary/answer box, or is that hardcoded?