4 ms·

It seems like that being an incredible software engineer is more important these days, look at Greg Brockman’s background.

tempusalaria 4y ago
Right but Ilya is the Chief Scientist of OpenAI not Greg Brockman
- fdgsdfogijq 4y ago
  I heard Ilya wasnt behind the big innnovations at OpenAI. It was lesser known scientists
  - tempusalaria 4y ago
    OpenAI from a research point of view haven’t really had any “big innovations”. At least I struggle to think of any published research they have done that would qualify in that category. Probably they keep the good stuff for themselves But Ilya definitely had some big papers before and he is widely acknowledged as a top researcher in the field.
    - fdgsdfogijq 4y ago
      I think the fact that there are no other systems publicly available that are comparable to GPT-4 (and I dont think Bard is as good), points to innovation they havent released
      - bugglebeetle 4y ago
        Possibly. Or they could’ve also just paid a lot of people in Africa or wherever else to create the highest quality RLHF dataset out there.
        ragnarsson 4y ago
        They mentioned in the Technical paper of GPT-4 that the capabilities of the model were not from RLHF. https://youtu.be/2zW33LfffPc?t=842 https://youtu.be/2zW33LfffPc?t=842
earthboundkid 4y ago
That's my bias as well. To me, it seems like every day someone releases a new AI toy, but the thing you would actually want is for a real software engineer to take the LLM or whatever, put it inside a black box, and then write actually useful software around it. Like off the top of my head, LLM + Google Calendar = useful product for managing schedules and emailing people. You could make it in a day of tinkering as a langchain demo, but actually making a real product that is useful and doesn't suck will require good old fashioned software engineering.
- tempusalaria 4y ago
  Based on the multitask generalisation capabilities shown so far of LLMs I’m kinda in the opposite camp - if we can figure out more data efficient and reliable architectures base language models will likely be enough to do just about anything and take general instructions. Like you can just tell the language model to directly operate on Google calendar with suitable supplied permissions and it can do it no integration needed
  - BarryMilo 4y ago
    What you're describing is AGI levels of autonomy. There are quite a lot of missing pieces for that to happen I think.
    - danielmarkbruce 4y ago
      Have you used GPT-4? People are already building agents to do things the above comment refers to.
      - earthboundkid 4y ago
        People are building toy demos in a day that are not actual useable products. It’s cool, but it’s the difference between “I made a Twitter clone in a weekend” and real Twitter.
        4y ago

tempusalaria 4y ago

Right but Ilya is the Chief Scientist of OpenAI not Greg Brockman

fdgsdfogijq 4y ago
I heard Ilya wasnt behind the big innnovations at OpenAI. It was lesser known scientists
- tempusalaria 4y ago
  OpenAI from a research point of view haven’t really had any “big innovations”. At least I struggle to think of any published research they have done that would qualify in that category. Probably they keep the good stuff for themselves But Ilya definitely had some big papers before and he is widely acknowledged as a top researcher in the field.
  - fdgsdfogijq 4y ago
    I think the fact that there are no other systems publicly available that are comparable to GPT-4 (and I dont think Bard is as good), points to innovation they havent released
    - bugglebeetle 4y ago
      Possibly. Or they could’ve also just paid a lot of people in Africa or wherever else to create the highest quality RLHF dataset out there.
      - ragnarsson 4y ago
        They mentioned in the Technical paper of GPT-4 that the capabilities of the model were not from RLHF. https://youtu.be/2zW33LfffPc?t=842 https://youtu.be/2zW33LfffPc?t=842

earthboundkid 4y ago

That's my bias as well. To me, it seems like every day someone releases a new AI toy, but the thing you would actually want is for a real software engineer to take the LLM or whatever, put it inside a black box, and then write actually useful software around it. Like off the top of my head, LLM + Google Calendar = useful product for managing schedules and emailing people. You could make it in a day of tinkering as a langchain demo, but actually making a real product that is useful and doesn't suck will require good old fashioned software engineering.

tempusalaria 4y ago
Based on the multitask generalisation capabilities shown so far of LLMs I’m kinda in the opposite camp - if we can figure out more data efficient and reliable architectures base language models will likely be enough to do just about anything and take general instructions. Like you can just tell the language model to directly operate on Google calendar with suitable supplied permissions and it can do it no integration needed
- BarryMilo 4y ago
  What you're describing is AGI levels of autonomy. There are quite a lot of missing pieces for that to happen I think.
  - danielmarkbruce 4y ago
    Have you used GPT-4? People are already building agents to do things the above comment refers to.
    - earthboundkid 4y ago
      People are building toy demos in a day that are not actual useable products. It’s cool, but it’s the difference between “I made a Twitter clone in a weekend” and real Twitter.
      - 4y ago