5 ms·

Jeff Dean is clearly one of the greatest software developers/engineers ever but there isn’t much evidence that he is a brilliant ML researcher And indeed Googl

Jeff Dean is clearly one of the greatest software developers/engineers ever but there isn’t much evidence that he is a brilliant ML researcher And indeed Google AI has achieved very little product wise during his time as CEO. Kind of suggests he is a big part of bureaucratic challenges they have faced

VirusNewbie 3y ago
It seems like that being an incredible software engineer is more important these days, look at Greg Brockman’s background.
- tempusalaria 3y ago
  Right but Ilya is the Chief Scientist of OpenAI not Greg Brockman
  - fdgsdfogijq 3y ago
    I heard Ilya wasnt behind the big innnovations at OpenAI. It was lesser known scientists
    - tempusalaria 3y ago
      OpenAI from a research point of view haven’t really had any “big innovations”. At least I struggle to think of any published research they have done that would qualify in that category. Probably they keep the good stuff for themselves But Ilya definitely had some big papers before and he is widely acknowledged as a top researcher in the field.
      - fdgsdfogijq 3y ago
        I think the fact that there are no other systems publicly available that are comparable to GPT-4 (and I dont think Bard is as good), points to innovation they havent released
        bugglebeetle 3y ago
        Possibly. Or they could’ve also just paid a lot of people in Africa or wherever else to create the highest quality RLHF dataset out there.
        ragnarsson 3y ago
        They mentioned in the Technical paper of GPT-4 that the capabilities of the model were not from RLHF. https://youtu.be/2zW33LfffPc?t=842 https://youtu.be/2zW33LfffPc?t=842
- earthboundkid 3y ago
  That's my bias as well. To me, it seems like every day someone releases a new AI toy, but the thing you would actually want is for a real software engineer to take the LLM or whatever, put it inside a black box, and then write actually useful software around it. Like off the top of my head, LLM + Google Calendar = useful product for managing schedules and emailing people. You could make it in a day of tinkering as a langchain demo, but actually making a real product that is useful and doesn't suck will require good old fashioned software engineering.
  - tempusalaria 3y ago
    Based on the multitask generalisation capabilities shown so far of LLMs I’m kinda in the opposite camp - if we can figure out more data efficient and reliable architectures base language models will likely be enough to do just about anything and take general instructions. Like you can just tell the language model to directly operate on Google calendar with suitable supplied permissions and it can do it no integration needed
    - BarryMilo 3y ago
      What you're describing is AGI levels of autonomy. There are quite a lot of missing pieces for that to happen I think.
      - danielmarkbruce 3y ago
        Have you used GPT-4? People are already building agents to do things the above comment refers to.
        earthboundkid 3y ago
        People are building toy demos in a day that are not actual useable products. It’s cool, but it’s the difference between “I made a Twitter clone in a weekend” and real Twitter.
        3y ago
forgot-my-pw 3y ago
He worked on TensorFlow. So even if the doesn't do ML research himself, at least he works on the tooling.
- tempusalaria 3y ago
  Of course he is someone any technology organisation would want to have as a resource. But probably not as chief scientist or ceo of an ML company based on the available evidence
- modeless 3y ago
  TensorFlow was honestly not that good. It had a lot of effort put into it, so it worked, but there are reasons people moved away from it. I think Jeff Dean is a great engineer, but I wouldn't hold up TensorFlow as a great example.
  - theGnuMe 3y ago
    I always thought he pair programmed so it's been Jeff + Sanjay.
  - VirusNewbie 3y ago
    Why wasn’t tensorflow good?
    - Dr_Birdbrain 3y ago
      I started my ML journey in TensorFlow, and now I am a happy PyTorch user, for many years. TensorFlow, yuck
- hungryforcodes 3y ago
  But who uses TensorFlow -- really -- these days.
  - hyperhopper 3y ago
    What else are people using then?
bartwr 3y ago
Furthermore, Jeff is not a great (or even good...) manager/director/leader. There were a lot of internal and external dramas because of his leadership, that he failed to address. How often you hear about dramas about other Chef Scientists at other, comparably sized, companies? He should stay a Fellow, in a "brilliant consultant" role.
- ragnarsson 3y ago
  He absolutely should have gotten rid of the troublemaker. Many folks used this publicity to leave for higher positions or higher pay (which is very common at google) but made it look like Jeff Dean was the problem.
- deleted 3y ago
  [deleted]
kortilla 3y ago
Being a brilliant ML researcher has approximately zero overlap with making products for people to use.
ReptileMan 3y ago
>Jeff Dean is clearly one of the greatest software developers/engineers ever but there isn’t much evidence that he is a brilliant ML researcher Google have oversupply of brilliant ML researchers. What they need is a engineer that sees the applications of the technology so it can be turned into a product. Someone that can bridge the gap between the R&D team and the Bureaucracy. Want an idea for a stupid product - input - description of a girl, hobbies, some minor flaws - output - create a poem. Have been using Vicuna quite successfully for that purpose.
karmasimida 3y ago
He had the perfect balance of being legendary engineer and an ML researcher Can't emphasize more on how much rigorous engineering practice could accelerate research delivery. It is THE key to have a productive research oriented team. Good research engineers are underrated, and very difficult to find.
HarHarVeryFunny 3y ago
> Jeff Dean is clearly one of the greatest software developers/engineers ever Based on what? I've heard all the Chuck Norris type jokes, but what has Jeff Dean actually accomplished that is so legendary as a software developer (or as a leader) ? Per his Google bio/CV his main claims to fame seem to have been work on large scale infrastructure projects such as BigTable, MapReduce, Protobuf and TensorFlow, which seem more like solid engineering accomplishments rather than the stuff of legend. https://research.google/people/jeff/ https://research.google/people/jeff/ Seems like he's perhaps being rewarded with the title of "Chief Scientist" rather than necessarily suited to it, but I guess that depends on what Sundar is expecting out of him.
- ericjang 3y ago
  Jeff was very early on in the "just scale up the big brain" idea, perhaps as early as 2012 (Andrew Ng training networks on 1000s of CPUs). This vision is sort of summarized in https://blog.google/technology/ai/introducing-pathways-next-generation-ai-architecture/ https://blog.google/technology/ai/introducing-pathways-next-... and fleshed out more in https://arxiv.org/abs/2203.12533 https://arxiv.org/abs/2203.12533, but he had been internally promoting this idea since before 2016. When I joined Brain in 2016, I had thought the idea of training billion/trillion-parameter sparsely gated mixtures of experts was a huge waste of resources, and that the idea was incredibly naive. But it turns out he was right, and it would take ~6 more years before that was abundantly obvious to the rest of the research community. Here's his scholar page (H index of 94) https://scholar.google.com/citations?hl=en&user=NMS69lQAAAAJ&view_op=list_works https://scholar.google.com/citations?hl=en&user=NMS69lQAAAAJ... As a leader, he also managed the development of TensorFlow and TPU. Consider the context / time frame - the year is 2014/2015 and a lot of academics still don't believe deep learning works. Jeff pivots a >100-person org to go all-in on deep learning, invest in an upgraded version of Theano (TF) and then give it away to the community for free, and develop Google's own training chip to compete with Nvidia. These are highly non-obvious ideas that show much more spine & vision than most tech leaders. Not to mention he designed & coded large parts of TF himself! And before that, he was doing systems engineering on non-ML stuff. It's rare to pivot as a very senior-level engineer to a completely new field and then do what he did. Jeff certainly has made mistakes as a leader (failing to translate Google Brain's numerous fundamental breakthroughs to more ambitious AI products, and consolidating the redundant big model efforts in google research) but I would consider his high level directional bets to be incredibly prescient.

VirusNewbie 3y ago

It seems like that being an incredible software engineer is more important these days, look at Greg Brockman’s background.

tempusalaria 3y ago
Right but Ilya is the Chief Scientist of OpenAI not Greg Brockman
- fdgsdfogijq 3y ago
  I heard Ilya wasnt behind the big innnovations at OpenAI. It was lesser known scientists
  - tempusalaria 3y ago
    OpenAI from a research point of view haven’t really had any “big innovations”. At least I struggle to think of any published research they have done that would qualify in that category. Probably they keep the good stuff for themselves But Ilya definitely had some big papers before and he is widely acknowledged as a top researcher in the field.
    - fdgsdfogijq 3y ago
      I think the fact that there are no other systems publicly available that are comparable to GPT-4 (and I dont think Bard is as good), points to innovation they havent released
      - bugglebeetle 3y ago
        Possibly. Or they could’ve also just paid a lot of people in Africa or wherever else to create the highest quality RLHF dataset out there.
        ragnarsson 3y ago
        They mentioned in the Technical paper of GPT-4 that the capabilities of the model were not from RLHF. https://youtu.be/2zW33LfffPc?t=842 https://youtu.be/2zW33LfffPc?t=842
earthboundkid 3y ago
That's my bias as well. To me, it seems like every day someone releases a new AI toy, but the thing you would actually want is for a real software engineer to take the LLM or whatever, put it inside a black box, and then write actually useful software around it. Like off the top of my head, LLM + Google Calendar = useful product for managing schedules and emailing people. You could make it in a day of tinkering as a langchain demo, but actually making a real product that is useful and doesn't suck will require good old fashioned software engineering.
- tempusalaria 3y ago
  Based on the multitask generalisation capabilities shown so far of LLMs I’m kinda in the opposite camp - if we can figure out more data efficient and reliable architectures base language models will likely be enough to do just about anything and take general instructions. Like you can just tell the language model to directly operate on Google calendar with suitable supplied permissions and it can do it no integration needed
  - BarryMilo 3y ago
    What you're describing is AGI levels of autonomy. There are quite a lot of missing pieces for that to happen I think.
    - danielmarkbruce 3y ago
      Have you used GPT-4? People are already building agents to do things the above comment refers to.
      - earthboundkid 3y ago
        People are building toy demos in a day that are not actual useable products. It’s cool, but it’s the difference between “I made a Twitter clone in a weekend” and real Twitter.
        3y ago

forgot-my-pw 3y ago

He worked on TensorFlow. So even if the doesn't do ML research himself, at least he works on the tooling.

tempusalaria 3y ago
Of course he is someone any technology organisation would want to have as a resource. But probably not as chief scientist or ceo of an ML company based on the available evidence
modeless 3y ago
TensorFlow was honestly not that good. It had a lot of effort put into it, so it worked, but there are reasons people moved away from it. I think Jeff Dean is a great engineer, but I wouldn't hold up TensorFlow as a great example.
- theGnuMe 3y ago
  I always thought he pair programmed so it's been Jeff + Sanjay.
- VirusNewbie 3y ago
  Why wasn’t tensorflow good?
  - Dr_Birdbrain 3y ago
    I started my ML journey in TensorFlow, and now I am a happy PyTorch user, for many years. TensorFlow, yuck
hungryforcodes 3y ago
But who uses TensorFlow -- really -- these days.
- hyperhopper 3y ago
  What else are people using then?

bartwr 3y ago

Furthermore, Jeff is not a great (or even good...) manager/director/leader. There were a lot of internal and external dramas because of his leadership, that he failed to address. How often you hear about dramas about other Chef Scientists at other, comparably sized, companies? He should stay a Fellow, in a "brilliant consultant" role.

ragnarsson 3y ago
He absolutely should have gotten rid of the troublemaker. Many folks used this publicity to leave for higher positions or higher pay (which is very common at google) but made it look like Jeff Dean was the problem.
deleted 3y ago
[deleted]

kortilla 3y ago

Being a brilliant ML researcher has approximately zero overlap with making products for people to use.

ReptileMan 3y ago

>Jeff Dean is clearly one of the greatest software developers/engineers ever but there isn’t much evidence that he is a brilliant ML researcher Google have oversupply of brilliant ML researchers. What they need is a engineer that sees the applications of the technology so it can be turned into a product. Someone that can bridge the gap between the R&D team and the Bureaucracy. Want an idea for a stupid product - input - description of a girl, hobbies, some minor flaws - output - create a poem. Have been using Vicuna quite successfully for that purpose.

karmasimida 3y ago

He had the perfect balance of being legendary engineer and an ML researcher Can't emphasize more on how much rigorous engineering practice could accelerate research delivery. It is THE key to have a productive research oriented team. Good research engineers are underrated, and very difficult to find.

HarHarVeryFunny 3y ago

> Jeff Dean is clearly one of the greatest software developers/engineers ever Based on what? I've heard all the Chuck Norris type jokes, but what has Jeff Dean actually accomplished that is so legendary as a software developer (or as a leader) ? Per his Google bio/CV his main claims to fame seem to have been work on large scale infrastructure projects such as BigTable, MapReduce, Protobuf and TensorFlow, which seem more like solid engineering accomplishments rather than the stuff of legend. https://research.google/people/jeff/ https://research.google/people/jeff/ Seems like he's perhaps being rewarded with the title of "Chief Scientist" rather than necessarily suited to it, but I guess that depends on what Sundar is expecting out of him.

ericjang 3y ago
Jeff was very early on in the "just scale up the big brain" idea, perhaps as early as 2012 (Andrew Ng training networks on 1000s of CPUs). This vision is sort of summarized in https://blog.google/technology/ai/introducing-pathways-next-generation-ai-architecture/ https://blog.google/technology/ai/introducing-pathways-next-... and fleshed out more in https://arxiv.org/abs/2203.12533 https://arxiv.org/abs/2203.12533, but he had been internally promoting this idea since before 2016. When I joined Brain in 2016, I had thought the idea of training billion/trillion-parameter sparsely gated mixtures of experts was a huge waste of resources, and that the idea was incredibly naive. But it turns out he was right, and it would take ~6 more years before that was abundantly obvious to the rest of the research community. Here's his scholar page (H index of 94) https://scholar.google.com/citations?hl=en&user=NMS69lQAAAAJ&view_op=list_works https://scholar.google.com/citations?hl=en&user=NMS69lQAAAAJ... As a leader, he also managed the development of TensorFlow and TPU. Consider the context / time frame - the year is 2014/2015 and a lot of academics still don't believe deep learning works. Jeff pivots a >100-person org to go all-in on deep learning, invest in an upgraded version of Theano (TF) and then give it away to the community for free, and develop Google's own training chip to compete with Nvidia. These are highly non-obvious ideas that show much more spine & vision than most tech leaders. Not to mention he designed & coded large parts of TF himself! And before that, he was doing systems engineering on non-ML stuff. It's rare to pivot as a very senior-level engineer to a completely new field and then do what he did. Jeff certainly has made mistakes as a leader (failing to translate Google Brain's numerous fundamental breakthroughs to more ambitious AI products, and consolidating the redundant big model efforts in google research) but I would consider his high level directional bets to be incredibly prescient.