3 ms·

Hm, and what do you think begat all of this LLM fun in the first place?

inerte 4y ago
Very coincidentally I am trying to understand AI better, and I am reading "Regular Expression Puzzles and AI Coding Assistants" and it says on its first few pages: Due to transformer deep neural networks mostly pushed forward to a 2017 paper called "Attention is all you need". Prompted by your question I searched where the people who wrote this paper work at, and... (unsurprisingly drum roll), a team from Google.
- kccqzy 4y ago
  It is well-known that Google had world-class researchers working on AI for a long time now, and that they pioneered a lot of techniques used in the recent AI craze.
  - TechBro8615 4y ago
    And yet they have nothing to show for it other than some words on paper. Meanwhile OpenAI is eating their lunch by offering an actual product that people can use and interact with. Nobody cares that Google did the research they used to make it. Talk is cheap.
    - potatoman22 4y ago
      Their AI efforts go towards google search and ads.
  - zamnos 4y ago
    I don't think it's that well known, and for an advertising company, Google's marketing department doesn't seem very good. For instance, Google translate has been an ML model for more than a decade. They dropped a bunch of hand written code and replaced it with a model. The Google home assistants use an ML model to do their speech to text transcription. Google's tensor hardware is world class for ML research. But all people say is that Google search sucks now and wah they killed off Reader/Stadia/etc. Transformers will possibly have a bigger effect on society than Google search did and I'm doubtful Google will get proper recognition for that outside of circles like ours.
    - kccqzy 4y ago
      > Transformers will possibly have a bigger effect on society than Google search did and I'm doubtful Google will get proper recognition for that outside of circles like ours. Do you think the general public gives Xerox PARC a lot of credit for modern computing? I don't think so, and it's fine. It will just be well known among technology enthusiasts and historians of technology.
      - zamnos 4y ago
        It actually does bother me when people refer to Xerox as a failed photocopy machine company without acknowledging PARC, but I'm aware I'm in the minority here. I also think The Dream Machine should be required reading in a history class in all CS curriculum, but I'm not in academia. https://www.goodreads.com/en/book/show/722412 https://www.goodreads.com/en/book/show/722412
  - saagarjha 4y ago
    > had

Very coincidentally I am trying to understand AI better, and I am reading "Regular Expression Puzzles and AI Coding Assistants" and it says on its first few pages: Due to transformer deep neural networks mostly pushed forward to a 2017 paper called "Attention is all you need". Prompted by your question I searched where the people who wrote this paper work at, and... (unsurprisingly drum roll), a team from Google.

kccqzy 4y ago
It is well-known that Google had world-class researchers working on AI for a long time now, and that they pioneered a lot of techniques used in the recent AI craze.
- TechBro8615 4y ago
  And yet they have nothing to show for it other than some words on paper. Meanwhile OpenAI is eating their lunch by offering an actual product that people can use and interact with. Nobody cares that Google did the research they used to make it. Talk is cheap.
  - potatoman22 4y ago
    Their AI efforts go towards google search and ads.
- zamnos 4y ago
  I don't think it's that well known, and for an advertising company, Google's marketing department doesn't seem very good. For instance, Google translate has been an ML model for more than a decade. They dropped a bunch of hand written code and replaced it with a model. The Google home assistants use an ML model to do their speech to text transcription. Google's tensor hardware is world class for ML research. But all people say is that Google search sucks now and wah they killed off Reader/Stadia/etc. Transformers will possibly have a bigger effect on society than Google search did and I'm doubtful Google will get proper recognition for that outside of circles like ours.
  - kccqzy 4y ago
    > Transformers will possibly have a bigger effect on society than Google search did and I'm doubtful Google will get proper recognition for that outside of circles like ours. Do you think the general public gives Xerox PARC a lot of credit for modern computing? I don't think so, and it's fine. It will just be well known among technology enthusiasts and historians of technology.
    - zamnos 4y ago
      It actually does bother me when people refer to Xerox as a failed photocopy machine company without acknowledging PARC, but I'm aware I'm in the minority here. I also think The Dream Machine should be required reading in a history class in all CS curriculum, but I'm not in academia. https://www.goodreads.com/en/book/show/722412 https://www.goodreads.com/en/book/show/722412
- saagarjha 4y ago
  > had