5 ms·
> Google et al. were at one point pinnacle technologies too, but this was 20 years ago. In 2017 Google literally gave us transformer architecture all current A
by nopurpose 1y ago
> Google et al. were at one point pinnacle technologies too, but this was 20 years ago.
In 2017 Google literally gave us transformer architecture all current AI boom is based on.
- noosphr 1y agoAnd what did they do with it for the next five years?
- Marazan 1y agoDamn, those goal posts moved fast.
- seanmcdirmid 1y agoUsed it to do things? This seems like a weird question. OpenAI took about the same amount of time to go big as well (Sam was excited about open AI in 2017, but it took 5+ years for it to pan out into something used by people).
- keeda 1y agoI think the point is that they hoarded the technology for internal use instead of opening it up to the public, like OpenAI did with ChatGPT, thus kicking off the current AI revolution. As sibling comments indicate, reasons may range from internal politics to innovator's dilemma. But the upshot is, even though the underlying technology was invented at Google, its inventors had to leave and join other companies to turn it into a publicly accessible innovation.
- seanmcdirmid 1y agoSo I started at Google in 2020 (after Sam closed our lab down in 2017 to focus on OpenAI), and if they were hoarding it, I at least had no clue about it. To be clear, my perspective is still limited.
- woooooo 1y agoI think "hoarding" is the wrong connotation. They were happy to have it be a fun research project alongside alphago while they continued making money from ads.
- keeda 1y agoFair enough, maybe a better way to put it is: why was the current AI boom sparked by ChatGPT and not something from Google? It's clear in retrospect that Google had similar capabilities in LaMDA, the precursor to Gemini. As I recall it was even announced a couple years before ChatGPT but wasn't released (as Bard?) until after ChatGPT. LaMDA is probably more famous for convincing a Google employee that it was sentient and getting him fired. When I heard that story I could not believe anybody could be deceived to that extent... until I saw ChatGPT. In hindsight, it was probably the first ever case of what is now called "AI psychosis". (Which may be a valid reason Google did not want to release it.)
- dekhn 1y agoGoogle had been burned badly in multiple previous launches of ML-based products and their leadership was extremely cautious about moving too quickly. It was convenient for Google that OpenAI acted as a first mover so that Google could enter the field after there was some level of cultural acceptance of the negative behaviors. There's a whole backstory where Noam Shazeer had come up with a bunch of nice innovations and wanted to launch them, but was only able to do so by leaving and launching through his startup- and then returned to Google, negotiating a phenomenal deal (Noam has been at Google for 25 years and has been doing various ML projects for much of that time).
- jimbo_joe 1y agoPre-ChatGPT OpenAI produced impressive RL results but their pivot to transformers was not guaranteed. With all internet data, infinite money, and ~800x more people, Google's internal LLMs were meh at best, probably because the innovators like Radford would constantly be snubbed by entrenched leaders (which almost happened in OpenAI).
- fooker 1y agoWell, there was this wild two year drama where they had people fight and smear each other over whether wasting energy for LLMs is ethical. https://www.cnet.com/tech/tech-industry/google-ai-chief-says-reputation-hit-to-unit-is-real-after-turmoil/ https://www.cnet.com/tech/tech-industry/google-ai-chief-says... That made plenty of scientists and engineers at google avoid AI for a while.
- canpan 1y agoThat does remind a little of Kodak, inventing the digital camera.
- HDThoreaun 1y agoand then sat on it for half a decade because they worried it would disrupt their search empire. Googles invention of transformers is a top 10 example of the innovators dilemma.