8 ms·

I wonder what happened at Google. They basically invented Transformers, and had useful LLMs for a while. Bert, Lambda, Palm, Sparrow were all cutting edge. Wh

I wonder what happened at Google. They basically invented Transformers, and had useful LLMs for a while. Bert, Lambda, Palm, Sparrow were all cutting edge. Why didn't they ship something like ChatGPT earlier? Lack of product vision? Perfectionism? Is this Xerox vs Apple all over again ?

k__ 4y ago
Maybe they thought they could do better than just "building a bigger model".
michaellee8 4y ago
They do have lamda and it is available for test in their AI test kitchen. Seems much better handling of sensetive and offensive content then ChatGPT for me, but still cannot perform basic addition like ChatGPT does. I think it is technically better than ChatGPT but maybe they are only going to release the perfect product. Tbf ChatGPT was far from production quality for serious applications, lots of misinformation and you can make it produce very offensive content. It is a good for toying around but you cannot take the output seriously.
- visarga 4y ago
  I think a token effort to avoid offensive content is ok, but chatGPT should quickly detect if the human wants to go outside the box and allow it. If a human pushes it means they understand the risks and take full responsibility for the outcome.
  - rvnx 4y ago
    This is not how Google's AI Test Kitchen is designed. AI Test Kitchen seems quite boring and very framed system, where you can ask what is the best Dyson model for example, or the old-style "GPT dungeon game", it doesn't really go off-rail (this is part of the product specifications sadly :/).
  - LoganDark 4y ago
    > chatGPT should quickly detect if the human wants to go outside the box and allow it This is why "jailbreaking" is a thing. Once you convince the model that it's OK, it'll let you do anything from then on. -Emily
  - criley2 4y ago
    I couldn't disagree more. ChatGPT would be extremely easy to convert to "HateGPT", and would be able to create some pretty powerful and useful political, racial, etc propaganda. I think it's right that the owners understand what the weaponization of ChatGPT could do and prevent it, and I think we need laws (and fast) before weaponized AI like ChatGPT turns into a disaster for humanity
- bsenftner 4y ago
  My experience it is like working with a genius idiot, the type that refuses to be wrong, which means the shithead (if it were human) requires verification and curation. So what if I need to verify? I do that anyway, because people have imperfect memory, documentation is often old, and who knows what unexpected whatever could be impacting my expectations. I welcome idiot savants.
- jimmySixDOF 4y ago
  People I know say the lamda Kitchen release is unbelievably limited by comparison. A Kitchen session has three sections: The 'Ask a question' prompt is limited to under 100 characters and response is like the existing Google search question snips. The 'Make a List' section is just lists like as in short bullet points. And the 'Creative' section is limited to respond with stories involving dogs for which is a little bizarre to say the least.
londons_explore 4y ago
> I wonder what happened at Google. They suffer from "scared to release an imperfect product that damages PR"-itis. They are so scared that an AI with Google branding might say something inappropriate, that they prefer to release nothing at all. For context, they put a lot of engineering effort into making sure "Okay Google" for the voice assistant couldn't be modded to "Yo Dick" or something else off-brand, simply because they were worried about media backlash.
- onlyrealcuzzo 4y ago
  So why not lease the tech to a different company under a different brand they own and release it? Nobody knows FB owns Instagram. Nobody is going to care if Google releases a ChatGPT bot on a different website.
  - londons_explore 4y ago
    That's what I'd do... specifically, I'd license Google tech to ex-google employees who want to start a company, in return for an 80% ownership stake. It would be a 'you can use all google internal code and infrastructure, for free, as long as you don't use google branding and don't use our user data' deal. The companies would have the benefits of google tech, without being tied down by internal politics, procedures, risk management, etc. They could afford to do things like just release a product to the world rather than have a legal and PR team go over every detail for every law in every country worldwide as part of a 6 month long review process.
    - hooo 4y ago
      Lol, think from the founder / teams perspectives. Giving 80% ownership would be insane. How could you build a company like that? It’s not so simple.
      - unicornmama 4y ago
        The sort of people who would accept those terms are not the sort of people you want to build a startup.
joshxyz 4y ago
They'll shut it down anyways that's why, it's hard to trust them these days. LLMs require years of commitment, check out google graveyard for list of products that also needed that kind of commitment.
spaceman_2020 4y ago
They’re REALLY fumbling the bag with search and maps lately as well. My experience with both as a user has never been worse. I’ve flirted with exiting Google search before but this time it got bad enough that I fully committed to Kagi.
- trpv 4y ago
  I've used DuckDuckGo for a couple of years now, so I've gotten very used to it, and on the odd occasion I try to use Google, the amount of ads, different sections with news, images, products, and other unnecessary clutter that isnt actual search results makes for a really uncomfortable experience
deleted 4y ago
[deleted]
flandish 4y ago
They’re a stockholder owned corporation now. Beholden to profit above all else. Making things like layoffs, hedged bets, and scared-to-execute moves all the more common. Remember when they were “don’t be evil?” - can’t be not evil, when on the market..
- astrange 4y ago
  > They’re a stockholder owned corporation now. Beholden to profit above all else. Neither of these two things are true. It's 51% owned by the founders and public companies are not required to only care about profit.
  - flandish 4y ago
    Your reply’s logic shows my point. 49% of the company can be purchased via the market. It is a public company. Public companies are legally required to put shareholder interest first - that’s what the whole point of this facet of capitalism is: to provide profit to the “owner” class of society, at the cost of worker exploitation. Get Adam Smith’s boot out of your mouth and recognize that literally anything a corporation does is for the goal of profit. Corporations are not human - and only have a decision making process that puts “return” first. Zero of them are “not evil.”
    - astrange 4y ago
      > Public companies are legally required to put shareholder interest first No, they are not. And they definitely aren't required to care about the interests of 49% of them. > Get Adam Smith’s boot out of your mouth and recognize that literally anything a corporation does is for the goal of profit. Is everything an animal does for the purpose of finding food? (No.)
      - Psychoshy_bc1q 4y ago
        everything an animal does is for the purpose of reproduction lol
bsenftner 4y ago
> I wonder what happened at Google. Finance and management, knowing their bread and butter is as advertising distribution company, undermine, kill, and dilute any projects that are not obviously, undeniably directly tied to improving their advertising distribution revenues. Their big problem is, they never communicated this core identity to the larger corporation, and somehow search is allowed to coast...
nodja 4y ago
They made T5 and released the weights, while T5 "only" goes up to 11B params, it's an efficient model for a wide variety of tasks and the base for a lot of other models/projects. Of note imagen[1] and open-assistant[2] use T5 and FlanT5. [1] https://imagen.research.google/ https://imagen.research.google/ [2] https://github.com/LAION-AI/Open-Assistant https://github.com/LAION-AI/Open-Assistant
dna_polymerase 4y ago
The outside world gets a glimpse into what is happening at Google right now by looking at the people who leave Google. I wouldn't have been surprised if Google had invested in Character, a startup which was formed by Ex-Google LM people. To cite The Information [0]: "At Google, Character’s co-founders Shazeer and Daniel De Freitas were instrumental in developing Google’s LaMDA language model, used for natural language processing research. They decided to launch their own startup after getting frustrated with big company bureaucracies, Shazeer previously told The Information." [0]: https://www.theinformation.com/articles/character-seeks-250-million-in-new-funding-amid-ai-boom?rc=bhi2pj https://www.theinformation.com/articles/character-seeks-250-...
mrtksn 4y ago
People say its to preserve the brand and be woke etc. but IMHO it is simply because doesn't fit Google's business. ChatGPT is very cool but they don't make money from it and GPT-3 had a paid public API for a year now and I read that the money they make from it is very little, insignificant compared to the funding they get from VCs. Currently, there's no clear path for monetisation but has the capacity to take away a lot of Google's use cases. In essence, it's like Napster destroying the Music CD business. Maybe it's also a bit like Kodak: Invents the technology but its current business is too good to be messed around with. This new tech is the future but we are happy in the moment.
- LewisVerstappen 4y ago
  Well, every single executive at Google has read the Innovator's Dilemma multiple times, so I doubt that's some novel insight that no one at Google has thought of.
  - deleted 4y ago
    [deleted]
- risyachka 4y ago
  Pretty much this. If Google wanted - they could've easily released ChatGPT version that is 100x better than current one considering they invented the technology and they have thousands times more and better data to train it. They just weren't interested.
  - 0XYZxyzXYZ0 4y ago
    This doesn't make sense. They did invest in PaLM, which is ~2.5 larger than GPT-3, but no where near 100x better. I do agree that the limiting factor is the data, not scale though. What data has Google that OpenAI (or any other LLM company) can't scrape from the web as well? Using Gmail data without consent would be definitely a move towards a mountain of law suits.
    - CuriouslyC 4y ago
monkeydust 4y ago
Something to be said for '“Move fast and break things.” Zuckerberg
richardw 4y ago
Because ChatGPT isn’t fully baked yet. You can get it to say the stupidest stuff, which is perfect for a research release with somewhat sophisticated users but when it’s “how granny gets information” is when the lawyers get excited. It doesn’t just filter info it finds on the internet, it generates it from whole cloth. This is the Tesla FSD for information - it’ll confidently drive you into a pole with a smile on its face. I love it, but it’s not ready for all Google users yet.
mkl95 4y ago
> Why didn't they ship something like ChatGPT earlier? Lack of product vision? Perfectionism? The simplest answer is that the people with the know-how did not work at Google. Knowing why they didn't is not so simple, though.
CommanderData 4y ago
Probably the lack of freedom to innovate or middle management suppressing free thought stopping cool stuff being made in a devs/eng spare downtime. I've seen so many cool ideas thrown around by devs that could actually go onto be monetized by companies go wasted. To think they'd be worked on for free (spare time) and often better then outsourcing because nothing drives true innovation than passion. Focus on BAU, keep your head down and be lucky you're not getting fired.
joxel 4y ago
Probably only hiring people that cram leetcode instead of actual smart people.
peterisza 4y ago
Lack of balls. Same as Kodak, etc. The fact that they are freaking out is actually a good sign though
habibur 4y ago
They thought it was too powerful to be considered safe for release. And their ad revenue nevertheless had been great even without it. No rush. They back stepped, and then watched someone else capture the market.
daydream 4y ago
Google is too big to seriously innovate. No top down focus or vision just recent Stanford graduates hawking random ideas. Then there’s the promo culture where everyone is busy trying to extract as much comp as possible instead of building. (I’m guessing)
- deleted 4y ago
  [deleted]
hanniabu 4y ago
> Why didn't they ship something like ChatGPT earlier? I would say they have just in different form. For a few years now when you google something you're given a quote/excerpt at the top of the results that typically answers your question.
onion2k 4y ago
Why didn't they ship something like ChatGPT earlier? In order to launch something you have to need to launch it. Either because you're losing market share, or because you need the revenue, or because your customers expect it. Google rarely has any of those things.

k__ 4y ago

Maybe they thought they could do better than just "building a bigger model".

michaellee8 4y ago

They do have lamda and it is available for test in their AI test kitchen. Seems much better handling of sensetive and offensive content then ChatGPT for me, but still cannot perform basic addition like ChatGPT does. I think it is technically better than ChatGPT but maybe they are only going to release the perfect product. Tbf ChatGPT was far from production quality for serious applications, lots of misinformation and you can make it produce very offensive content. It is a good for toying around but you cannot take the output seriously.

visarga 4y ago
I think a token effort to avoid offensive content is ok, but chatGPT should quickly detect if the human wants to go outside the box and allow it. If a human pushes it means they understand the risks and take full responsibility for the outcome.
- rvnx 4y ago
  This is not how Google's AI Test Kitchen is designed. AI Test Kitchen seems quite boring and very framed system, where you can ask what is the best Dyson model for example, or the old-style "GPT dungeon game", it doesn't really go off-rail (this is part of the product specifications sadly :/).
- LoganDark 4y ago
  > chatGPT should quickly detect if the human wants to go outside the box and allow it This is why "jailbreaking" is a thing. Once you convince the model that it's OK, it'll let you do anything from then on. -Emily
- criley2 4y ago
  I couldn't disagree more. ChatGPT would be extremely easy to convert to "HateGPT", and would be able to create some pretty powerful and useful political, racial, etc propaganda. I think it's right that the owners understand what the weaponization of ChatGPT could do and prevent it, and I think we need laws (and fast) before weaponized AI like ChatGPT turns into a disaster for humanity
bsenftner 4y ago
My experience it is like working with a genius idiot, the type that refuses to be wrong, which means the shithead (if it were human) requires verification and curation. So what if I need to verify? I do that anyway, because people have imperfect memory, documentation is often old, and who knows what unexpected whatever could be impacting my expectations. I welcome idiot savants.
jimmySixDOF 4y ago
People I know say the lamda Kitchen release is unbelievably limited by comparison. A Kitchen session has three sections: The 'Ask a question' prompt is limited to under 100 characters and response is like the existing Google search question snips. The 'Make a List' section is just lists like as in short bullet points. And the 'Creative' section is limited to respond with stories involving dogs for which is a little bizarre to say the least.

londons_explore 4y ago

> I wonder what happened at Google. They suffer from "scared to release an imperfect product that damages PR"-itis. They are so scared that an AI with Google branding might say something inappropriate, that they prefer to release nothing at all. For context, they put a lot of engineering effort into making sure "Okay Google" for the voice assistant couldn't be modded to "Yo Dick" or something else off-brand, simply because they were worried about media backlash.

onlyrealcuzzo 4y ago
So why not lease the tech to a different company under a different brand they own and release it? Nobody knows FB owns Instagram. Nobody is going to care if Google releases a ChatGPT bot on a different website.
- londons_explore 4y ago
  That's what I'd do... specifically, I'd license Google tech to ex-google employees who want to start a company, in return for an 80% ownership stake. It would be a 'you can use all google internal code and infrastructure, for free, as long as you don't use google branding and don't use our user data' deal. The companies would have the benefits of google tech, without being tied down by internal politics, procedures, risk management, etc. They could afford to do things like just release a product to the world rather than have a legal and PR team go over every detail for every law in every country worldwide as part of a 6 month long review process.
  - hooo 4y ago
    Lol, think from the founder / teams perspectives. Giving 80% ownership would be insane. How could you build a company like that? It’s not so simple.
    - unicornmama 4y ago
      The sort of people who would accept those terms are not the sort of people you want to build a startup.

joshxyz 4y ago

They'll shut it down anyways that's why, it's hard to trust them these days. LLMs require years of commitment, check out google graveyard for list of products that also needed that kind of commitment.

spaceman_2020 4y ago

They’re REALLY fumbling the bag with search and maps lately as well. My experience with both as a user has never been worse. I’ve flirted with exiting Google search before but this time it got bad enough that I fully committed to Kagi.

trpv 4y ago
I've used DuckDuckGo for a couple of years now, so I've gotten very used to it, and on the odd occasion I try to use Google, the amount of ads, different sections with news, images, products, and other unnecessary clutter that isnt actual search results makes for a really uncomfortable experience

deleted 4y ago

[deleted]

flandish 4y ago

They’re a stockholder owned corporation now. Beholden to profit above all else. Making things like layoffs, hedged bets, and scared-to-execute moves all the more common. Remember when they were “don’t be evil?” - can’t be not evil, when on the market..

astrange 4y ago
> They’re a stockholder owned corporation now. Beholden to profit above all else. Neither of these two things are true. It's 51% owned by the founders and public companies are not required to only care about profit.
- flandish 4y ago
  Your reply’s logic shows my point. 49% of the company can be purchased via the market. It is a public company. Public companies are legally required to put shareholder interest first - that’s what the whole point of this facet of capitalism is: to provide profit to the “owner” class of society, at the cost of worker exploitation. Get Adam Smith’s boot out of your mouth and recognize that literally anything a corporation does is for the goal of profit. Corporations are not human - and only have a decision making process that puts “return” first. Zero of them are “not evil.”
  - astrange 4y ago
    > Public companies are legally required to put shareholder interest first No, they are not. And they definitely aren't required to care about the interests of 49% of them. > Get Adam Smith’s boot out of your mouth and recognize that literally anything a corporation does is for the goal of profit. Is everything an animal does for the purpose of finding food? (No.)
    - Psychoshy_bc1q 4y ago
      everything an animal does is for the purpose of reproduction lol

bsenftner 4y ago

> I wonder what happened at Google. Finance and management, knowing their bread and butter is as advertising distribution company, undermine, kill, and dilute any projects that are not obviously, undeniably directly tied to improving their advertising distribution revenues. Their big problem is, they never communicated this core identity to the larger corporation, and somehow search is allowed to coast...

nodja 4y ago

They made T5 and released the weights, while T5 "only" goes up to 11B params, it's an efficient model for a wide variety of tasks and the base for a lot of other models/projects. Of note imagen[1] and open-assistant[2] use T5 and FlanT5. [1] https://imagen.research.google/ https://imagen.research.google/ [2] https://github.com/LAION-AI/Open-Assistant https://github.com/LAION-AI/Open-Assistant

dna_polymerase 4y ago

The outside world gets a glimpse into what is happening at Google right now by looking at the people who leave Google. I wouldn't have been surprised if Google had invested in Character, a startup which was formed by Ex-Google LM people. To cite The Information [0]: "At Google, Character’s co-founders Shazeer and Daniel De Freitas were instrumental in developing Google’s LaMDA language model, used for natural language processing research. They decided to launch their own startup after getting frustrated with big company bureaucracies, Shazeer previously told The Information." [0]: https://www.theinformation.com/articles/character-seeks-250-million-in-new-funding-amid-ai-boom?rc=bhi2pj https://www.theinformation.com/articles/character-seeks-250-...

mrtksn 4y ago

People say its to preserve the brand and be woke etc. but IMHO it is simply because doesn't fit Google's business. ChatGPT is very cool but they don't make money from it and GPT-3 had a paid public API for a year now and I read that the money they make from it is very little, insignificant compared to the funding they get from VCs. Currently, there's no clear path for monetisation but has the capacity to take away a lot of Google's use cases. In essence, it's like Napster destroying the Music CD business. Maybe it's also a bit like Kodak: Invents the technology but its current business is too good to be messed around with. This new tech is the future but we are happy in the moment.

LewisVerstappen 4y ago
Well, every single executive at Google has read the Innovator's Dilemma multiple times, so I doubt that's some novel insight that no one at Google has thought of.
- deleted 4y ago
  [deleted]
risyachka 4y ago
Pretty much this. If Google wanted - they could've easily released ChatGPT version that is 100x better than current one considering they invented the technology and they have thousands times more and better data to train it. They just weren't interested.
- 0XYZxyzXYZ0 4y ago
  This doesn't make sense. They did invest in PaLM, which is ~2.5 larger than GPT-3, but no where near 100x better. I do agree that the limiting factor is the data, not scale though. What data has Google that OpenAI (or any other LLM company) can't scrape from the web as well? Using Gmail data without consent would be definitely a move towards a mountain of law suits.
  - CuriouslyC 4y ago

monkeydust 4y ago

Something to be said for '“Move fast and break things.” Zuckerberg

richardw 4y ago

Because ChatGPT isn’t fully baked yet. You can get it to say the stupidest stuff, which is perfect for a research release with somewhat sophisticated users but when it’s “how granny gets information” is when the lawyers get excited. It doesn’t just filter info it finds on the internet, it generates it from whole cloth. This is the Tesla FSD for information - it’ll confidently drive you into a pole with a smile on its face. I love it, but it’s not ready for all Google users yet.

mkl95 4y ago

> Why didn't they ship something like ChatGPT earlier? Lack of product vision? Perfectionism? The simplest answer is that the people with the know-how did not work at Google. Knowing why they didn't is not so simple, though.

CommanderData 4y ago

Probably the lack of freedom to innovate or middle management suppressing free thought stopping cool stuff being made in a devs/eng spare downtime. I've seen so many cool ideas thrown around by devs that could actually go onto be monetized by companies go wasted. To think they'd be worked on for free (spare time) and often better then outsourcing because nothing drives true innovation than passion. Focus on BAU, keep your head down and be lucky you're not getting fired.

joxel 4y ago

Probably only hiring people that cram leetcode instead of actual smart people.

peterisza 4y ago

Lack of balls. Same as Kodak, etc. The fact that they are freaking out is actually a good sign though

habibur 4y ago

They thought it was too powerful to be considered safe for release. And their ad revenue nevertheless had been great even without it. No rush. They back stepped, and then watched someone else capture the market.

daydream 4y ago

Google is too big to seriously innovate. No top down focus or vision just recent Stanford graduates hawking random ideas. Then there’s the promo culture where everyone is busy trying to extract as much comp as possible instead of building. (I’m guessing)

deleted 4y ago
[deleted]

hanniabu 4y ago

> Why didn't they ship something like ChatGPT earlier? I would say they have just in different form. For a few years now when you google something you're given a quote/excerpt at the top of the results that typically answers your question.

onion2k 4y ago

Why didn't they ship something like ChatGPT earlier? In order to launch something you have to need to launch it. Either because you're losing market share, or because you need the revenue, or because your customers expect it. Google rarely has any of those things.