5 ms·

I know a lot of people chide Google for being behind OpenAI in their commercial offerings. We also dunk on them for the over-protective nature of their fine-tun

I know a lot of people chide Google for being behind OpenAI in their commercial offerings. We also dunk on them for the over-protective nature of their fine-tuning. But Google is scarily capable on the LLM front and we shouldn't count them out. OpenAI might have the advantage of being quick to move, but when the juggernaut gets passed its resting inertia and starts to gain momentum it is going to leave an impression. That became clear to me after watching the recent Jeff Dean video [1] which was posted a few days ago. The depth of institutional knowledge that is going to be unlocked inside Google is actually frightening for me to consider. I hope the continued competition on the open source front, which we can really thank Facebook and Llama for, keeps these behemoths sharing. As OpenAI moves further from its original mission into capitalizing on its technological lead, we have to remember why the original vision they had is important. So thank you, Google, for this. 1. https://www.youtube.com/watch?v=oSCRZkSQ1CE&ab_channel=RiceKenKennedyInstitute https://www.youtube.com/watch?v=oSCRZkSQ1CE&ab_channel=RiceK...

whimsicalism 3y ago
Realistically, if Google has all this talent, they should have gotten the juggernaut moving in 2020. Google has had years to get to this stage, and they've lost a lot of the talent that made their initial big splashes to OAI and competitors. Try finding someone on a sparse MoE paper from Google prior to 2022 who is still working there and not at OAI. With respect, they can hardly even beat Mistral, resorting to rounding down a 7.8b model (w/o embeddings) to 7b.
- freedomben 3y ago
  Organizational dysfunction can squash/squander even the most talented engineers. Especially in a big org in big tech. My bet is that their inability to deliver before is probably a result of non-comittal funders/decision makers, product whiplash, corporate politics, and other non-technical challenges. Google has been the home of the talent for many years. They came on my radar in the late 00s when I used Peter Norvig's textbook in college, and they hired Ray Kurzweil in like 2012 or 2013 IIRC. They were hiring ML PhDs with talent for many years, and they pioneered most of the major innovations. They just got behind on productizing and shipping.
  - whimsicalism 3y ago
    Right, which was fine for them before there was major competition. But starting in 2020, they have basically attrited most of their talented labor force to OAI and competitors who were not similarly dysfunctional.
dguest 3y ago
Maybe someone who knows google better can answer my question here: are they behind simply because LLMs are not really their core business? In other words, it wasn't (and still isn't) obvious that LLMs will help them sell add space. And of course writing that gives me a terrible realization: product placement in LLMs is going to be a very big thing in the near future.
- freedomben 3y ago
  I'm an outsider and am speculating based on what I've heard, so maybe I shouldn't even comment, but to me it seems like it's been entirely corporate/organizational reasons. Non-serious funding, shifting priorities, personnel transfers/fluctuations, internal fragmentation, and more. Lack of talent has never been their problem.
- elwell 3y ago
  LLM bad because cannibalizes search ads. Wait as long as possible. OpenAI opens pandora's box. Now full speed ahead; catch up and overtake.
brigadier132 3y ago
There was a podcast yesterday that explained well why Google is in a tough position. https://youtu.be/-i9AGk3DJ90?t=616 https://youtu.be/-i9AGk3DJ90?t=616 In essence, Google already rules information retrieval. Their margins are insane. Switching to LLM based search cuts into their margins and increases their costs dramatically. Also, the advantage they've built over decades has been cut down. All of this means there is potential for less profit and a shrinking valuation. A shrinking valuation means issues with employee retention and it could lead to long term stagnation.
- corysama 3y ago
  The Innovator's Dilemma over and over again.
- brikym 3y ago
  I’m sure Kodak had the same problem with the digital camera.
  - Eisenstein 3y ago
    They did. They invented and patented the digital camera back in the 70s, refused to improve on it for fear of eating their own market base, and then went bankrupt. * https://spectrum.ieee.org/first-digital-camera-history https://spectrum.ieee.org/first-digital-camera-history
llm_nerd 3y ago
While I generally agree with you, who has ever counted Google out? We've made fun of Google for lagging while they instead spend their engineering time renaming projects and performing algorithmic white-erasure, but we all knew they're a potent force. Google has as much or more computing power than anyone. They're massively capitalized and have a market cap of almost $2T and colossal cashflow, and have the ability to throw enormous resources at the problem until they have a competitor. They have an enormous, benchmark-setting amount of data across their various projects to train on. That we're talking like they're some scrappy upstart is super weird. >As OpenAI moves further from its original mission into capitalizing on its technological lead, we have to remember why the original vision they had is important. I'm way more cynical about the open source models released by the megas, and OpenAI is probably the most honest about their intentions. Meta and Google are releasing these models arguably to kneecap any possible next OpenAI. They want to basically set the market value of anything below state of the art at $0.00, ensuring that there is no breathing room below the $2T cos. These models (Llama, Gemma, etc) are fun toys, but in the end they're completely uncompetitive and will yield zero "wins", so to speak.
- jerpint 3y ago
  > Meta and Google are releasing these models arguably to kneecap any possible next OpenAI. They want to basically set the market value of anything below state of the art at $0.00, ensuring that there is no breathing room below the $2T cos Never thought about it that way, but it makes a lot of sense. It’s also true these models are not up to par with SOTA no matter what the benchmarks say
- loudmax 3y ago
  I certainly would not count out Google's engineering talent. But all the technical expertise in the world won't matter when the leadership is incompetent and dysfunctional. Rolling out a new product takes vision, and it means taking some risks. This is diametrically opposed to how Google operates today. Gemini could be years ahead of ChatGPT (and maybe it is now, if it weren't neutered), but Google's current leadership would have no idea what to do with it. Google has the technical resources to become a major player here, maybe even the dominant player. But it won't happen under current management. I won't count out Google entirely, and there's still time for the company to be saved. It starts with new leadership.
refulgentis 3y ago
There's nothing provided here other than Jeff Dean gave a stock entry-level presentation to students at Rice, therefore "The depth of institutional knowledge that is going to be unlocked inside Google is actually frightening for me to consider." You should see Google's turnover numbers from 4 years ago, much less now. It's been years, it's broken internally, we see the results. Here, we're in awe of 1KLOC of C++ code that runs inference on the CPU. Nobody serious is running inference on CPU unless you're on the extreme cutting edge. (ex. I need to on Android and on the Chrome OS Linux VM, but I still use llama.cpp because it does support GPU everywhere else) I'm not sure what else to say. (n.b. i am a xoogler)
- janwas 3y ago
  We understand that some teams prefer to use CPU even when mobile GPU would be available. This code is also intended to facilitate research & experimentation, which may not fall under your definition of 'serious' :)
- whitten 3y ago
  Are there any online transcripts or recordings of the Rice presentation from Jeff Dean ?
  - refulgentis 3y ago
    Yes, footer of parent: https://youtu.be/oSCRZkSQ1CE?si=Na1d1cK3TApDhkSO https://youtu.be/oSCRZkSQ1CE?si=Na1d1cK3TApDhkSO
- iaseiadit 3y ago
  > You should see Google's turnover numbers from 4 years ago, much less now. High turnover was industry-wide a few years back because pay went through the roof and job hopping was the best way to capture that. I suspect it’s lower now, following mass layoffs and substantially fewer openings.
rldjbpin 3y ago
more than 5 years ago, Google already had an advanced platform for serving llm with dialogflow (ex-api.ai before they got acquired by Google). their feature list vastly outdo what openai is offering today. the core difference is the core model's performance, but the maximum potential is night and day imho. they had a good chance to do well in this front but did not end up making the most yet.

whimsicalism 3y ago

Realistically, if Google has all this talent, they should have gotten the juggernaut moving in 2020. Google has had years to get to this stage, and they've lost a lot of the talent that made their initial big splashes to OAI and competitors. Try finding someone on a sparse MoE paper from Google prior to 2022 who is still working there and not at OAI. With respect, they can hardly even beat Mistral, resorting to rounding down a 7.8b model (w/o embeddings) to 7b.

freedomben 3y ago
Organizational dysfunction can squash/squander even the most talented engineers. Especially in a big org in big tech. My bet is that their inability to deliver before is probably a result of non-comittal funders/decision makers, product whiplash, corporate politics, and other non-technical challenges. Google has been the home of the talent for many years. They came on my radar in the late 00s when I used Peter Norvig's textbook in college, and they hired Ray Kurzweil in like 2012 or 2013 IIRC. They were hiring ML PhDs with talent for many years, and they pioneered most of the major innovations. They just got behind on productizing and shipping.
- whimsicalism 3y ago
  Right, which was fine for them before there was major competition. But starting in 2020, they have basically attrited most of their talented labor force to OAI and competitors who were not similarly dysfunctional.

dguest 3y ago

Maybe someone who knows google better can answer my question here: are they behind simply because LLMs are not really their core business? In other words, it wasn't (and still isn't) obvious that LLMs will help them sell add space. And of course writing that gives me a terrible realization: product placement in LLMs is going to be a very big thing in the near future.

freedomben 3y ago
I'm an outsider and am speculating based on what I've heard, so maybe I shouldn't even comment, but to me it seems like it's been entirely corporate/organizational reasons. Non-serious funding, shifting priorities, personnel transfers/fluctuations, internal fragmentation, and more. Lack of talent has never been their problem.
elwell 3y ago
LLM bad because cannibalizes search ads. Wait as long as possible. OpenAI opens pandora's box. Now full speed ahead; catch up and overtake.

brigadier132 3y ago

There was a podcast yesterday that explained well why Google is in a tough position. https://youtu.be/-i9AGk3DJ90?t=616 https://youtu.be/-i9AGk3DJ90?t=616 In essence, Google already rules information retrieval. Their margins are insane. Switching to LLM based search cuts into their margins and increases their costs dramatically. Also, the advantage they've built over decades has been cut down. All of this means there is potential for less profit and a shrinking valuation. A shrinking valuation means issues with employee retention and it could lead to long term stagnation.

corysama 3y ago
The Innovator's Dilemma over and over again.
brikym 3y ago
I’m sure Kodak had the same problem with the digital camera.
- Eisenstein 3y ago
  They did. They invented and patented the digital camera back in the 70s, refused to improve on it for fear of eating their own market base, and then went bankrupt. * https://spectrum.ieee.org/first-digital-camera-history https://spectrum.ieee.org/first-digital-camera-history

llm_nerd 3y ago

While I generally agree with you, who has ever counted Google out? We've made fun of Google for lagging while they instead spend their engineering time renaming projects and performing algorithmic white-erasure, but we all knew they're a potent force. Google has as much or more computing power than anyone. They're massively capitalized and have a market cap of almost $2T and colossal cashflow, and have the ability to throw enormous resources at the problem until they have a competitor. They have an enormous, benchmark-setting amount of data across their various projects to train on. That we're talking like they're some scrappy upstart is super weird. >As OpenAI moves further from its original mission into capitalizing on its technological lead, we have to remember why the original vision they had is important. I'm way more cynical about the open source models released by the megas, and OpenAI is probably the most honest about their intentions. Meta and Google are releasing these models arguably to kneecap any possible next OpenAI. They want to basically set the market value of anything below state of the art at $0.00, ensuring that there is no breathing room below the $2T cos. These models (Llama, Gemma, etc) are fun toys, but in the end they're completely uncompetitive and will yield zero "wins", so to speak.

jerpint 3y ago
> Meta and Google are releasing these models arguably to kneecap any possible next OpenAI. They want to basically set the market value of anything below state of the art at $0.00, ensuring that there is no breathing room below the $2T cos Never thought about it that way, but it makes a lot of sense. It’s also true these models are not up to par with SOTA no matter what the benchmarks say
loudmax 3y ago
I certainly would not count out Google's engineering talent. But all the technical expertise in the world won't matter when the leadership is incompetent and dysfunctional. Rolling out a new product takes vision, and it means taking some risks. This is diametrically opposed to how Google operates today. Gemini could be years ahead of ChatGPT (and maybe it is now, if it weren't neutered), but Google's current leadership would have no idea what to do with it. Google has the technical resources to become a major player here, maybe even the dominant player. But it won't happen under current management. I won't count out Google entirely, and there's still time for the company to be saved. It starts with new leadership.

refulgentis 3y ago

There's nothing provided here other than Jeff Dean gave a stock entry-level presentation to students at Rice, therefore "The depth of institutional knowledge that is going to be unlocked inside Google is actually frightening for me to consider." You should see Google's turnover numbers from 4 years ago, much less now. It's been years, it's broken internally, we see the results. Here, we're in awe of 1KLOC of C++ code that runs inference on the CPU. Nobody serious is running inference on CPU unless you're on the extreme cutting edge. (ex. I need to on Android and on the Chrome OS Linux VM, but I still use llama.cpp because it does support GPU everywhere else) I'm not sure what else to say. (n.b. i am a xoogler)

janwas 3y ago
We understand that some teams prefer to use CPU even when mobile GPU would be available. This code is also intended to facilitate research & experimentation, which may not fall under your definition of 'serious' :)
whitten 3y ago
Are there any online transcripts or recordings of the Rice presentation from Jeff Dean ?
- refulgentis 3y ago
  Yes, footer of parent: https://youtu.be/oSCRZkSQ1CE?si=Na1d1cK3TApDhkSO https://youtu.be/oSCRZkSQ1CE?si=Na1d1cK3TApDhkSO
iaseiadit 3y ago
> You should see Google's turnover numbers from 4 years ago, much less now. High turnover was industry-wide a few years back because pay went through the roof and job hopping was the best way to capture that. I suspect it’s lower now, following mass layoffs and substantially fewer openings.

rldjbpin 3y ago

more than 5 years ago, Google already had an advanced platform for serving llm with dialogflow (ex-api.ai before they got acquired by Google). their feature list vastly outdo what openai is offering today. the core difference is the core model's performance, but the maximum potential is night and day imho. they had a good chance to do well in this front but did not end up making the most yet.