6 ms·

> The next obvious reason for Google to invest in pure research is for the breakthrough discoveries it has yielded and can continue to yield. As a rudimentary b

> The next obvious reason for Google to invest in pure research is for the breakthrough discoveries it has yielded and can continue to yield. As a rudimentary brag sheet, Brain gave Google TensorFlow, TPUs, significantly improved Translate, JAX, and Transformers. Except that these advances have made other companies an existential threat for Google. 2 years ago it was hard to imagine what could topple Google. Now a lot of people can see a clear path: large language models. From a business perspective it's astounding what a massive failure Google Brain has been. Basically nothing has spun out of it to benefit Google. And yet at the same time, so much has leaked out, and so many people have left with that knowledge Google paid for, that Google might go the way of Yahoo in 10 years. This is the simpler explanation of the Brain-Deep Mind merger: both Brain and Deep Mind have fundamentally failed as businesses.

deleted 4y ago
[deleted]
cma 4y ago
> And yet at the same time, so much has leaked out, and so many people have left with that knowledge Google paid for, that Google might go the way of Yahoo in 10 years. Google couldn't have hired the talent they did without allowing them to publish.
dekhn 4y ago
Google never talked much about it externally, but Google Research (the predecessor to Brain) had a single project which almost entirely funded the entire division- a growth-oriented machine learnign system called Sibyl. What was sibyl used for? Growing youtube and google play and other products by making the more addictive. Sibyl wasn't a very good system (I've never seen a product that had more technical debt) but it did basically "pay for" all of the research for a while.
- temac 4y ago
  Seems to be quite evil though.
  - onlyrealcuzzo 4y ago
    Every business is essentially in the business of making people addicted to their products in some way. You're an oil company - you want people to drive as much as possible. You're an airline company - you want people to fly all over the world as much as possible. You're a fashion company - you want people to buy new clothes constantly. You're a beverage company - you want people drinking your drink all the time instead of water. You're an Internet advertising company - you want people's eyeballs on your products as much as possible (to blast them with ads). It's just business.
    - wetmore 4y ago
      That doesn't mean it's not evil though?
  - packetslave 4y ago
    It's evil if you phrase it (as OP did) as "getting people addicted to YouTube". Less so if you phrase it "show people recommendations that they're likely to actually click on, based on what they've watched previously", which is what Sybil really was.
- throwaway29303 4y ago
  In case anyone is wondering what Sibyl is all about, here's a video https://www.youtube.com/watch?v=3SaZ5UAQrQM https://www.youtube.com/watch?v=3SaZ5UAQrQM
  - dekhn 4y ago
    Yep- that goes into a fair amount of detail. Sibyl was retired but many of the ideas lived on in TFX. I worked on it a bit and it was definitely the weirdest, most technically-debt-ridden systems I've ever seen, but it was highly effective at getting people additicted to watching Youtube and downloading games that showed ads.
abtinf 4y ago
It’s like Xerox PARC all over again.
jeffbee 4y ago
> Basically nothing has spun out of it to benefit Google Quite a ridiculous statement. Google has inserted ML all over their products. Maybe you just don't notice, to their credit. But for example the fact that YouTube can automatically generate subtitles for any written language from any spoken language is a direct outcome of Google ML research. There are lots of machine-inferred search ranking signals. Google Sheets will automatically fill in your formulas, that's in-house ML research, too.
- light_hue_1 4y ago
  > Quite a ridiculous statement. Google has inserted ML all over their products. Maybe you just don't notice, to their credit. But for example the fact that YouTube can automatically generate subtitles for any written language from any spoken language is a direct outcome of Google ML research. There are lots of machine-inferred search ranking signals. Google Sheets will automatically fill in your formulas, that's in-house ML research, too. I noticed all the toy demos. None of these have provided Google with any competitive advantage over anyone. For the investment, Google Brain has been a massive failure. It provided Google with essentially zero value. And helped create competitors.
  - sudosysgen 4y ago
    Automatic subtitles and translation is actually a huge feature which is very useful to the many people that don't speak English. It definitely did provide Google with a lot of value.
    - light_hue_1 4y ago
      > Automatic subtitles and translation is actually a huge feature which is very useful to the many people that don't speak English. It definitely did provide Google with a lot of value. It lost Google immense value. Before Google Brain the only speech recognizers that halfway worked were at Google, IBM and Amazon. And Amazon had to buy a company to get access. After Google Brain, anyone can run a speech recognizer. One that is state of the art. There are many models out there that just work well enough. Google went from having an ok speech recognizer that sort of worked in a few languages and gave YouTube an advantage that no company aside from IBM and Amazon could touch. Neither of which compete with Google much. No startup could have anything like Google's captioning. It was untouchable. Like, speech recognition researchers actively avoided this competition, that's how inferior everyone was. To now, post Google Brain, any startup can have captions that are as good as YouTube's captions. You can run countless models on your laptop today. This is a huge competitive loss for Google. They got a minor feature for YouTube and lost one of the key ML advantages they had.
asdfman123 4y ago
Google has very good LLMs. It just let OpenAI beat them to the punch by releasing them earlier. As an established business, Google felt it had a lot to lose by releasing "unsafe" AI into the world. But OpenAI doesn't have a money printing machine, and it's sink-or-swim for them.
- snapcaster 4y ago
  I keep hearing this, but Bard sucks so badly when i've tried to use it like GPT-4 or compare results its like night and day. What makes you so confident they have "secret" LLMs that are superior?
  - asdfman123 4y ago
    Have you tried PaLM? I work for Google and have been playing with it. It’s pretty good. The decision to release Bard, an LLM that was clearly not as good as ChatGPT, struck me as reactive and is why people think Google is behind. I’d think so too if I had just demoed Bard.
    - snapcaster 4y ago
      No, but would love to try it. I'm using these models 20-30 times a day throughout the average work day for random tasks so have a pretty good sense of performance levels. Didn't think it was available to public yet but just saw it's apparently on google cloud now, i'll have to try it out. How do you compare Palm with GPT4 if you've had a chance to try both?
      - asdfman123 4y ago
        Seems pretty similar. In general Google LLMs seem better suited for just conversation and ChatGPT is built to honor “write me X in the style of Y” prompts. The latter is more interesting to play around with, granted, and I think it’s an area where Google can catch up, but it doesn’t seem like a huge technical hurdle.
  - jeffbee 4y ago
sushid 4y ago
It truly feels like Google Brain could be considered Google's equivalent of Bell Labs in the 70s.

deleted 4y ago

[deleted]

cma 4y ago

> And yet at the same time, so much has leaked out, and so many people have left with that knowledge Google paid for, that Google might go the way of Yahoo in 10 years. Google couldn't have hired the talent they did without allowing them to publish.

dekhn 4y ago

Google never talked much about it externally, but Google Research (the predecessor to Brain) had a single project which almost entirely funded the entire division- a growth-oriented machine learnign system called Sibyl. What was sibyl used for? Growing youtube and google play and other products by making the more addictive. Sibyl wasn't a very good system (I've never seen a product that had more technical debt) but it did basically "pay for" all of the research for a while.

temac 4y ago
Seems to be quite evil though.
- onlyrealcuzzo 4y ago
  Every business is essentially in the business of making people addicted to their products in some way. You're an oil company - you want people to drive as much as possible. You're an airline company - you want people to fly all over the world as much as possible. You're a fashion company - you want people to buy new clothes constantly. You're a beverage company - you want people drinking your drink all the time instead of water. You're an Internet advertising company - you want people's eyeballs on your products as much as possible (to blast them with ads). It's just business.
  - wetmore 4y ago
    That doesn't mean it's not evil though?
- packetslave 4y ago
  It's evil if you phrase it (as OP did) as "getting people addicted to YouTube". Less so if you phrase it "show people recommendations that they're likely to actually click on, based on what they've watched previously", which is what Sybil really was.
throwaway29303 4y ago
In case anyone is wondering what Sibyl is all about, here's a video https://www.youtube.com/watch?v=3SaZ5UAQrQM https://www.youtube.com/watch?v=3SaZ5UAQrQM
- dekhn 4y ago
  Yep- that goes into a fair amount of detail. Sibyl was retired but many of the ideas lived on in TFX. I worked on it a bit and it was definitely the weirdest, most technically-debt-ridden systems I've ever seen, but it was highly effective at getting people additicted to watching Youtube and downloading games that showed ads.

abtinf 4y ago

It’s like Xerox PARC all over again.

jeffbee 4y ago

> Basically nothing has spun out of it to benefit Google Quite a ridiculous statement. Google has inserted ML all over their products. Maybe you just don't notice, to their credit. But for example the fact that YouTube can automatically generate subtitles for any written language from any spoken language is a direct outcome of Google ML research. There are lots of machine-inferred search ranking signals. Google Sheets will automatically fill in your formulas, that's in-house ML research, too.

light_hue_1 4y ago
> Quite a ridiculous statement. Google has inserted ML all over their products. Maybe you just don't notice, to their credit. But for example the fact that YouTube can automatically generate subtitles for any written language from any spoken language is a direct outcome of Google ML research. There are lots of machine-inferred search ranking signals. Google Sheets will automatically fill in your formulas, that's in-house ML research, too. I noticed all the toy demos. None of these have provided Google with any competitive advantage over anyone. For the investment, Google Brain has been a massive failure. It provided Google with essentially zero value. And helped create competitors.
- sudosysgen 4y ago
  Automatic subtitles and translation is actually a huge feature which is very useful to the many people that don't speak English. It definitely did provide Google with a lot of value.
  - light_hue_1 4y ago
    > Automatic subtitles and translation is actually a huge feature which is very useful to the many people that don't speak English. It definitely did provide Google with a lot of value. It lost Google immense value. Before Google Brain the only speech recognizers that halfway worked were at Google, IBM and Amazon. And Amazon had to buy a company to get access. After Google Brain, anyone can run a speech recognizer. One that is state of the art. There are many models out there that just work well enough. Google went from having an ok speech recognizer that sort of worked in a few languages and gave YouTube an advantage that no company aside from IBM and Amazon could touch. Neither of which compete with Google much. No startup could have anything like Google's captioning. It was untouchable. Like, speech recognition researchers actively avoided this competition, that's how inferior everyone was. To now, post Google Brain, any startup can have captions that are as good as YouTube's captions. You can run countless models on your laptop today. This is a huge competitive loss for Google. They got a minor feature for YouTube and lost one of the key ML advantages they had.

asdfman123 4y ago

Google has very good LLMs. It just let OpenAI beat them to the punch by releasing them earlier. As an established business, Google felt it had a lot to lose by releasing "unsafe" AI into the world. But OpenAI doesn't have a money printing machine, and it's sink-or-swim for them.

snapcaster 4y ago
I keep hearing this, but Bard sucks so badly when i've tried to use it like GPT-4 or compare results its like night and day. What makes you so confident they have "secret" LLMs that are superior?
- asdfman123 4y ago
  Have you tried PaLM? I work for Google and have been playing with it. It’s pretty good. The decision to release Bard, an LLM that was clearly not as good as ChatGPT, struck me as reactive and is why people think Google is behind. I’d think so too if I had just demoed Bard.
  - snapcaster 4y ago
    No, but would love to try it. I'm using these models 20-30 times a day throughout the average work day for random tasks so have a pretty good sense of performance levels. Didn't think it was available to public yet but just saw it's apparently on google cloud now, i'll have to try it out. How do you compare Palm with GPT4 if you've had a chance to try both?
    - asdfman123 4y ago
      Seems pretty similar. In general Google LLMs seem better suited for just conversation and ChatGPT is built to honor “write me X in the style of Y” prompts. The latter is more interesting to play around with, granted, and I think it’s an area where Google can catch up, but it doesn’t seem like a huge technical hurdle.
- jeffbee 4y ago

sushid 4y ago

It truly feels like Google Brain could be considered Google's equivalent of Bell Labs in the 70s.