8 ms·

MapReduce, TensorFlow, Vertex: Google's bet to avoid repeating history in AI

139 points by coloneltcb 3y ago

FrustratedMonky 3y ago
Nice recap.
JohnMakin 3y ago
> The final piece of Google’s strategy today came in the form of a subtle, and very vague, announcement from Nvidia CEO Jensen Huang on stage in a brief appearance of only a few minutes. Huang announced that Google and Nvidia had been collaborating with a new language model development framework, PaxML, built on top of Google’s cutting-edge machine learning framework JAX and its Accelerated Linear Algebra framework (or XLA). My only thought is, I wonder how well the nvidia/google partnership will do against Azure/Intel (I believe Azure invested heavily in FPGA's for their ML use cases).
- ipsum2 3y ago
  Azure doesn't use FPGA for ML, they use Nvidia like everyone else. Azure used FPGAs a few years ago for network switches though.
  - JohnMakin 3y ago
    Ah, I see, looks like that might have been a few years ago, when I last worked in that space.
- brucethemoose2 3y ago
  Does Microsft use Gaudi 2 or Ponte Vecchio much? If Azure offers them, I never hear about projects using them out in the wild. And Microsoft is seemingly shooting for their own AI hardware: https://www.nextplatform.com/2023/07/12/microsofts-chiplet-cloud-to-bring-the-cost-of-llms-way-down/ https://www.nextplatform.com/2023/07/12/microsofts-chiplet-c...
- choppaface 3y ago
  Wow, XLA has historically been absolute crap outside of TPUs and even for TPUs the error messages are incredibly poor. If nvidia actually wants to support XLA now perhaps that means the TPU 5 is the last TPU, and/or future TPUs might be targeted at just inference and efficiency (like TPU 5) and then nvidia owns the training game. After all if you compare Nvidia’s success with H100 sales versus GCloud TPU sales, it would be easy for Sundar to say “if you can’t beat em join em” and just maintain TPU team for inference which is more closely tied to wall street margins.
jeffbee 3y ago
The part about Vertex might be right but the establishing story about mapreduce is totally wrong. By the time Hadoop took off, mapreduce at Google already had one foot in the grave. If you are using Hadoop today you have adopted a technology stack that Google recognized as obsolete 15 years ago. It is difficult to see how Google lost that battle. They effectively disabled an entire industry by suggesting an obsolete stack, while simultaneously moving on to cheaper, better sequels.
- sberens 3y ago
  [Genuinely curious,] what have people moved on to?
  - moandcompany 3y ago
    Google itself moved on to "Flume" and later created "Dataflow" the precursor for Apache Beam. While Dataflow/Beam aren't execution engines for data processing themselves, they abstract away the language of expressing data computation from the engines themselves. At Google for example, a data processing job might be expressed using Beam on top of Flume for processing. Outside of Google, most organizations with large distributed data processing problems moved on to Hadoop2 (YARN/MapReduce2) and later in present day to Apache Spark. When organizations say they are using "Databricks" they are using Apache Spark provided as a service, from a company started by the creators of Apache Spark, which happens to be Databricks. Apache Beam is also used outside of Google on top of other data processing "engines" or runners for these jobs, such as Google's Cloud Dataflow service, Apache Flink, Apache Spark, etc.
    - gravypod 3y ago
      Opinions are my own. Some info on flume: https://research.google/pubs/pub35650/ https://research.google/pubs/pub35650/ To quote from there: "MapReduce and similar systems significantly ease the task of writing data-parallel code. However, many real-world computations require a pipeline of MapReduces, and programming and managing such pipelines can be difficult."
flakiness 3y ago
This is over-indexing the event. XLA (backing Jax) has been supporting the NVIDIA GPU for a long time. (Otherwise TensorFlow were TPU-only, which cannot be true.) I think the announcement is more ceremonial than technical, maybe expecting this kind of superficial reaction.
Upvoter33 3y ago
Google didn't miss on MapReduce; it missed on Cloud. Amazon was light years behind in datacenter technology, but made it all available via AWS, while Google kept everything to themselves. It was a colossal failure. LLMs are shaping up to be the second such failure.
- wrs 3y ago
  In both cases Google regarded the technology as a competitive advantage in the business they were in (web search), so naturally wanted to keep it internal. Maybe almost as important, they were so far ahead on those technologies that making a viable product out of them would have been a huge effort with no benefit to search. Google tech has always been an "island". Even when they did release GCP, the offerings like AppEngine and transparent networking were incomprehensible to customers who just wanted to lift-and-shift their existing datacenter, not adopt Google practices. Amazon, on the other hand, has no qualms about converting their internal expertise into products ("turn every major cost into a source of revenue" [0]) and giving customers what they ask for. [0] https://twitter.com/BrianFeroldi/status/1284795114187919362 https://twitter.com/BrianFeroldi/status/1284795114187919362
- ggm 3y ago
  Who invented Kubernetes? Ok, water it down if you must. Which of the FAANG was in at the birth? Yep. What Google missed on, was taking cloud outside of itself as a visible customer product. AWS leapt into the breach, but the irony is that we want to use AWS to run a technology platform which Google has significant DNA in.
  - vasco 3y ago
    > the irony is that we want to use AWS to run a technology platform which Google has significant DNA in. More of a proof of how much it was google's to lose than anything against AWS.
  - nsonha 3y ago
    > invented Kubernetes So? Does it translate to them escaping the reputation of mostly making money from ad?
rhelz 3y ago
Happens at every big company. The reason we don't hear about it more often isn't that big companies don't know this is a problem. Its just that the only solution they seem to be able to come up with is to enforce keeping unused breakthroughs secret. We can be happy that at Google, at least, these things can seep out and be of some benefit to the rest of us.
ilaksh 3y ago
They supposedly just improved Cody by "up to" 25%. I wonder how it compares to GPT-4 now.
shmerl 3y ago
Google didn't release MapReduce implementation? It took off with Hadoop. If anything, it should have been open source from the start, then Hadoop wouldn't have been needed.
deleted 3y ago
[deleted]
hn_throwaway_99 3y ago
This analysis just feels totally off to me. Google didn't miss anything with MapReduce and TensorFlow. So what that Hadoop and PyTorch came out and became open source favorites? It's not like that had any significant negative business impact to Google. On the flip side, Google's open sourcing of Kubernetes did win the container orchestration "war" (RIP Mesos), and it's not like that brought them any great benefits. I agree with another commenter, where Google missed the boat was cloud. Ironically, AppEngine came out pretty early in 2008, but its original incarnation was much too "Google-specific". Google just didn't have the corporate DNA to understand that, sure, Datastore can allow infinite horizontal scalability, but most people don't want to deal with eventual consistency, and they just want a plain-old SQL DB. It took Google a long time to come around to understand how other businesses use their infrastructure.
- acdha 3y ago
  AppEngine also hit the classic Google problem of missing attention to detail. I used it for a couple of projects in 2008/2009 and it just wasn’t competitive - cold starts were painful, you’d hit all of these gaps in features or random bugs in the tooling, etc. and while Datastore might scale it was consistently much slower than any mainstream SQL database so you really had to need that scaling. That was the first project which trained me not to use Google products. I remember reporting by a ton of stuff and then never hearing back and things never improving until much (multiple years?) later when I’d stopped using it. I was profoundly unsurprised when AWS ate their lunch.
  - hn_throwaway_99 3y ago
    > AppEngine also hit the classic Google problem of missing attention to detail. Amen, amen, amen. This is so frustrating to me because in general I really like GCP as a platform, and I think they're doing great things with stuff like Cloud Run. Case in point, I like Firebase a lot, and I think Firebase Auth (aka Google Identity Platform) is a great product. But it took them about 2.5 years between releasing their initial support for a second factor, which was originally SMS only, and allowing a TOTP (e.g. Google Authenticator/Auth) 2nd factor. And they still don't support "remember this device" functionality, so every time someone logs in they always need to enter the 2nd factor. I literally can't think of any major site (especially Google's own) that doesn't support "remember this device", and yet it just must be a detail that's not "important enough" to get someone at Google a promotion.
quantum_state 3y ago
MapReduce is basically SIMD .. it existed long before google …
Ozzie_osman 3y ago
My first job after college was at Google. I had read the MapReduce paper and was so excited to run one, and luckily they did have a tutorial and I ended up running my first MapReduce within I think a couple weeks of joining, over Google's index of the internet at the time, which was really mind-blowing / exciting. I remember vaguely one task where i tried to join the index of the internet to some other dataset, which required joining across datacenters. The job ran with some warning. Then I got a friendly message from an SRE saying something like "it can cost thousands of dollars to run a join like that, so, no worries, just only do it if its something important." Of course, I wasn't doing anything important. I was just enjoying running MapReduce.
- hiddencost 3y ago
  Hm. What SRE gets out of bed for $X,000 batch compute?
nologic01 3y ago
Add Angular to the list. Seriously though, the AI race has just started. On a horizon that matters for large scale and ongoing adoption (and thus persistent corporate profits and valuations as opposed to manic hypes) nothing has been decided. The hardware and software mix that will deliver this large scale adoption is not clear yet. Past experience (and reason) suggests it should be relatively cheap (commoditized) and easy to use. Achieving X but at 0.1% of the cost will be the game that people will thrive at, betting on gargantuan volumes instead of gargantuan prices. Given "AI" is more or less linear algebra the world is crying for commoditized vectorized compute. Its a solved problem. The world will get what it wants.

139 points by coloneltcb 3y ago

Nice recap.

> The final piece of Google’s strategy today came in the form of a subtle, and very vague, announcement from Nvidia CEO Jensen Huang on stage in a brief appearance of only a few minutes. Huang announced that Google and Nvidia had been collaborating with a new language model development framework, PaxML, built on top of Google’s cutting-edge machine learning framework JAX and its Accelerated Linear Algebra framework (or XLA). My only thought is, I wonder how well the nvidia/google partnership will do against Azure/Intel (I believe Azure invested heavily in FPGA's for their ML use cases).

ipsum2 3y ago
Azure doesn't use FPGA for ML, they use Nvidia like everyone else. Azure used FPGAs a few years ago for network switches though.
- JohnMakin 3y ago
  Ah, I see, looks like that might have been a few years ago, when I last worked in that space.
brucethemoose2 3y ago
Does Microsft use Gaudi 2 or Ponte Vecchio much? If Azure offers them, I never hear about projects using them out in the wild. And Microsoft is seemingly shooting for their own AI hardware: https://www.nextplatform.com/2023/07/12/microsofts-chiplet-cloud-to-bring-the-cost-of-llms-way-down/ https://www.nextplatform.com/2023/07/12/microsofts-chiplet-c...
choppaface 3y ago
Wow, XLA has historically been absolute crap outside of TPUs and even for TPUs the error messages are incredibly poor. If nvidia actually wants to support XLA now perhaps that means the TPU 5 is the last TPU, and/or future TPUs might be targeted at just inference and efficiency (like TPU 5) and then nvidia owns the training game. After all if you compare Nvidia’s success with H100 sales versus GCloud TPU sales, it would be easy for Sundar to say “if you can’t beat em join em” and just maintain TPU team for inference which is more closely tied to wall street margins.

jeffbee 3y ago

The part about Vertex might be right but the establishing story about mapreduce is totally wrong. By the time Hadoop took off, mapreduce at Google already had one foot in the grave. If you are using Hadoop today you have adopted a technology stack that Google recognized as obsolete 15 years ago. It is difficult to see how Google lost that battle. They effectively disabled an entire industry by suggesting an obsolete stack, while simultaneously moving on to cheaper, better sequels.

sberens 3y ago
[Genuinely curious,] what have people moved on to?
- moandcompany 3y ago
  Google itself moved on to "Flume" and later created "Dataflow" the precursor for Apache Beam. While Dataflow/Beam aren't execution engines for data processing themselves, they abstract away the language of expressing data computation from the engines themselves. At Google for example, a data processing job might be expressed using Beam on top of Flume for processing. Outside of Google, most organizations with large distributed data processing problems moved on to Hadoop2 (YARN/MapReduce2) and later in present day to Apache Spark. When organizations say they are using "Databricks" they are using Apache Spark provided as a service, from a company started by the creators of Apache Spark, which happens to be Databricks. Apache Beam is also used outside of Google on top of other data processing "engines" or runners for these jobs, such as Google's Cloud Dataflow service, Apache Flink, Apache Spark, etc.
  - gravypod 3y ago
    Opinions are my own. Some info on flume: https://research.google/pubs/pub35650/ https://research.google/pubs/pub35650/ To quote from there: "MapReduce and similar systems significantly ease the task of writing data-parallel code. However, many real-world computations require a pipeline of MapReduces, and programming and managing such pipelines can be difficult."

flakiness 3y ago

This is over-indexing the event. XLA (backing Jax) has been supporting the NVIDIA GPU for a long time. (Otherwise TensorFlow were TPU-only, which cannot be true.) I think the announcement is more ceremonial than technical, maybe expecting this kind of superficial reaction.

Upvoter33 3y ago

Google didn't miss on MapReduce; it missed on Cloud. Amazon was light years behind in datacenter technology, but made it all available via AWS, while Google kept everything to themselves. It was a colossal failure. LLMs are shaping up to be the second such failure.

wrs 3y ago
In both cases Google regarded the technology as a competitive advantage in the business they were in (web search), so naturally wanted to keep it internal. Maybe almost as important, they were so far ahead on those technologies that making a viable product out of them would have been a huge effort with no benefit to search. Google tech has always been an "island". Even when they did release GCP, the offerings like AppEngine and transparent networking were incomprehensible to customers who just wanted to lift-and-shift their existing datacenter, not adopt Google practices. Amazon, on the other hand, has no qualms about converting their internal expertise into products ("turn every major cost into a source of revenue" [0]) and giving customers what they ask for. [0] https://twitter.com/BrianFeroldi/status/1284795114187919362 https://twitter.com/BrianFeroldi/status/1284795114187919362
ggm 3y ago
Who invented Kubernetes? Ok, water it down if you must. Which of the FAANG was in at the birth? Yep. What Google missed on, was taking cloud outside of itself as a visible customer product. AWS leapt into the breach, but the irony is that we want to use AWS to run a technology platform which Google has significant DNA in.
- vasco 3y ago
  > the irony is that we want to use AWS to run a technology platform which Google has significant DNA in. More of a proof of how much it was google's to lose than anything against AWS.
- nsonha 3y ago
  > invented Kubernetes So? Does it translate to them escaping the reputation of mostly making money from ad?

rhelz 3y ago

Happens at every big company. The reason we don't hear about it more often isn't that big companies don't know this is a problem. Its just that the only solution they seem to be able to come up with is to enforce keeping unused breakthroughs secret. We can be happy that at Google, at least, these things can seep out and be of some benefit to the rest of us.

ilaksh 3y ago

They supposedly just improved Cody by "up to" 25%. I wonder how it compares to GPT-4 now.

shmerl 3y ago

Google didn't release MapReduce implementation? It took off with Hadoop. If anything, it should have been open source from the start, then Hadoop wouldn't have been needed.

deleted 3y ago

[deleted]

hn_throwaway_99 3y ago

This analysis just feels totally off to me. Google didn't miss anything with MapReduce and TensorFlow. So what that Hadoop and PyTorch came out and became open source favorites? It's not like that had any significant negative business impact to Google. On the flip side, Google's open sourcing of Kubernetes did win the container orchestration "war" (RIP Mesos), and it's not like that brought them any great benefits. I agree with another commenter, where Google missed the boat was cloud. Ironically, AppEngine came out pretty early in 2008, but its original incarnation was much too "Google-specific". Google just didn't have the corporate DNA to understand that, sure, Datastore can allow infinite horizontal scalability, but most people don't want to deal with eventual consistency, and they just want a plain-old SQL DB. It took Google a long time to come around to understand how other businesses use their infrastructure.

acdha 3y ago
AppEngine also hit the classic Google problem of missing attention to detail. I used it for a couple of projects in 2008/2009 and it just wasn’t competitive - cold starts were painful, you’d hit all of these gaps in features or random bugs in the tooling, etc. and while Datastore might scale it was consistently much slower than any mainstream SQL database so you really had to need that scaling. That was the first project which trained me not to use Google products. I remember reporting by a ton of stuff and then never hearing back and things never improving until much (multiple years?) later when I’d stopped using it. I was profoundly unsurprised when AWS ate their lunch.
- hn_throwaway_99 3y ago
  > AppEngine also hit the classic Google problem of missing attention to detail. Amen, amen, amen. This is so frustrating to me because in general I really like GCP as a platform, and I think they're doing great things with stuff like Cloud Run. Case in point, I like Firebase a lot, and I think Firebase Auth (aka Google Identity Platform) is a great product. But it took them about 2.5 years between releasing their initial support for a second factor, which was originally SMS only, and allowing a TOTP (e.g. Google Authenticator/Auth) 2nd factor. And they still don't support "remember this device" functionality, so every time someone logs in they always need to enter the 2nd factor. I literally can't think of any major site (especially Google's own) that doesn't support "remember this device", and yet it just must be a detail that's not "important enough" to get someone at Google a promotion.

quantum_state 3y ago

MapReduce is basically SIMD .. it existed long before google …

Ozzie_osman 3y ago

My first job after college was at Google. I had read the MapReduce paper and was so excited to run one, and luckily they did have a tutorial and I ended up running my first MapReduce within I think a couple weeks of joining, over Google's index of the internet at the time, which was really mind-blowing / exciting. I remember vaguely one task where i tried to join the index of the internet to some other dataset, which required joining across datacenters. The job ran with some warning. Then I got a friendly message from an SRE saying something like "it can cost thousands of dollars to run a join like that, so, no worries, just only do it if its something important." Of course, I wasn't doing anything important. I was just enjoying running MapReduce.

hiddencost 3y ago
Hm. What SRE gets out of bed for $X,000 batch compute?

nologic01 3y ago

Add Angular to the list. Seriously though, the AI race has just started. On a horizon that matters for large scale and ongoing adoption (and thus persistent corporate profits and valuations as opposed to manic hypes) nothing has been decided. The hardware and software mix that will deliver this large scale adoption is not clear yet. Past experience (and reason) suggests it should be relatively cheap (commoditized) and easy to use. Achieving X but at 0.1% of the cost will be the game that people will thrive at, betting on gargantuan volumes instead of gargantuan prices. Given "AI" is more or less linear algebra the world is crying for commoditized vectorized compute. Its a solved problem. The world will get what it wants.