5 ms·

Perhaps the biggest "fake-out" in 21st century computing: Google publicly released its MapReduce paper -- directing most of the rest of the industry toward loos

by drewda 5y ago

Perhaps the biggest "fake-out" in 21st century computing: Google publicly released its MapReduce paper -- directing most of the rest of the industry toward loosely coupled, overly complex distributed data processing systems like Hadoop for the following decade -- but internally they just bought a bunch of atomic clocks and built a distributed RDBMS. I know this is a somewhat simplified story, but it does make me chuckle.

skj 5y ago
Google uses map reduce extensively... where it's appropriate. True time helps with things like spanner transactions. It's just a totally different use case.
- drewda 5y ago
  Yes, which is why it's amusing in hindsight that for a decade everyone* outside Google was forcing all* their distributed data tasks into the MapReduce paradigm, without considering alternative approaches like the one used by Spanner. * slight exaggerations, I know
  - mrep 5y ago
    I'm not sure how you think a distributed data processing technology would "fake-out" other companies when building/choosing database technology. They are totally different problem sets. MapReduce does not have a set in stone data source/sink and can use multiple things like bigtable and spanner so they are complementary technologies.
    - rp1 5y ago
      I think the parent commenter might be referring to systems like Hive or HBase built on top of Hadoop and do have a lot of overlap with a large scale database system.
      - drewda 5y ago
        Exactly, thanks. "MapReduce paradigm" wasn't precise, and I can see why that makes everyone want to give a distributed systems 101 lecture in this comment thread :)
      - quin3 5y ago
        HBase isn’t really related to MapReduce though, more akin to BigTable.
  - quin3 5y ago
    It’s not even related. No one was running OLTP workloads as MapReduce jobs at any point.
  - LaserToy 5y ago
    Spanner didn’t exist in 2012.
    - jeffbee 5y ago
      Maybe, but this 2010 presentation mentions it. https://cloud.google.com/files/storage_architecture_and_challenges.pdf https://cloud.google.com/files/storage_architecture_and_chal...
      - dekhn 5y ago
        at that time, only aristocrats could use spanner.
    - teraflop 5y ago
      Yes it did. Google published a paper about it in 2012, and claimed that at that point it had been in development for 5 years and in production for more than 1.
      - LaserToy 5y ago
        Mmm, I worked at google in 2012 and F1 was in active development. I’ve never heard of Spanner internally. Maybe it was in development, but it was not in use. Edit: went and read more. Looks like Spanner existed but didn’t have sql, so it wasn’t what it is today. And looks like I don’t remember things any more.
        ithkuil 5y ago
        at least by 2013 it had some basic SQL (although much more limited subset that one usable today); if you needed more you would be using f1; IIRC I used spanner (without F1) in production around 2013
- dekhn 5y ago
  The tech lead of the Google MapReduce team (which no longer exists) just received their award for turning down mapreduce. IIRC it was officially done 5 years ago. However I believe the code to delete MR was never checked in and I'm not sure if there are still users. MapReduce was used at Google for highly inappropriate things. For example, the machine learning system I worked on, Sibyl https://www.datanami.com/2014/07/17/inside-sibyl-googles-massively-parallel-machine-learning-platform/ https://www.datanami.com/2014/07/17/inside-sibyl-googles-mas... was implemented using mapreduce but there was no real technical justification for that- it's just that there was no other system that could scale to the volumes required or handle the constant failures endemic to GOogle's internal systems. It ended up requiring all sorts of heroic work to make MR scale, for example map-side combiners (which "reduced" items with common keys in the map output before it gets flushed to the shuffle files). All of this got replaced with TensorFlow and only the good bits of Sibyl were extracted to TFX.
  - deleted 5y ago
    [deleted]
  - sokoloff 5y ago
    It seems to me “there is no other technical system in the company capable to perform the task” is a valid technical justification.
    - jpgvm 5y ago
      i.e this is a crappy tool but it's best tool we have come up with for this problem thus far. There are lots of such tools which are used begrudgingly by people that have an intuition for the fact it can be done better but not the concrete idea and/or time to implement it.
      - dekhn 5y ago
        it wasn't a crappy tool (mapreduce was amazing) but it definitely was an impedence mismatch for this particular job. Later, we tried to get SIbyl to move to the underlying compute engine that Flume was built on top of it but it turned out to be more profitable to just let it die slowly.
d_watt 5y ago
Those aren't really equivalent, are they? Hadoop is for analytics, spanner is transactional. In terms of popular nosql vs google sql products, it's more Hadoop : BigQuery :: Mongo? : Spanner You're pretty explicitly not supposed to run OLAP queries on spanner.
deleted 5y ago
[deleted]
deleted 5y ago
[deleted]
VirusNewbie 5y ago
It's rare to see a comment on HN that misunderstands basic distributed systems concepts. MapReduce the paper has nothing to do with a database. You're likely conflating the fact that to achieve fault tolerant distributed computation, hadoop and hadoop like systems use a database like filesystem. However, no one is looking at Map Reduce type jobs as a replacement for a database and vice versa. That's like saying "wow linkedin made kafka why do we need a webserver too". Those two technologies are only related in the loosest sense.
- drewda 5y ago
  > You're likely conflating the fact that to achieve fault tolerant distributed computation, hadoop and hadoop like systems use a database like filesystem. Yes, I specifically mentioned non-Google users adopting Hadoop, since it encompassed both a MapReduce implementation and supporting infrastructure. Once on the bandwagon inspired by the MapReduce paper, many orgs didn't just use MapReduce itself for parallelized batch analytic purposes, but also HBase and Hive and other stuff with actual longer term state atop HDFS, YARN, etc. > However, no one is looking at Map Reduce type jobs as a replacement for a database and vice versa. The marketing and sales teams of HortonWorks, Cloudera, etc certainly sold Hadoop platforms, related Apache projects, and "MapReduce" (as a broad brand name for all this, not the specific technical concept) as replacements for databases, broadly speaking. It's that culture that was a bit shocked when Spanner was unveiled.
  - VirusNewbie 5y ago
    The reason spanner is a big deal is because it provides relational semantics including cross table transactions. That is why spanner is interesting. No one, not even the most ambitious and clueless market folks at Cloudera, has ever sold Hadoop and MR type technologies as a replacement for RDBMs. Spanner can be used as a replacement. These two technologies don't solve the same problem. You still need MR to do analytics at scale on spanner!
dekhn 5y ago
amazing. you literally got everything wrong. MapReduce predates TrueTime by a decade or more. MR was critical to scaling internet systems at the time it was released. However, Flume + Spanner was a much nicer system to work with than MR + GFS, I'll give you that.
deleted 5y ago
[deleted]

by drewda 5y ago

skj 5y ago

Google uses map reduce extensively... where it's appropriate. True time helps with things like spanner transactions. It's just a totally different use case.

drewda 5y ago
Yes, which is why it's amusing in hindsight that for a decade everyone* outside Google was forcing all* their distributed data tasks into the MapReduce paradigm, without considering alternative approaches like the one used by Spanner. * slight exaggerations, I know
- mrep 5y ago
  I'm not sure how you think a distributed data processing technology would "fake-out" other companies when building/choosing database technology. They are totally different problem sets. MapReduce does not have a set in stone data source/sink and can use multiple things like bigtable and spanner so they are complementary technologies.
  - rp1 5y ago
    I think the parent commenter might be referring to systems like Hive or HBase built on top of Hadoop and do have a lot of overlap with a large scale database system.
    - drewda 5y ago
      Exactly, thanks. "MapReduce paradigm" wasn't precise, and I can see why that makes everyone want to give a distributed systems 101 lecture in this comment thread :)
    - quin3 5y ago
      HBase isn’t really related to MapReduce though, more akin to BigTable.
- quin3 5y ago
  It’s not even related. No one was running OLTP workloads as MapReduce jobs at any point.
- LaserToy 5y ago
  Spanner didn’t exist in 2012.
  - jeffbee 5y ago
    Maybe, but this 2010 presentation mentions it. https://cloud.google.com/files/storage_architecture_and_challenges.pdf https://cloud.google.com/files/storage_architecture_and_chal...
    - dekhn 5y ago
      at that time, only aristocrats could use spanner.
  - teraflop 5y ago
    Yes it did. Google published a paper about it in 2012, and claimed that at that point it had been in development for 5 years and in production for more than 1.
    - LaserToy 5y ago
      Mmm, I worked at google in 2012 and F1 was in active development. I’ve never heard of Spanner internally. Maybe it was in development, but it was not in use. Edit: went and read more. Looks like Spanner existed but didn’t have sql, so it wasn’t what it is today. And looks like I don’t remember things any more.
      - ithkuil 5y ago
        at least by 2013 it had some basic SQL (although much more limited subset that one usable today); if you needed more you would be using f1; IIRC I used spanner (without F1) in production around 2013
dekhn 5y ago
The tech lead of the Google MapReduce team (which no longer exists) just received their award for turning down mapreduce. IIRC it was officially done 5 years ago. However I believe the code to delete MR was never checked in and I'm not sure if there are still users. MapReduce was used at Google for highly inappropriate things. For example, the machine learning system I worked on, Sibyl https://www.datanami.com/2014/07/17/inside-sibyl-googles-massively-parallel-machine-learning-platform/ https://www.datanami.com/2014/07/17/inside-sibyl-googles-mas... was implemented using mapreduce but there was no real technical justification for that- it's just that there was no other system that could scale to the volumes required or handle the constant failures endemic to GOogle's internal systems. It ended up requiring all sorts of heroic work to make MR scale, for example map-side combiners (which "reduced" items with common keys in the map output before it gets flushed to the shuffle files). All of this got replaced with TensorFlow and only the good bits of Sibyl were extracted to TFX.
- deleted 5y ago
  [deleted]
- sokoloff 5y ago
  It seems to me “there is no other technical system in the company capable to perform the task” is a valid technical justification.
  - jpgvm 5y ago
    i.e this is a crappy tool but it's best tool we have come up with for this problem thus far. There are lots of such tools which are used begrudgingly by people that have an intuition for the fact it can be done better but not the concrete idea and/or time to implement it.
    - dekhn 5y ago
      it wasn't a crappy tool (mapreduce was amazing) but it definitely was an impedence mismatch for this particular job. Later, we tried to get SIbyl to move to the underlying compute engine that Flume was built on top of it but it turned out to be more profitable to just let it die slowly.

d_watt 5y ago

Those aren't really equivalent, are they? Hadoop is for analytics, spanner is transactional. In terms of popular nosql vs google sql products, it's more Hadoop : BigQuery :: Mongo? : Spanner You're pretty explicitly not supposed to run OLAP queries on spanner.

deleted 5y ago

[deleted]

deleted 5y ago

[deleted]

VirusNewbie 5y ago

It's rare to see a comment on HN that misunderstands basic distributed systems concepts. MapReduce the paper has nothing to do with a database. You're likely conflating the fact that to achieve fault tolerant distributed computation, hadoop and hadoop like systems use a database like filesystem. However, no one is looking at Map Reduce type jobs as a replacement for a database and vice versa. That's like saying "wow linkedin made kafka why do we need a webserver too". Those two technologies are only related in the loosest sense.

drewda 5y ago
> You're likely conflating the fact that to achieve fault tolerant distributed computation, hadoop and hadoop like systems use a database like filesystem. Yes, I specifically mentioned non-Google users adopting Hadoop, since it encompassed both a MapReduce implementation and supporting infrastructure. Once on the bandwagon inspired by the MapReduce paper, many orgs didn't just use MapReduce itself for parallelized batch analytic purposes, but also HBase and Hive and other stuff with actual longer term state atop HDFS, YARN, etc. > However, no one is looking at Map Reduce type jobs as a replacement for a database and vice versa. The marketing and sales teams of HortonWorks, Cloudera, etc certainly sold Hadoop platforms, related Apache projects, and "MapReduce" (as a broad brand name for all this, not the specific technical concept) as replacements for databases, broadly speaking. It's that culture that was a bit shocked when Spanner was unveiled.
- VirusNewbie 5y ago
  The reason spanner is a big deal is because it provides relational semantics including cross table transactions. That is why spanner is interesting. No one, not even the most ambitious and clueless market folks at Cloudera, has ever sold Hadoop and MR type technologies as a replacement for RDBMs. Spanner can be used as a replacement. These two technologies don't solve the same problem. You still need MR to do analytics at scale on spanner!

dekhn 5y ago

amazing. you literally got everything wrong. MapReduce predates TrueTime by a decade or more. MR was critical to scaling internet systems at the time it was released. However, Flume + Spanner was a much nicer system to work with than MR + GFS, I'll give you that.

deleted 5y ago

[deleted]