5 ms·

To me, and a few people that I know, the most interesting part of MapR was their NFS server over multiple nodes. It solved a real problem that real customers (i

To me, and a few people that I know, the most interesting part of MapR was their NFS server over multiple nodes. It solved a real problem that real customers (i.e. enterprises with money for whom tech made stuff work and not was work in itself) were willing to pay a lot of money for ( see Isilon, EMC, NetApp, etc ) but that was a non-sexy part of the MapR business -- Big Data part was much sexier. Over the last 5 years the real money making enterprises solved their "it should look like a big file system" problem so "the rest of our stuff works" issue stopped being an issue ( in a process Isilon got bought by the EMC, EMC got bought by Dell ) either by buying those specialized solutions, moving to object store or building home grown systems that worked with their specific applications and big data people went with newer, more shiny big data solutions, leaving MapR with no market.

heyyyouu 7y ago
Which of the shinier, newer solutions do you think people have found more attractive in particular? Genuinely curious, thanks!
- notyourday 7y ago
  I'm not someone very much in the "big data" [0] but from what I see streaming analytics especially Spark-based solutions are eating the world that used to be dominated by Hadoop. I also think cloud based data lakes and tooling around them seriously decreased the appeal of Hadoop. [0] If I can fit your big data into my memory, it is not "big data".
  - brokensegue 7y ago
    i mean spark is often deployed on top of yarn where it's still in the "hadoop" sphere.
    - notyourday 7y ago
      But I'm not loading up on my SuperMicro, Dell or HP servers. So there's no need to buy a decent Hadoop implementation from MapR.
  - threeseed 7y ago
    Spark is very much part of Hadoop. It uses Hadoop libraries throughout it including for the core part of reading/writing data And that adage of big data can't fit in memory is nonsense these days. We run clusters with hundreds of terabytes of RAM which is very much big data. It's pretty easy and affordable with the cloud.
    - notyourday 7y ago
      When was the last time you bought a MapR license to use it in a cloud? When MapR was selling them the M5 and M7 were deployed in a data center on servers.
notacoward 7y ago
The increasing adoption of object stores and other not-quite-filesystems, even when a real filesystem would be the more appropriate choice, is definitely part of this. For good or ill, people would rather work on or with one of the simplified alternatives. I'm in that camp myself now, working on a system with lame HDFS-like semantics after two decades of proving the "POSIX can't scale" folks wrong. It's just not a battle worth fighting any more, at either the technical or business level.
- notyourday 7y ago
  That's why I think MapR missed the mark. They had a software only solution one could deploy on a top of generic hardware in a data center to solve hundreds of TB to small number of PB storage problem that was real for enterprises and since those nodes would also be able to do compute jobs that was the way to get enterprises to adapt that technology. Going to object store was possible but it required dev time and in the game of build or buy the buy of the non-core tech is nearly always preferable. And, in case of MapR, it could be done immediately not after a massive refactoring to move to object store. But the MapR sales people were impossible to deal with. They wanted to talk about Big Data. And all the wonderful things that we could do with it. Me: I have lots of files. Think billions. Lets talk about what and how your stuff works to solve this problem because I'm hearing some people successfully used it on a million files scale. Them: That's great. Let me tell you about big data that you can do using it. Me: That's ok. I really want to solve my files problem. Them: Big data is the future! You can change your application to do big data. Needless to say it went nowhere fast.
  - trhway 7y ago
    Sounds familiar and not surprising. s/Big Data/Java/g. Their Chief Marketing Officer is the guy who was the CMO back then at those dusk years of Sun and presided over everything getting named "Java", even the things completely unrelated to Java ... even Sun's stock ticker got renamed into JAVA :)
  - noir-york 7y ago
    Interesting post. Out of curiosity: aren't there others who have developed cluster storage exposed as NFS / FUSE?

heyyyouu 7y ago

Which of the shinier, newer solutions do you think people have found more attractive in particular? Genuinely curious, thanks!

notyourday 7y ago
I'm not someone very much in the "big data" [0] but from what I see streaming analytics especially Spark-based solutions are eating the world that used to be dominated by Hadoop. I also think cloud based data lakes and tooling around them seriously decreased the appeal of Hadoop. [0] If I can fit your big data into my memory, it is not "big data".
- brokensegue 7y ago
  i mean spark is often deployed on top of yarn where it's still in the "hadoop" sphere.
  - notyourday 7y ago
    But I'm not loading up on my SuperMicro, Dell or HP servers. So there's no need to buy a decent Hadoop implementation from MapR.
- threeseed 7y ago
  Spark is very much part of Hadoop. It uses Hadoop libraries throughout it including for the core part of reading/writing data And that adage of big data can't fit in memory is nonsense these days. We run clusters with hundreds of terabytes of RAM which is very much big data. It's pretty easy and affordable with the cloud.
  - notyourday 7y ago
    When was the last time you bought a MapR license to use it in a cloud? When MapR was selling them the M5 and M7 were deployed in a data center on servers.

notacoward 7y ago

The increasing adoption of object stores and other not-quite-filesystems, even when a real filesystem would be the more appropriate choice, is definitely part of this. For good or ill, people would rather work on or with one of the simplified alternatives. I'm in that camp myself now, working on a system with lame HDFS-like semantics after two decades of proving the "POSIX can't scale" folks wrong. It's just not a battle worth fighting any more, at either the technical or business level.

notyourday 7y ago
That's why I think MapR missed the mark. They had a software only solution one could deploy on a top of generic hardware in a data center to solve hundreds of TB to small number of PB storage problem that was real for enterprises and since those nodes would also be able to do compute jobs that was the way to get enterprises to adapt that technology. Going to object store was possible but it required dev time and in the game of build or buy the buy of the non-core tech is nearly always preferable. And, in case of MapR, it could be done immediately not after a massive refactoring to move to object store. But the MapR sales people were impossible to deal with. They wanted to talk about Big Data. And all the wonderful things that we could do with it. Me: I have lots of files. Think billions. Lets talk about what and how your stuff works to solve this problem because I'm hearing some people successfully used it on a million files scale. Them: That's great. Let me tell you about big data that you can do using it. Me: That's ok. I really want to solve my files problem. Them: Big data is the future! You can change your application to do big data. Needless to say it went nowhere fast.
- trhway 7y ago
  Sounds familiar and not surprising. s/Big Data/Java/g. Their Chief Marketing Officer is the guy who was the CMO back then at those dusk years of Sun and presided over everything getting named "Java", even the things completely unrelated to Java ... even Sun's stock ticker got renamed into JAVA :)
- noir-york 7y ago
  Interesting post. Out of curiosity: aren't there others who have developed cluster storage exposed as NFS / FUSE?