3 ms·

Ex-Google, Yahoo, Facebook employees start company to support Hadoop

37 points by qhoxie 18y ago

quasimojo 18y ago
huge geek karma, but i suspect bound to fail this is basically a consulting gig...i doubt they intend to try to compete head-head with the major web cos with their giant colo farms. and who would fund a new massive server farm after famous flame-outs like cuil? what is the market? those companies technical enough to know hadoop exists, know they need it, yet incapable of implementing it while being cash-rich enough to pay someone else to do so. are there even five potential clients? i love hadoop, its awesome...but my guess is that like mysql, apache etc, people will just look to redhat etc to do the integration on private networks, or just use it as a service on something like amazon's platform
liuliu 18y ago
I do not quite understand why people build hadoop and lucene with java. Is a python & c++ implementation of mapreduce a better choice?
- quasimojo 18y ago
  you can write hadoop client tasks in any language you like, afaik
- mseebach 18y ago
  Because Java is an incredibly powerful tool that works across a large selection of platforms with minimal effort. And, Hadoop is only the infrastructure, they have APIs for C, C++ and shell for the applications. http://wiki.apache.org/hadoop/FAQ http://wiki.apache.org/hadoop/FAQ
  - wayne 18y ago
    Python: http://www.michael-noll.com/wiki/Writing_An_Hadoop_MapReduce_Program_In_Python http://www.michael-noll.com/wiki/Writing_An_Hadoop_MapReduce... Ruby: http://blog.rapleaf.com/dev/?p=16 http://blog.rapleaf.com/dev/?p=16 It's pretty hokey, either running through the shell or calling a REST service through a proxy, but it's been done if you're allergic to non-scripting languages.
cellis 18y ago
Is this the result of that cyprus trip?
wayne 18y ago
Really cool. Hadoop is pretty mature and Yahoo's shown it works well on large clusters, but it can be a bear to install, maintain, and learn. I'm sure many companies wouldn't hesitate to pay a few bucks to get another company to help them get up and running faster.
- quasimojo 18y ago
  after doing consulting i can tell you that you need a very big client list to make a go of it full time. you need companies like telcos, insurance companies, food industry...not just core tech companies...that won't last long. i just don't see hadoop being a pressing need in the wider consulting space then again, maybe they just want to get their names out there...as long as they keep overhead low, might be worth it
- deleted 18y ago
  [deleted]
  - aaronblohowiak 18y ago
    "Their welfare depends on the usability and reliability of the product being mediocre or worse. " Some shops have an attitude that is the opposite of "NIH" and "DIY", preferring to source existing experts instead of building in-house talent. There are few things that are easy in the edge cases, and fewer that are easy and worthwhile. With Hadoop, however, there is a lot of money to be made supporting the edge cases. EDIT: I replied to a comment that has since been deleted? Neato!

37 points by qhoxie 18y ago

huge geek karma, but i suspect bound to fail this is basically a consulting gig...i doubt they intend to try to compete head-head with the major web cos with their giant colo farms. and who would fund a new massive server farm after famous flame-outs like cuil? what is the market? those companies technical enough to know hadoop exists, know they need it, yet incapable of implementing it while being cash-rich enough to pay someone else to do so. are there even five potential clients? i love hadoop, its awesome...but my guess is that like mysql, apache etc, people will just look to redhat etc to do the integration on private networks, or just use it as a service on something like amazon's platform

liuliu 18y ago

I do not quite understand why people build hadoop and lucene with java. Is a python & c++ implementation of mapreduce a better choice?

quasimojo 18y ago
you can write hadoop client tasks in any language you like, afaik
mseebach 18y ago
Because Java is an incredibly powerful tool that works across a large selection of platforms with minimal effort. And, Hadoop is only the infrastructure, they have APIs for C, C++ and shell for the applications. http://wiki.apache.org/hadoop/FAQ http://wiki.apache.org/hadoop/FAQ
- wayne 18y ago
  Python: http://www.michael-noll.com/wiki/Writing_An_Hadoop_MapReduce_Program_In_Python http://www.michael-noll.com/wiki/Writing_An_Hadoop_MapReduce... Ruby: http://blog.rapleaf.com/dev/?p=16 http://blog.rapleaf.com/dev/?p=16 It's pretty hokey, either running through the shell or calling a REST service through a proxy, but it's been done if you're allergic to non-scripting languages.

cellis 18y ago

Is this the result of that cyprus trip?

wayne 18y ago

Really cool. Hadoop is pretty mature and Yahoo's shown it works well on large clusters, but it can be a bear to install, maintain, and learn. I'm sure many companies wouldn't hesitate to pay a few bucks to get another company to help them get up and running faster.

quasimojo 18y ago
after doing consulting i can tell you that you need a very big client list to make a go of it full time. you need companies like telcos, insurance companies, food industry...not just core tech companies...that won't last long. i just don't see hadoop being a pressing need in the wider consulting space then again, maybe they just want to get their names out there...as long as they keep overhead low, might be worth it
deleted 18y ago
[deleted]
- aaronblohowiak 18y ago
  "Their welfare depends on the usability and reliability of the product being mediocre or worse. " Some shops have an attitude that is the opposite of "NIH" and "DIY", preferring to source existing experts instead of building in-house talent. There are few things that are easy in the edge cases, and fewer that are easy and worthwhile. With Hadoop, however, there is a lot of money to be made supporting the edge cases. EDIT: I replied to a comment that has since been deleted? Neato!