4 ms·

Storm sounds great, but this post probably should have waited until it was actually open-sourced. As it is, it just comes across as naked self-promotion based o

Storm sounds great, but this post probably should have waited until it was actually open-sourced. As it is, it just comes across as naked self-promotion based on a technology that could for all we know be vaporware.

bengl3rt 15y ago
Precisely. I read the whole post looking for a link to the source on github or something, and then the last sentence was just a huge letdown.
ora600 15y ago
Also, the lack of any scalability charts or diagrams of architecture is suspicious. If you can't make it opensource, at least write a serious paper to support the claims. Like Google did for Big-Table. A lot of people think their systems are scalable and fault-tolerant. Most are not. And from the information provided, we can't tell.
- konsl 15y ago
  We've released open source projects (most notably ElephantDB and Cascalog) in the past that are successfully used in production by us as well as other companies. You should check them out if you're interested in a measure of quality, though I understand your concern. We're a startup — we're not going to write an academic paper supporting the claims in the post. Nevertheless, Storm's an exciting project many people are curious to learn more about; that's why we've written something about it now. We have a demo coming soon, and Storm itself will be open sourced soon enough.
  - mceachen 15y ago
    I absolutely understand the issue of being resource constrained. It seems like this is buzz-worthy, (like http://mailchimp.com/omnivore/ http://mailchimp.com/omnivore/), but this pitch is nerd-focused, not potential-customer focused. If you pitch to nerds, you want a github link. If you pitch to potential customers, highlight the benefits that are now possible due to this innovation. At least in our batch, we got drilled this repeatedly: Don't talk features. Talk benefits.
  - justincormack 15y ago
    Dont call it "the Hadoop of" if it is not open source. Hadoop is notable as an open source project not actually a new idea...
nathanmarz 15y ago
(I'm the author of Storm) Your criticism is totally fair. People have been curious about Storm so we wanted to provide a little bit of information about it. We'll have demos soon, and of course it will be open sourced within a few months. If you're curious about our credibility, I think our other open source projects speak to the quality of software we produce: https://github.com/nathanmarz/cascalog https://github.com/nathanmarz/cascalog https://github.com/nathanmarz/elephantdb https://github.com/nathanmarz/elephantdb
- SeoxyS 15y ago
  I think people here are a little too harsh. Storm sounds like an amazing product and I can't wait to play with something like that. Right now, we run a bunch of cron jobs every minute with intense MapReduce queries on mongodb to generate relatively up-to-date analytics. Something like this would be immensely useful. (As well as Mongo's new 2.0 Aggregation pipeline features.) Now, I agree that it's kind of a bummer we can't play with it right now, but the fact that you guys made this are are going to open source it is already awesome in itself.
  - yummyfajitas 15y ago
    As a happy user of ElephantDB, I'd say people are definitely too harsh. Elephantdb is awesome - my company has completely replaced HBase with ElephantDB and MaryJane (a lightweight way of putting data into hadoop that we wrote, https://github.com/stucchio/MaryJane- https://github.com/stucchio/MaryJane- ). That said, I'd love to see some code released, even if it isn't ready for primetime.
    - omakase 15y ago
      Cool, great to see you're making use of EDB. Would love to hear more about how you're using, how the transition was, etc. mm@backtype.com.
    - gojomo 15y ago
      Can you say anything about what made ElephantDB + MaryJane better than HBase for your workload? (Occasional batch loads that then need random reads but not random inserts?)
nphase 15y ago
I'm still waiting on Twitter's rainbird (http://www.slideshare.net/kevinweil/rainbird-realtime-analytics-at-twitter-strata-2011 http://www.slideshare.net/kevinweil/rainbird-realtime-analyt...) to come out!
- matclayton 15y ago
  Me too, I asked them the other day about it, Response - http://twitter.com/#!/kevinweil/status/73263430873792512 http://twitter.com/#!/kevinweil/status/73263430873792512

bengl3rt 15y ago

Precisely. I read the whole post looking for a link to the source on github or something, and then the last sentence was just a huge letdown.

ora600 15y ago

Also, the lack of any scalability charts or diagrams of architecture is suspicious. If you can't make it opensource, at least write a serious paper to support the claims. Like Google did for Big-Table. A lot of people think their systems are scalable and fault-tolerant. Most are not. And from the information provided, we can't tell.

konsl 15y ago
We've released open source projects (most notably ElephantDB and Cascalog) in the past that are successfully used in production by us as well as other companies. You should check them out if you're interested in a measure of quality, though I understand your concern. We're a startup — we're not going to write an academic paper supporting the claims in the post. Nevertheless, Storm's an exciting project many people are curious to learn more about; that's why we've written something about it now. We have a demo coming soon, and Storm itself will be open sourced soon enough.
- mceachen 15y ago
  I absolutely understand the issue of being resource constrained. It seems like this is buzz-worthy, (like http://mailchimp.com/omnivore/ http://mailchimp.com/omnivore/), but this pitch is nerd-focused, not potential-customer focused. If you pitch to nerds, you want a github link. If you pitch to potential customers, highlight the benefits that are now possible due to this innovation. At least in our batch, we got drilled this repeatedly: Don't talk features. Talk benefits.
- justincormack 15y ago
  Dont call it "the Hadoop of" if it is not open source. Hadoop is notable as an open source project not actually a new idea...

nathanmarz 15y ago

(I'm the author of Storm) Your criticism is totally fair. People have been curious about Storm so we wanted to provide a little bit of information about it. We'll have demos soon, and of course it will be open sourced within a few months. If you're curious about our credibility, I think our other open source projects speak to the quality of software we produce: https://github.com/nathanmarz/cascalog https://github.com/nathanmarz/cascalog https://github.com/nathanmarz/elephantdb https://github.com/nathanmarz/elephantdb

SeoxyS 15y ago
I think people here are a little too harsh. Storm sounds like an amazing product and I can't wait to play with something like that. Right now, we run a bunch of cron jobs every minute with intense MapReduce queries on mongodb to generate relatively up-to-date analytics. Something like this would be immensely useful. (As well as Mongo's new 2.0 Aggregation pipeline features.) Now, I agree that it's kind of a bummer we can't play with it right now, but the fact that you guys made this are are going to open source it is already awesome in itself.
- yummyfajitas 15y ago
  As a happy user of ElephantDB, I'd say people are definitely too harsh. Elephantdb is awesome - my company has completely replaced HBase with ElephantDB and MaryJane (a lightweight way of putting data into hadoop that we wrote, https://github.com/stucchio/MaryJane- https://github.com/stucchio/MaryJane- ). That said, I'd love to see some code released, even if it isn't ready for primetime.
  - omakase 15y ago
    Cool, great to see you're making use of EDB. Would love to hear more about how you're using, how the transition was, etc. mm@backtype.com.
  - gojomo 15y ago
    Can you say anything about what made ElephantDB + MaryJane better than HBase for your workload? (Occasional batch loads that then need random reads but not random inserts?)

nphase 15y ago

I'm still waiting on Twitter's rainbird (http://www.slideshare.net/kevinweil/rainbird-realtime-analytics-at-twitter-strata-2011 http://www.slideshare.net/kevinweil/rainbird-realtime-analyt...) to come out!

matclayton 15y ago
Me too, I asked them the other day about it, Response - http://twitter.com/#!/kevinweil/status/73263430873792512 http://twitter.com/#!/kevinweil/status/73263430873792512