3 ms·

Accumulo: NSA's Apache-licensed BigTable-based key-value store

79 points by gojomo 13y ago

druiid 13y ago
Well.. uhh... Now we know what this project was specifically used for. But seriously, usage concerns for the project notwithstanding, it looks like a cool K/V storage system. I wonder how it performs compared to similar systems and how it handles failure.
- cfeduke 13y ago
  It operates similarly to HBase but has built in security levels and easy extensibility for custom iterators and aggregators. Failure I cannot speak to since I really only develop against a local cluster that I blow away often.
- elserj 13y ago
  http://www.pdl.cmu.edu/SDI/2013/052013-a.html http://www.pdl.cmu.edu/SDI/2013/052013-a.html is a rather interesting, recently published talk on the subject of scalability of Accumulo
danso 13y ago
> In June 2012 Congress's Senate Armed Services Committee (SASC) released the Draft 2012 Department of Defense (DoD) Authorization Bill, which included references to Apache Accumulo. In the draft bill SASC required DoD to evaluate whether Apache Accumulo could achieve commercial viability before implementing it throughout DoD.[5] Specific criteria were not included in the draft language, but the establishment of commercial entities supporting Apache Accumulo could be considered a success factor This was actually pleasantly surprising to read...the government actually wants to see if its software is useful and competently designed, rather than just buying it because the contractor has influence and no one will know how crappy it is anyway since it'll be classified.
cfeduke 13y ago
When Accumulo was developed in-house at the NSA it was called Cloudbase. When it became part of the Apache project it was renamed Accumulo. I've worked with both HBase and Cloudbase/Accumulo. HBase tooling is more mature than Accumulo's but performance characteristics are about the same. The extension model for Accumulo - for custom iterators and aggregators - is pretty easy to get the hang of right away. (In fairness I never looked into extensibility for HBase.) Both systems play well with Hadoop. Accumulo is a little bit more difficult to use with Oozie. Accumulo's built in monitoring is decent.

79 points by gojomo 13y ago

Well.. uhh... Now we know what this project was specifically used for. But seriously, usage concerns for the project notwithstanding, it looks like a cool K/V storage system. I wonder how it performs compared to similar systems and how it handles failure.

cfeduke 13y ago
It operates similarly to HBase but has built in security levels and easy extensibility for custom iterators and aggregators. Failure I cannot speak to since I really only develop against a local cluster that I blow away often.
elserj 13y ago
http://www.pdl.cmu.edu/SDI/2013/052013-a.html http://www.pdl.cmu.edu/SDI/2013/052013-a.html is a rather interesting, recently published talk on the subject of scalability of Accumulo

danso 13y ago

> In June 2012 Congress's Senate Armed Services Committee (SASC) released the Draft 2012 Department of Defense (DoD) Authorization Bill, which included references to Apache Accumulo. In the draft bill SASC required DoD to evaluate whether Apache Accumulo could achieve commercial viability before implementing it throughout DoD.[5] Specific criteria were not included in the draft language, but the establishment of commercial entities supporting Apache Accumulo could be considered a success factor This was actually pleasantly surprising to read...the government actually wants to see if its software is useful and competently designed, rather than just buying it because the contractor has influence and no one will know how crappy it is anyway since it'll be classified.

cfeduke 13y ago

When Accumulo was developed in-house at the NSA it was called Cloudbase. When it became part of the Apache project it was renamed Accumulo. I've worked with both HBase and Cloudbase/Accumulo. HBase tooling is more mature than Accumulo's but performance characteristics are about the same. The extension model for Accumulo - for custom iterators and aggregators - is pretty easy to get the hang of right away. (In fairness I never looked into extensibility for HBase.) Both systems play well with Hadoop. Accumulo is a little bit more difficult to use with Oozie. Accumulo's built in monitoring is decent.