4 ms·

Crate: Elastic Data Store

22 points by panarky 12y ago

adrusi 12y ago
OT: the domain name crate.io confused me into thinking this was about Cargo, rust's package manager, which is documented on crates.io
- forsaken 12y ago
  This domain used to be Python's new package manager as well. It's still live under /packages/: https://crate.io/packages/requests/ https://crate.io/packages/requests/
  - joshfinnie 12y ago
    They mention that here: https://crate.io/thanks-dstufft/ https://crate.io/thanks-dstufft/ This confused me greatly as well...
- Argorak 12y ago
  crate.io is older then crates.io, though.
watson 12y ago
Have anybody any experience with this DB they would like to share here?
- mping 12y ago
  I took a spin a few weeks ago, it seemed like it would be a great fit to our needs: - we store alot of metrics for custom KPIs - we don't do joins - we don't need transactions - we're write once read heavy, sometimes re-write - crate's sharding, partitioning and replication, as well as "sql" api is great for our use case I even implemented some custom aggregation functions to check out the code, and it was easy. But what I liked the most was the responsiveness of the devs, the project seems to be moving at a good pace and the devs were very helpful over github. I wished the site made clear that it's using ElasticSearch under the hood, though. I don't know a thing about ES so I can't really comment on how the sharding, partitioning and replication are achieved. What I can remember from aphyr's blog (jepsen series) is that it's not recommended as a primary database though.
Kequc 12y ago
Looks great, really. I'm pretty tired of Postgresql all I want is a simple modern data store that can handle arrays and blobs. Relationships and data validation is all handled at the application level for me. I'm very interested to try this out as soon as there is an implementation for active record. Thanks for the heads up.
- sanderjd 12y ago
  > all I want is a simple modern data store that can handle arrays and blobs I'm curious in what way Postgresql doesn't fit that, for you. Is it not "simple"?
  - Kequc 12y ago
    It has a lot of features that aren't needed in modern application development. I just need indexible columns. Arrays were added later so you have to include a module, I mean maybe it's just a feeling but it all feels dated. More importantly it isn't elastic. Constructing a heavy load or large database server cluster is difficult. More complicated than it should be. If there is on offer an elastic solution with simple arrays built in, I'm all over it right now. Blob support in postgresql is seen as something you shouldn't do as a best practice. If it's a data store then what I want is to be able to use it to store data easily without a fuss. Postgresql is the king right now and I use it but I'm very open to looking at alternatives.
    - sanderjd 12y ago
      Your last two paragraphs make a bunch of sense - clustering should be easier and blobs definitely aren't the "happy path". I don't relate much to the first paragraph though. You don't have to include a module to use arrays. I'm mystified by the implication that data integrity, normalization, and ad-hoc querying are useless for "modern application development".
      - Kequc 12y ago
        That stuff is handled at the application level a data store should just be there to store and retrieve data, I think databases used to need to do a lot more than they do now. I thought Postgres required one to include a module in order to gain array support. Looking into it again now it seems I was confusing Arrays for Hashes, with regard to the hStore extension. The application is in charge of what data goes into the database. It is performing sanitisation, validation, returning errors, all of that before persisting data. I don't want to have to deal with errors from the database nor do I want to have to manage data integrity from two separate locations. I want to be able to tell the database to store something and it stores it modern development frameworks can handle data integrity stuff.
otterley 12y ago
Previous discussion on HN, 6 months ago: https://news.ycombinator.com/item?id=7609323 https://news.ycombinator.com/item?id=7609323

22 points by panarky 12y ago

OT: the domain name crate.io confused me into thinking this was about Cargo, rust's package manager, which is documented on crates.io

forsaken 12y ago
This domain used to be Python's new package manager as well. It's still live under /packages/: https://crate.io/packages/requests/ https://crate.io/packages/requests/
- joshfinnie 12y ago
  They mention that here: https://crate.io/thanks-dstufft/ https://crate.io/thanks-dstufft/ This confused me greatly as well...
Argorak 12y ago
crate.io is older then crates.io, though.

watson 12y ago

Have anybody any experience with this DB they would like to share here?

mping 12y ago
I took a spin a few weeks ago, it seemed like it would be a great fit to our needs: - we store alot of metrics for custom KPIs - we don't do joins - we don't need transactions - we're write once read heavy, sometimes re-write - crate's sharding, partitioning and replication, as well as "sql" api is great for our use case I even implemented some custom aggregation functions to check out the code, and it was easy. But what I liked the most was the responsiveness of the devs, the project seems to be moving at a good pace and the devs were very helpful over github. I wished the site made clear that it's using ElasticSearch under the hood, though. I don't know a thing about ES so I can't really comment on how the sharding, partitioning and replication are achieved. What I can remember from aphyr's blog (jepsen series) is that it's not recommended as a primary database though.

Kequc 12y ago

Looks great, really. I'm pretty tired of Postgresql all I want is a simple modern data store that can handle arrays and blobs. Relationships and data validation is all handled at the application level for me. I'm very interested to try this out as soon as there is an implementation for active record. Thanks for the heads up.

sanderjd 12y ago
> all I want is a simple modern data store that can handle arrays and blobs I'm curious in what way Postgresql doesn't fit that, for you. Is it not "simple"?
- Kequc 12y ago
  It has a lot of features that aren't needed in modern application development. I just need indexible columns. Arrays were added later so you have to include a module, I mean maybe it's just a feeling but it all feels dated. More importantly it isn't elastic. Constructing a heavy load or large database server cluster is difficult. More complicated than it should be. If there is on offer an elastic solution with simple arrays built in, I'm all over it right now. Blob support in postgresql is seen as something you shouldn't do as a best practice. If it's a data store then what I want is to be able to use it to store data easily without a fuss. Postgresql is the king right now and I use it but I'm very open to looking at alternatives.
  - sanderjd 12y ago
    Your last two paragraphs make a bunch of sense - clustering should be easier and blobs definitely aren't the "happy path". I don't relate much to the first paragraph though. You don't have to include a module to use arrays. I'm mystified by the implication that data integrity, normalization, and ad-hoc querying are useless for "modern application development".
    - Kequc 12y ago
      That stuff is handled at the application level a data store should just be there to store and retrieve data, I think databases used to need to do a lot more than they do now. I thought Postgres required one to include a module in order to gain array support. Looking into it again now it seems I was confusing Arrays for Hashes, with regard to the hStore extension. The application is in charge of what data goes into the database. It is performing sanitisation, validation, returning errors, all of that before persisting data. I don't want to have to deal with errors from the database nor do I want to have to manage data integrity from two separate locations. I want to be able to tell the database to store something and it stores it modern development frameworks can handle data integrity stuff.

otterley 12y ago

Previous discussion on HN, 6 months ago: https://news.ycombinator.com/item?id=7609323 https://news.ycombinator.com/item?id=7609323