6 ms·

By no means an expert, but isn’t it Apples and Oranges? Cassandra is an eventual consistent database. Postgres is ACID. My 2c: you don’t need Cassandra or ot

By no means an expert, but isn’t it Apples and Oranges? Cassandra is an eventual consistent database. Postgres is ACID. My 2c: you don’t need Cassandra or other BigData databases. If you did, you probably wouldn’t ask the question.

peatmoss 9y ago
> you don’t need Cassandra or other BigData database. If you did, you wouldn’t ask the question probably. ...and by the time you do, Postgres improvements will have rendered your Big Data medium-sized anyway :-)
- fortytw2 9y ago
  you can comfortably fit 8TB of data on a single box running postgres, more dependent on your hardware :)
  - kstrauser 9y ago
    That's about 1/250th of our working set size. :) Of course, that's in a Spark cluster running massively parallel queries and not on a single (however large) node.
    - ibotty 9y ago
      You did not ask that question. I assume you knew before that PostgreSQL is not the tool for your workload.
      - kstrauser 9y ago
        I didn’t ask a question. I love PostgreSQL for small relational stuff (where “small” is up to a few TB) and use it any time it’s appropriate. I wouldn’t classify it as a Big Data store, though. That’s no criticism at all. I wouldn’t use Spark for a transactional DB because it’s not the best tool for the job. Same for PostgreSQL when the dataset grows far beyond what a single instance can reasonably process.
  - riku_iki 9y ago
    Curious, how long it will take to create index on single node for a such large table?
wjossey 9y ago
Having provisioned these sorts of "big data" systems in the past, it's now about how much you have today, it's how much you'll have over a growth period. The advantages of a scale-out system like cassandra, riak (RIP), memsql, big-table, cockroach, etc., are that they can grow with you from 3TB, to 9TB, to 81TB (as an example, if you're on some exponential growth curve with your data). It's not that you can't do it with Postgres and MySQL via sharding, it's that you don't have to if your access patterns match one of the models that one of the above databases fits well within. So, that person above may not need it today, but they may need it sooner rather than later.
- gshulegaard 9y ago
  This is a really insightful comment. It's not that you can't get Postgres to handle any volume...it's how much work is required on your part.
- noncoml 9y ago
  Designing for the future is a guaranteed project failure. If one is starting a new project, hence contemplating what DB to use, starting with an ACID db is a safe bet in most cases(unless of course they are already starting with a huge amount of data). By the time the outgrow the ACID database they will have a better idea of what exactly they need and more importantly they will have the resources to make the switch. BTW: riak is hopefully not dead yet, let's see what bet365 can do with it..
  - Thaxll 9y ago
    No it's not, say that have 2TB of logs daily, good luck designing that with a regular RDMS without sharding / clustering / re-balancing capabilities.
    - noncoml 9y ago
      In which case they would not be designing for the future. Did you miss this: "(unless of course they are already starting with a huge amount of data)"?
threeseed 9y ago
> Cassandra is an eventual consistent database. No. Cassandra is a database with selectable consistency. You can set it to be strongly consistent (ALL or QUORUM) with the tradeoff being worse performance.

peatmoss 9y ago

> you don’t need Cassandra or other BigData database. If you did, you wouldn’t ask the question probably. ...and by the time you do, Postgres improvements will have rendered your Big Data medium-sized anyway :-)

fortytw2 9y ago
you can comfortably fit 8TB of data on a single box running postgres, more dependent on your hardware :)
- kstrauser 9y ago
  That's about 1/250th of our working set size. :) Of course, that's in a Spark cluster running massively parallel queries and not on a single (however large) node.
  - ibotty 9y ago
    You did not ask that question. I assume you knew before that PostgreSQL is not the tool for your workload.
    - kstrauser 9y ago
      I didn’t ask a question. I love PostgreSQL for small relational stuff (where “small” is up to a few TB) and use it any time it’s appropriate. I wouldn’t classify it as a Big Data store, though. That’s no criticism at all. I wouldn’t use Spark for a transactional DB because it’s not the best tool for the job. Same for PostgreSQL when the dataset grows far beyond what a single instance can reasonably process.
- riku_iki 9y ago
  Curious, how long it will take to create index on single node for a such large table?

wjossey 9y ago

Having provisioned these sorts of "big data" systems in the past, it's now about how much you have today, it's how much you'll have over a growth period. The advantages of a scale-out system like cassandra, riak (RIP), memsql, big-table, cockroach, etc., are that they can grow with you from 3TB, to 9TB, to 81TB (as an example, if you're on some exponential growth curve with your data). It's not that you can't do it with Postgres and MySQL via sharding, it's that you don't have to if your access patterns match one of the models that one of the above databases fits well within. So, that person above may not need it today, but they may need it sooner rather than later.

gshulegaard 9y ago
This is a really insightful comment. It's not that you can't get Postgres to handle any volume...it's how much work is required on your part.
noncoml 9y ago
Designing for the future is a guaranteed project failure. If one is starting a new project, hence contemplating what DB to use, starting with an ACID db is a safe bet in most cases(unless of course they are already starting with a huge amount of data). By the time the outgrow the ACID database they will have a better idea of what exactly they need and more importantly they will have the resources to make the switch. BTW: riak is hopefully not dead yet, let's see what bet365 can do with it..
- Thaxll 9y ago
  No it's not, say that have 2TB of logs daily, good luck designing that with a regular RDMS without sharding / clustering / re-balancing capabilities.
  - noncoml 9y ago
    In which case they would not be designing for the future. Did you miss this: "(unless of course they are already starting with a huge amount of data)"?

threeseed 9y ago

> Cassandra is an eventual consistent database. No. Cassandra is a database with selectable consistency. You can set it to be strongly consistent (ALL or QUORUM) with the tradeoff being worse performance.