5 ms·

> We plan to do more research on scalability, since column-based databases can’t grow as big as their NoSQL counterparts. This one sentence makes me question t

> We plan to do more research on scalability, since column-based databases can’t grow as big as their NoSQL counterparts. This one sentence makes me question this firm's expertise. "Column-based database" has a meaning, and Postgres definitely isn't one of them.

duxup 4y ago
I’m confused by that line and it made me wonder about their experience or PoV as well. But as someone who doesn’t word smith good, could just be that.
deleted 4y ago
[deleted]
jackconsidine 4y ago
Author here- poor choice of words, thanks for pointing out. I was getting at the fact that SQL systems don't scale as well horizontally (it's difficult to distribute the same column across multiple machines), and inadvertently used a technical term connoting something else.
- randomdata 4y ago
  > I was getting at the fact that SQL systems don't scale as well horizontally Google, for example, claims that BigQuery, which uses SQL, scales horizontally. Funnily enough, it is also column-based. Row-stores, like MySQL Cluster, can also scale horizontally and uses SQL. But no matter what you are at the mercy of CAP theorem. Pick your poison.
  - jackconsidine 4y ago
    100% agree on the CAP theorem conundrum. I didn't know that BigQuery was SQL under the hood, interesting.
    - randomdata 4y ago
      SQL is a protocol. I'm not sure I would consider that "under the hood". What happens under the hood is up to the implementer, and implementations can vary widely (BigQuery is quite unlike Postgres, for example). It is the visible part that makes up a particular public API designed for interacting with databases. Conceivably even Firestore could expose SQL as one of its APIs. In fact, I once built a tool to do exactly that so I could perform ad-hoc Firestore queries using SQL. Worked beautifully.
      - cachemiss 4y ago
        Correct, a common mistake people make is conflating these things. I wrote this several years ago about MongoDB: One thing that helps is if people stop referring to things as SQL / NoSQL as what ends up happening is various things get conflated. When talking about stores, it's important to be explicit about a few things: 1. Storage model 2. Distribution model 3. Access model 4. Transaction model 5. Maturity and competence of implementation What happens is people talk about "SQL" as either an NSM or DSM storage model, over either a single node, or possibly more than that in some of the MPP systems, using SQL as an access model, with linearizable transactions, and a mature competent implementation. NoSQL when most people refer to it can be any combination of those things, as long as the access model isn't SQL. I work on database engines, and it's important to decouple these things and be explicit about them when discussing various tradeoffs. You can do SQL the language over a distributed k/v store (not always a great idea) and other non-tabular / relational models and you can distribute relational engines (though scaling linearizable transactions is difficult and doesn't scale for certain use cases due to physics, but that's unrelated to the relational part of it). Generally people talk about joins not scaling in some normalized form, but then what they do is just materialize the join into whatever they are using to store things in a denormalized model, which has its own drawbacks. As to the comment above you, SQL vs NoSQL also doesn't have anything to do with the relative maturity of anything. Some of the newer non-relational engines have some operational issues, but that doesn't really have anything to do with their storage model or access method, it just has to due with the competence of the implementation. MongoDB is difficult operationally not because it's not a relational engine, but because it wasn't well designed. Just like people put SQL over non-tabular stores, you can build non-tabular / relational engines over relational engines (sharding PostgreSQL etc.). In fact major cloud vendors do just that.
- simonw 4y ago
  I'd caution against believing that NoSQL databases as a category are more scalable than relational databases. We have decades of experience scaling relational databases at the moment, including horizontally - Flickr was using read-replicas and sharding MySQL back in 2005.

duxup 4y ago

I’m confused by that line and it made me wonder about their experience or PoV as well. But as someone who doesn’t word smith good, could just be that.

deleted 4y ago

[deleted]

jackconsidine 4y ago

Author here- poor choice of words, thanks for pointing out. I was getting at the fact that SQL systems don't scale as well horizontally (it's difficult to distribute the same column across multiple machines), and inadvertently used a technical term connoting something else.

randomdata 4y ago
> I was getting at the fact that SQL systems don't scale as well horizontally Google, for example, claims that BigQuery, which uses SQL, scales horizontally. Funnily enough, it is also column-based. Row-stores, like MySQL Cluster, can also scale horizontally and uses SQL. But no matter what you are at the mercy of CAP theorem. Pick your poison.
- jackconsidine 4y ago
  100% agree on the CAP theorem conundrum. I didn't know that BigQuery was SQL under the hood, interesting.
  - randomdata 4y ago
    SQL is a protocol. I'm not sure I would consider that "under the hood". What happens under the hood is up to the implementer, and implementations can vary widely (BigQuery is quite unlike Postgres, for example). It is the visible part that makes up a particular public API designed for interacting with databases. Conceivably even Firestore could expose SQL as one of its APIs. In fact, I once built a tool to do exactly that so I could perform ad-hoc Firestore queries using SQL. Worked beautifully.
    - cachemiss 4y ago
      Correct, a common mistake people make is conflating these things. I wrote this several years ago about MongoDB: One thing that helps is if people stop referring to things as SQL / NoSQL as what ends up happening is various things get conflated. When talking about stores, it's important to be explicit about a few things: 1. Storage model 2. Distribution model 3. Access model 4. Transaction model 5. Maturity and competence of implementation What happens is people talk about "SQL" as either an NSM or DSM storage model, over either a single node, or possibly more than that in some of the MPP systems, using SQL as an access model, with linearizable transactions, and a mature competent implementation. NoSQL when most people refer to it can be any combination of those things, as long as the access model isn't SQL. I work on database engines, and it's important to decouple these things and be explicit about them when discussing various tradeoffs. You can do SQL the language over a distributed k/v store (not always a great idea) and other non-tabular / relational models and you can distribute relational engines (though scaling linearizable transactions is difficult and doesn't scale for certain use cases due to physics, but that's unrelated to the relational part of it). Generally people talk about joins not scaling in some normalized form, but then what they do is just materialize the join into whatever they are using to store things in a denormalized model, which has its own drawbacks. As to the comment above you, SQL vs NoSQL also doesn't have anything to do with the relative maturity of anything. Some of the newer non-relational engines have some operational issues, but that doesn't really have anything to do with their storage model or access method, it just has to due with the competence of the implementation. MongoDB is difficult operationally not because it's not a relational engine, but because it wasn't well designed. Just like people put SQL over non-tabular stores, you can build non-tabular / relational engines over relational engines (sharding PostgreSQL etc.). In fact major cloud vendors do just that.
simonw 4y ago
I'd caution against believing that NoSQL databases as a category are more scalable than relational databases. We have decades of experience scaling relational databases at the moment, including horizontally - Flickr was using read-replicas and sharding MySQL back in 2005.