5 ms·

They’re for teams that still refuse a proper RDBMS but desire an Apache-level of infrastructure complexity.

zinclozenge 5y ago
I love this comment, but presumably there's a need for databases that are well suited for write heavy workloads. Also there's no great story for starting out with an RDBMS and then scaling out. Right now if you're with MySQL you have some options to scale out with read replicas, and then eventually replicate to a TiDB cluster. PostgreSQL I guess there's cockroachdb, but it's kind of meh.
- skyde 5y ago
  I believe Cloudflare usage of PostgreSQL is "write heavy" and is a RDBMS that scaled out very well for them. They are using CitusDB but it's still RDBMS and support most PostgreSQL SQL query without the need to rewrite your query or change your schema.
  - zinclozenge 5y ago
    That's true, I forgot about Citus.
  - skyde 5y ago
    In fact Youtube for a long time was running on MySQL with a tiny service called "vitess" on top of it to handle (Sharding/partitioning) automatically. I dont know if they are still using Vitess or migrated to Google other relational Database "Spanner"
    - jeffbee 5y ago
      YouTube switched some Vitess workloads over to Procella. http://www.vldb.org/pvldb/vol12/p2022-chattopadhyay.pdf http://www.vldb.org/pvldb/vol12/p2022-chattopadhyay.pdf In general I've noticed that people outside Google assume that Spanner is much more popular than it really is.
      - jjirsa 5y ago
        Something something bigtable
      - skyde 5y ago
        To be honest I am surprised its not more popular. if you work at google and its one of the option to choose from! Is there any reason to "not" choose spanner for new project and instead use something else?
        jeffbee 5y ago
        It's expensive.
  - doliveira 5y ago
    They moved to Clickhouse recently, though: https://blog.cloudflare.com/http-analytics-for-6m-requests-per-second-using-clickhouse/ https://blog.cloudflare.com/http-analytics-for-6m-requests-p...
  - deleted 5y ago
    [deleted]
doliveira 5y ago
There are other requirements other than low-volume transactional data and CRUDs...
- dreyfan 5y ago
  I’m guessing your usage of a RDBMS hasn’t extended beyond an ORM library.
  - doliveira 5y ago
    I'm obviously being hyperbolic, not sure about you. Point remains that scaling RDBMSs is not exactly trivial and it does look like most companies eventually give up.
    - dreyfan 5y ago
      Eh, there are numerous options today (vitess, citus, clickhouse, aurora, redshift, bigquery, snowflake). Clickhouse outperforms Scylla by a wide margin at a fraction of the cost and complexity. Any of the above makes your data accessible via standard SQL. You can hire and utilize proficient data analysts, not people writing esoteric queries in whatever flavor of NoSQL, while actually solving business needs.
      - doliveira 5y ago
        Not sure what you're talking about then? It seems you're just talking about the syntax used to query (which I agree SQL is a no-brainer) and not really about the "R" in "RDBMS"
        deleted 5y ago
        [deleted]
        dreyfan 5y ago
        You haven't made a point yourself, so honestly, I don't fucking know what we're talking about either. You came in with a hyperbolic comment that echoes the stance from the NoSQL community I've seen for the past decade. The one that basically says "We have zero fucking clue how to model our data so we're just going to jam everything into bunch of loosely connected JSON documents and spend the next decade adding basic SQL features from the 70s".
VirusNewbie 5y ago
You remind me of some ex coworkers who thought everything could be done on RDBMS systems and then when they had to export large amounts of data topped out at 3500 tps. Meanwhile I'm over here using Scylla and 3 nodes can give me roughly 3.5M TPS for data extraction. RDBMS are great for certain use cases but they do not scale and if you think they do you don't really know what scale means. Sure you can cite things like Github using MySql, but i'll also point out they've invested thousands upon thousands of man hours customizing layers on top of MySql to support their sharding which also, oops, had a bug that gave them almost half a day of downtime at one point. This is not a good use of someone's time, GH definitely made the wrong decision. Don't steer other people into making similar mistakes especially given it seems you have a lack of experience on designing systems at scale.
- deleted 5y ago
  [deleted]
- dreyfan 5y ago
  Exactly where did I suggest using a single mysql instance or bandaiding a fleet of them together? Clickhouse vastly outperforms and outscales Scylla with a fraction of the hardware and cost.
  - PeterCorless 5y ago
    Different purposes — OLAP vs. OLTP. Reminder of this thread: https://news.ycombinator.com/item?id=22478407 https://news.ycombinator.com/item?id=22478407
    - deleted 5y ago
      [deleted]
  - VirusNewbie 5y ago
    Clickhouse is in no way an RDBMS.
    - 5y ago
cocoonkid 5y ago
is there a way to have a short private chat? I am an AppSec Engineer. Really max 10 minutes of your precious time.

I love this comment, but presumably there's a need for databases that are well suited for write heavy workloads. Also there's no great story for starting out with an RDBMS and then scaling out. Right now if you're with MySQL you have some options to scale out with read replicas, and then eventually replicate to a TiDB cluster. PostgreSQL I guess there's cockroachdb, but it's kind of meh.

skyde 5y ago
I believe Cloudflare usage of PostgreSQL is "write heavy" and is a RDBMS that scaled out very well for them. They are using CitusDB but it's still RDBMS and support most PostgreSQL SQL query without the need to rewrite your query or change your schema.
- zinclozenge 5y ago
  That's true, I forgot about Citus.
- skyde 5y ago
  In fact Youtube for a long time was running on MySQL with a tiny service called "vitess" on top of it to handle (Sharding/partitioning) automatically. I dont know if they are still using Vitess or migrated to Google other relational Database "Spanner"
  - jeffbee 5y ago
    YouTube switched some Vitess workloads over to Procella. http://www.vldb.org/pvldb/vol12/p2022-chattopadhyay.pdf http://www.vldb.org/pvldb/vol12/p2022-chattopadhyay.pdf In general I've noticed that people outside Google assume that Spanner is much more popular than it really is.
    - jjirsa 5y ago
      Something something bigtable
    - skyde 5y ago
      To be honest I am surprised its not more popular. if you work at google and its one of the option to choose from! Is there any reason to "not" choose spanner for new project and instead use something else?
      - jeffbee 5y ago
        It's expensive.
- doliveira 5y ago
  They moved to Clickhouse recently, though: https://blog.cloudflare.com/http-analytics-for-6m-requests-per-second-using-clickhouse/ https://blog.cloudflare.com/http-analytics-for-6m-requests-p...
- deleted 5y ago
  [deleted]

doliveira 5y ago

There are other requirements other than low-volume transactional data and CRUDs...

dreyfan 5y ago
I’m guessing your usage of a RDBMS hasn’t extended beyond an ORM library.
- doliveira 5y ago
  I'm obviously being hyperbolic, not sure about you. Point remains that scaling RDBMSs is not exactly trivial and it does look like most companies eventually give up.
  - dreyfan 5y ago
    Eh, there are numerous options today (vitess, citus, clickhouse, aurora, redshift, bigquery, snowflake). Clickhouse outperforms Scylla by a wide margin at a fraction of the cost and complexity. Any of the above makes your data accessible via standard SQL. You can hire and utilize proficient data analysts, not people writing esoteric queries in whatever flavor of NoSQL, while actually solving business needs.
    - doliveira 5y ago
      Not sure what you're talking about then? It seems you're just talking about the syntax used to query (which I agree SQL is a no-brainer) and not really about the "R" in "RDBMS"
      - deleted 5y ago
        [deleted]
      - dreyfan 5y ago
        You haven't made a point yourself, so honestly, I don't fucking know what we're talking about either. You came in with a hyperbolic comment that echoes the stance from the NoSQL community I've seen for the past decade. The one that basically says "We have zero fucking clue how to model our data so we're just going to jam everything into bunch of loosely connected JSON documents and spend the next decade adding basic SQL features from the 70s".

VirusNewbie 5y ago

You remind me of some ex coworkers who thought everything could be done on RDBMS systems and then when they had to export large amounts of data topped out at 3500 tps. Meanwhile I'm over here using Scylla and 3 nodes can give me roughly 3.5M TPS for data extraction. RDBMS are great for certain use cases but they do not scale and if you think they do you don't really know what scale means. Sure you can cite things like Github using MySql, but i'll also point out they've invested thousands upon thousands of man hours customizing layers on top of MySql to support their sharding which also, oops, had a bug that gave them almost half a day of downtime at one point. This is not a good use of someone's time, GH definitely made the wrong decision. Don't steer other people into making similar mistakes especially given it seems you have a lack of experience on designing systems at scale.

deleted 5y ago
[deleted]
dreyfan 5y ago
Exactly where did I suggest using a single mysql instance or bandaiding a fleet of them together? Clickhouse vastly outperforms and outscales Scylla with a fraction of the hardware and cost.
- PeterCorless 5y ago
  Different purposes — OLAP vs. OLTP. Reminder of this thread: https://news.ycombinator.com/item?id=22478407 https://news.ycombinator.com/item?id=22478407
  - deleted 5y ago
    [deleted]
- VirusNewbie 5y ago
  Clickhouse is in no way an RDBMS.
  - 5y ago

cocoonkid 5y ago

is there a way to have a short private chat? I am an AppSec Engineer. Really max 10 minutes of your precious time.