4 ms·

I don't know databases and there's quite a number of them on the market, so posts like these are great. Sidetrack though, does anyone have a list for pros and

I don't know databases and there's quite a number of them on the market, so posts like these are great. Sidetrack though, does anyone have a list for pros and cons of each db, with a preference towards low latency? Also how does it compare with say Postgres?

Transfinity 4y ago
That depends heavily on the shape of your data, what your workload looks like, what sort of consistency guarantees you need, etc. I recommend Designing Data Intensive Applications for getting a handle on this - it's the book I suggest to SDE IIs who are hungry for the jump to senior. Not a quick read, but well written, and there's not really a shortcut to deep understanding other than deep study.
- oreally 4y ago
  thanks, will look that up
- dominotw 4y ago
  > I suggest to SDE IIs who are hungry for the jump to senior Any suggestions for seniors who are hungry for a staff title. Is there any value to specializing in tech anymore after senior or is it all 'soft skills' at this point ?
  - asdfasgasdgasdg 4y ago
    It is not all any one thing. If you’re completely lacking in soft skills, you’ll probably have more trouble the higher up you go, and you may get stuck at some point. Hard technical skills become less important the higher you go, but even the CEO of a tech company probably still needs to have some basic understanding of technology. Being good at both things is obviously the best way forward.
  - dharmaturtle 4y ago
    I enjoyed this: https://staffeng.com/book https://staffeng.com/book Not a staff though so can't attest to how true/accurate/effective it is.
PeterCorless 4y ago
I did a talk about establishing criteria for differences between distributed SQL and NoSQL databases. https://www.youtube.com/watch?v=nJvnq8ZD_7Q https://www.youtube.com/watch?v=nJvnq8ZD_7Q There are a couple of different dimensions to consider. I didn't focus on this in the talk but firstly you should consider these elements... • Speed (latency & throughputs) • Scale (gigabytes? terabytes? petabytes? exabytes?) • Workload types (OLAP, OLTP, HTAP, data warehouses/lakes/lakehouses) • Consistency Models ("ACID vs. BASE", eventual vs. strong, etc.) • Data Model - RDBMS, NoSQL (Key-Value, Wide Column, Document, Graph, Column Store, etc.) • Query Language (SQL, streaming SQL [Flink], CQL [Cassandra, ScyllaDB, DataStax, CosmosDB, Keyspaces, Yugabyte, etc.], Gremlin/Tinkerpop [JanusGraph, DSE Graph, OrientDB, etc.], and so on. What I did focus on was more the "distributed system" attributes... • Clustering & distribution (local clustering, cross-clustering, multidatacenter clustering) • Node roles, high availability & failover strategies (primary/replica, peer-to-peer leaderless/active-active, load balancing) • Data replication/sharding strategies (replication factors, consistency levels, manual vs. auto-sharing, topology awareness [rack and datacenter awareness]) I only focused on a handful. There are literally hundreds of databases tracked on DB-engines: https://db-engines.com/en/ranking https://db-engines.com/en/ranking [Disclosure, I work for ScyllaDB]
- jeffbee 4y ago
  None of Bigtable, Spanner, or DynamoDB made your shortlist?
  - PeterCorless 4y ago
    For the sake of that talk I focused only on open source you could self-host. SQL • PostgreSQL • CockroachDB NoSQL • MongoDB • Redis • ScyllaDB [Cassandra] There are a lot of points in the DBaaS world to also consider, but there's also issues of visibility — can these issues be known, perceived or understood from a user's POV? Are they documented? Or are some of these considerations obscured and trade secreted? So I had to pare down. I planned to do a whole different talk on DBaaS systems, and yes, I'd likely shortlist Bigtable, Spanner, DynamoDB, and a few others for that. Only so much to do in an hour! :)

Transfinity 4y ago

That depends heavily on the shape of your data, what your workload looks like, what sort of consistency guarantees you need, etc. I recommend Designing Data Intensive Applications for getting a handle on this - it's the book I suggest to SDE IIs who are hungry for the jump to senior. Not a quick read, but well written, and there's not really a shortcut to deep understanding other than deep study.

oreally 4y ago
thanks, will look that up
dominotw 4y ago
> I suggest to SDE IIs who are hungry for the jump to senior Any suggestions for seniors who are hungry for a staff title. Is there any value to specializing in tech anymore after senior or is it all 'soft skills' at this point ?
- asdfasgasdgasdg 4y ago
  It is not all any one thing. If you’re completely lacking in soft skills, you’ll probably have more trouble the higher up you go, and you may get stuck at some point. Hard technical skills become less important the higher you go, but even the CEO of a tech company probably still needs to have some basic understanding of technology. Being good at both things is obviously the best way forward.
- dharmaturtle 4y ago
  I enjoyed this: https://staffeng.com/book https://staffeng.com/book Not a staff though so can't attest to how true/accurate/effective it is.

PeterCorless 4y ago

I did a talk about establishing criteria for differences between distributed SQL and NoSQL databases. https://www.youtube.com/watch?v=nJvnq8ZD_7Q https://www.youtube.com/watch?v=nJvnq8ZD_7Q There are a couple of different dimensions to consider. I didn't focus on this in the talk but firstly you should consider these elements... • Speed (latency & throughputs) • Scale (gigabytes? terabytes? petabytes? exabytes?) • Workload types (OLAP, OLTP, HTAP, data warehouses/lakes/lakehouses) • Consistency Models ("ACID vs. BASE", eventual vs. strong, etc.) • Data Model - RDBMS, NoSQL (Key-Value, Wide Column, Document, Graph, Column Store, etc.) • Query Language (SQL, streaming SQL [Flink], CQL [Cassandra, ScyllaDB, DataStax, CosmosDB, Keyspaces, Yugabyte, etc.], Gremlin/Tinkerpop [JanusGraph, DSE Graph, OrientDB, etc.], and so on. What I did focus on was more the "distributed system" attributes... • Clustering & distribution (local clustering, cross-clustering, multidatacenter clustering) • Node roles, high availability & failover strategies (primary/replica, peer-to-peer leaderless/active-active, load balancing) • Data replication/sharding strategies (replication factors, consistency levels, manual vs. auto-sharing, topology awareness [rack and datacenter awareness]) I only focused on a handful. There are literally hundreds of databases tracked on DB-engines: https://db-engines.com/en/ranking https://db-engines.com/en/ranking [Disclosure, I work for ScyllaDB]

jeffbee 4y ago
None of Bigtable, Spanner, or DynamoDB made your shortlist?
- PeterCorless 4y ago
  For the sake of that talk I focused only on open source you could self-host. SQL • PostgreSQL • CockroachDB NoSQL • MongoDB • Redis • ScyllaDB [Cassandra] There are a lot of points in the DBaaS world to also consider, but there's also issues of visibility — can these issues be known, perceived or understood from a user's POV? Are they documented? Or are some of these considerations obscured and trade secreted? So I had to pare down. I planned to do a whole different talk on DBaaS systems, and yes, I'd likely shortlist Bigtable, Spanner, DynamoDB, and a few others for that. Only so much to do in an hour! :)