4 ms·
We’re doing something similar at Codeball. A single Aurora RDS database (with read replicas), with a single table called “blobs”. The blobs table has three col
by zegl 4y ago
We’re doing something similar at Codeball.
A single Aurora RDS database (with read replicas), with a single table called “blobs”. The blobs table has three columns: id, type, and json.
It’s scaling really really well, to many many millions of rows, and high traffic volumes (many thousands of writes per second).
- wizwit999 4y agoWhy not just use DynamoDB at that point, would likely be cheaper.
- senttoschool 4y agoYea I agree. Seems like it should have been a NoSQL solution from that start. Even today, it should be pretty straightforward to port the data over to a NoSQL db.
- manfre 4y agoDynamodb can be very painful with certain usage patterns or if you need to change/add usage patterns. For startups or MVP apps/features, it's a lot easier and cheaper to adhoc SQL.
- ianberdin 4y agoSorry but I do not believe in this magic. How many queries in realities do you need to fetch all “page” data?
- Thorentis 4y agoWny even use RDS? There is nothing relational about that schema. Just use DynamoDB. It'll be cheaper, you can actually query the JSON, and if you decide to flesh out the documents and go all in on NoSQL then you can do that easily.
- zegl 4y agoIt's PostgreSQL, so we can and do query the JSON data, and update deepely nested fields in the JSON natively as well (no need to take a lock, read the entire blob, modify it in the application, save the data, and release the lock). PostgreSQL supports setting values, appending to arrays, etc etc natively in an ACID compliant way.
- paavohtl 4y ago> so we can and do query the JSON data Also supported in DynamoDB - it's a JSON-database after all. > update deepely nested fields in the JSON natively as well https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.UpdateExpressions.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg... > PostgreSQL supports setting values, appending to arrays, etc etc natively in an ACID compliant way. https://aws.amazon.com/blogs/aws/new-amazon-dynamodb-transactions/ https://aws.amazon.com/blogs/aws/new-amazon-dynamodb-transac... I'm not saying PostgreSQL is a bad choice by any means, or that DynamoDB is suitable for all projects. But good support for JSON is in no way a capability unique to PostgreSQL. At least ideally, a competently built document database should be at least as good if not better than an SQL database emulating a document database.
- platz 4y agoDynamoDB doesn't let you store JSON natively. You have to translate that json into dynamodb keys and attributes.
- deleted 4y ago[deleted]
- commandlinefan 4y ago
- blowski 4y agoSounds like an entity, attribute, value type schema? But where the value is nested data.
- poulsbohemian 4y agoHere's where I've never been smart enough to use this model... help me explain how you deal with scenarios like this... 1) There's an incident and we are trying to replicate the issue, but our database is just shapeless blobs and we can't track down the data that is pertinent to resolving the issue. 2) Product management wants to better understand our users and what they are doing so that they can design a new feature. But they can't figure out how to query our database or how to make sense of the unstructured data. 3) QA wants to be able to build tests and set up test data sets that match those tests, but rather than a sql script, they've got to roll all this json that doesn't have any constraints. These are just some examples, but maybe you get the gist of why I don't quite understand how this model scales "outside" the database.
- zegl 4y ago1) I think that the biggest difference between how you use KV/NoSQL databases is that your objects tends to be much bigger than a "normal" SQL row. Each object can be a parent entity, plus many sub-objects and sub-arrays of information. In a relational model, that data would have to go in other tables that you'd have to join etc. I believe that this model makes things much easier for us, as all you need to debug a single failure usually is stored in a single object, instead of spread out across the entire database. 2) Very true. We're using other tools (like PostHog) for collecting and analysing user behaviour. At a bigger scale, we'd probably ship of event data to something like BigQuery for analysis. 3) We don't have QA, but for testing, objects are created in code just like any other object, and unit tests (aka most of the tests), don't need the database at all. :-)