5 ms·
Forget about software or data architecture. S3 is the most reliable data storage mechanism in the world, and insanely simpler than a relational database. There
by throwawaaarrgh 3y ago
Forget about software or data architecture. S3 is the most reliable data storage mechanism in the world, and insanely simpler than a relational database. There is no operational failure mode to S3, other than "region went down". There is no instance to go down, no replication to fail, no worry about whether there's enough capacity for writes or too many connections, no thought to a schema, no migrations to manage, no storage to grow, no logs to rotate, no software to upgrade on a maintenance window. Plus S3 is versioned, has multiple kinds of access control built in, is a protocol supported by many vendors and open source projects, and is (on AWS) strongly read-after-write consistent. I would also argue (though I don't have figures) that it's faster than RDS. Almost guaranteed it's cheaper. And it's integrated into many services in AWS making it easier to build more functionality into the system without programming.
On a less technical note: Always avoid the fancy option when it makes sense. (From a veteran of building and maintaining large scale high performance high availability systems)
- ilyt 3y agoThe "fancy" option here is trying to act S3 to act like database instead of simple blob storage...
- throwawaaarrgh 3y agoFancy would be writing an application to talk to RDS, creating an RDS instance, creating a database, creating whatever IAM link is needed for auth into the db so you don't need a second set of credentials, creating a schema, creating columns with different data types, and then modifying the application to handle edge cases for the different data types, logic to insert, update, delete rows, select items, yadda yadda yadda. Dumb is `aws s3 cp` and being done in 5 minutes.
- zetalyrae 3y ago[flagged]
- HumanOstrich 3y agoThis comment is unnecessarily antagonistic, and you are consistently and willfully missing all the points in your replies. He could have set up a more complex architecture and paid much more in hosting costs over the years to overengineer the solution. What would the benefits be? It might have avoided this one outage or saved a few hours restoring the data. The drawbacks? Much more time developing and maintaining the solution and higher subscription costs for users. The solutions you are familiar with and comfortable with are perfectly valid. But you are falling into the trap of thinking "what I'm familiar with and comfortable with is the only valid answer and everyone else is wrong and stupid".
- dang 3y agoPlease see https://news.ycombinator.com/item?id=36897868 https://news.ycombinator.com/item?id=36897868 and please stop posting in the flamewar style to HN. Regardless of how right you are / how much smarter you are or feel you are, it's exactly what the rules here ask you not to do. We're trying for a very different quality of conversation here. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- ilyt 3y agoI envy job where you think a database is something fancy and hard to do...
- otabdeveloper4 3y agoI'm only wildly guessing here, but most likely the "cloud storage" backing all those managed databases is actually S3-like blob storage under the hood.
- zetalyrae 3y agoThe database engine implements ACID and crash recovery so you don't have to. This is precisely what failed in this outage.
- zetalyrae 3y ago>S3 is the most reliable data storage mechanism in the world S3 is not the problem here. The problem is building a database on top of S3, and having to reimplement all the consistency, atomicity, transactions etc. on top. >no thought to a schema, no migrations to manage There is, in fact, always a schema. Some people choose to ignore it's there, to their detriment. >Always avoid the fancy option when it makes sense. It's not the 1980s. Postgres is not fancy, and Greenspunning it is a mistake. >Almost guaranteed it's cheaper. Cheaper than a 26-hour outage?
- throwawaaarrgh 3y ago> having to reimplement all the consistency, atomicity, transactions etc. on top. Did you miss where I said it's read-after-write strongly consistent?
- zetalyrae 3y agoHaving those things at the base layer is not the same as having them in higher-level behaviour. This is why God invented SQL transactions.
- mattigames 3y agoI never knew Donald Chamberlin and Raymond Boyce were God, it doesn't sound right but I don't know enough about theology to refute it.
- bastawhiz 3y ago> having to reimplement all the consistency, atomicity, transactions etc. on top. Most of those problems are moot if you're only ever writing from a single head node. If all your data is strictly ordered and you have no meaningful concurrency, this is a far, far simpler system.
- zetalyrae 3y agoDid I fall into a timewarp into the 70's? How on Earth, by what sane standard, is a Postgres instance too complex? If you're `fopen`ing files as a "database" you are wasting your time and lowering the world's economic productivity. Complex is Greenspunning a database and having it blow up in your face and cause a twenty-six hour outage. You never hear about such things with Postgres because Postgres is rock-solid.