3 ms·
> I think most of these questions are first and most importantly _hardware_, _money_ and _physics_ questions. Actually, money (more accurately, cost) is a cons
by hintymad 3y ago
> I think most of these questions are first and most importantly _hardware_, _money_ and _physics_ questions.
Actually, money (more accurately, cost) is a constraint instead of a resource. S3 is known for its low cost, and S3 can easily dote out a 70% discount to its large customers and still make a profit. So, an interesting question is how to build an low-cost object store/
> interconnecting _a lot_ of the fastest available storage
"A lot of" leads to real challenges. Sooner or later you'll find that managing metadata will become a tough challenge. Case in point, Open-source systems often use systems like Zookeeper or etcd or single-node name server with hot standby for metadata management, which certainly won't be able to handle the scale of S3.
- emmanueloga_ 3y agoAbout cost, see [1]. Also, S3 prices have been increasing and there's been a bunch of alternative offers for object store from other companies. I think people in here (HN) comment often about increasing costs of AWS offerings. Distributed systems and consensus are inherently hard problem, but there are a lot of implementations that you can study (like Etcd that you mention, or NATS [2], which I've been playing with and looks super cool so far :-p) if you want to understand the internals, on top of many books and papers released. Again, I never said it was "easy" to build distributed systems, I just don't think there's any esoteric knowledge to what S3 provides. -- 1: https://en.wikipedia.org/wiki/Economies_of_scale https://en.wikipedia.org/wiki/Economies_of_scale 2: https://nats.io/ https://nats.io/
- mwarkentin 3y agoUh, reference for s3 prices going up? I’ve only ever seen them heading down over time.
- emmanueloga_ 3y agoSorry, this is just anecdotal from my recollection of reading random hacker news threads; I think people talk more about the bandwidth being expensive more than the storage itself [1]. -- 1: https://hn.algolia.com/?dateEnd=1696464000&dateRange=custom&dateStart=1611792000&page=0&prefix=true&query=S3%20expensive&sort=byPopularity&type=all https://hn.algolia.com/?dateEnd=1696464000&dateRange=custom&...