3 ms·
Seems like it's a much higher rate limit, but it exists none the less, and Shopify's scale has also grown significantly since 2018 (when that article was writte
by pattycake23 5y ago
Seems like it's a much higher rate limit, but it exists none the less, and Shopify's scale has also grown significantly since 2018 (when that article was written) - so it was probably a valid way for them to go.
- sciurus 5y agoI think two things happened that are covered in that blog post 1) The performance per partition increased 2) The way AWS created partitions changed When I was at Mozilla, one thing I worked on was Firefox's crash reporting system. It's S3 storage backend wrote raw crash data with the key in the format `{prefix}/v2/{name_of_thing}/{entropy}/{date}/{id}`. If I remember correctly, we considered this a limitation since the entropy was so far down in the key. However, when we talked to AWS Support they told us their was no longer a need to have the entropy early on; effectively S3 would "figure it out" and partition as needed. EDIT: https://news.ycombinator.com/item?id=30373375 https://news.ycombinator.com/item?id=30373375 is a good related comment.