3 ms·

Disagree. Considering the original goal had nothing to do with expecting S3 to go down, it just happened to be super useful during this incident. We get more mo

Disagree. Considering the original goal had nothing to do with expecting S3 to go down, it just happened to be super useful during this incident. We get more more benefits even without S3 breaking.

cakoose 10y ago
OP is a little abrupt, but it isn't downplaying the system's usefulness to you guys; just warning other readers that this isn't a general solution for, as the post is titled, "dodging S3 downtime". It's a "fluke" precisely because it "just happened" to help with something it wasn't designed for. It's not uncommon for applications to have MRU access patterns and be able to keep partially functioning during partial data availability. For these applications, a cache will lower costs and mitigate S3 outages. It would have been nice for the article to give the criteria up front.
- mattrobenolt 10y ago
  Is paragraph 2 of the post not upfront enough? I clearly stated our goals of the project, and the fact that this also saves us through the incident was a great side effect. But I never set out to mitigate S3 failures like this.
  - cakoose 10y ago
    > Is paragraph 2 of the post not upfront enough? I clearly stated our goals of the project That paragraph says your connection to S3 is slow. Your solution doesn't fix general S3 slowness. However, it does make S3 slowness less of a problem for your application. That's why it would have been useful to describe how your application accesses S3, so others can quickly determine if their applications would also benefit from a similar solution. For example: "90% of our S3 reads are of blobs that were read in the last 30 days." Many applications might upload data to S3, then quickly download it for processing. For those applications, this solution won't work. > But I never set out to mitigate S3 failures like this. We know. It's just that some people might read the title and think this is a more general solution than it is. OP was just warning people against that.

Disagree. Considering the original goal had nothing to do with expecting S3 to go down, it just happened to be super useful during this incident. We get more more benefits even without S3 breaking.

cakoose 10y ago

OP is a little abrupt, but it isn't downplaying the system's usefulness to you guys; just warning other readers that this isn't a general solution for, as the post is titled, "dodging S3 downtime". It's a "fluke" precisely because it "just happened" to help with something it wasn't designed for. It's not uncommon for applications to have MRU access patterns and be able to keep partially functioning during partial data availability. For these applications, a cache will lower costs and mitigate S3 outages. It would have been nice for the article to give the criteria up front.

mattrobenolt 10y ago
Is paragraph 2 of the post not upfront enough? I clearly stated our goals of the project, and the fact that this also saves us through the incident was a great side effect. But I never set out to mitigate S3 failures like this.
- cakoose 10y ago
  > Is paragraph 2 of the post not upfront enough? I clearly stated our goals of the project That paragraph says your connection to S3 is slow. Your solution doesn't fix general S3 slowness. However, it does make S3 slowness less of a problem for your application. That's why it would have been useful to describe how your application accesses S3, so others can quickly determine if their applications would also benefit from a similar solution. For example: "90% of our S3 reads are of blobs that were read in the last 30 days." Many applications might upload data to S3, then quickly download it for processing. For those applications, this solution won't work. > But I never set out to mitigate S3 failures like this. We know. It's just that some people might read the title and think this is a more general solution than it is. OP was just warning people against that.