4 ms·

This is definitely a good tip. Here's a blog post with more info on the topic: http://aws.typepad.com/aws/2012/03/amazon-s3-performance-tips-tricks-seattle-hir

• moe 13y ago
  This seems a rather odd implementation detail to burden the user with. Why don't you just use a hash of the filename for the partition key, so all files are distributed randomly regardless of their name? Is there an advantage for amazon or for the user in having S3 files cluster up by filename by default?
  • penguindev 13y ago
    most likely so dir listings are a lot faster. (they are sorted by key, and can use delimiters for subdir emulation..)
    • dllthomas 13y ago
      ... but for that they just need to store the filenames sorted somewhere. Content could be located based on a hash of the filename, still.
  • sitkack 13y ago
    If this is true, it is umm, naively suboptimal (I have harsher words in my head).
• dbarlett 13y ago
  Relevant sessions from re:Invent: Building Scalable Applications on Amazon Simple Storage Service (2012) https://www.youtube.com/watch?v=YYnVRYbUR6A https://www.youtube.com/watch?v=YYnVRYbUR6A Maximizing Amazon S3 Performance (2013) https://www.youtube.com/watch?v=uXHw0Xae2ww https://www.youtube.com/watch?v=uXHw0Xae2ww