3 ms·
So if I have a bunch of objects whose names are hashes like 2df6ad6ca44d06566cffde51155e82ad0947c736 that I expect to access randomly, is there any performance
by ebingdom 5y ago
So if I have a bunch of objects whose names are hashes like 2df6ad6ca44d06566cffde51155e82ad0947c736 that I expect to access randomly, is there any performance benefit to introducing artificial delimiters like 2d/f6/ad6ca44d06566cffde51155e82ad0947c736? I've seen this used in some places.
- jstarfish 5y agoI don't know what impact that partitioning pattern has on s3, but it has some obvious benefits if your app needs to revert to write to a normal filesystem instead (like for testing).
- elcomet 5y agoNo difference other than readability. And amazon may distribute your application with another prefix anyway, like "2d/f6/ad6c"
- dale_glass 5y agoTo AWS S3, '/' isn't a delimiter, it's a character that's part of the filename. So for instance "/foo/bar.txt" and "/foo//bar.txt" are different files in S3, even though they'd be the same file in a filesystem. This gets pretty fun if you want to mirror a S3 structure on-disk, because the above suddenly causes a collision.