4 ms·
we use DynamoDB like a big hash table of s3 file locations.. we look up these locations via a key (at the time, it sounded like a pretty good use-case for it).
by cmollis 5y ago
we use DynamoDB like a big hash table of s3 file locations.. we look up these locations via a key (at the time, it sounded like a pretty good use-case for it). I suppose we could have used some other managed redis or memcached thing, but being an AWS shop, it was, and is, pretty useful. I have to say, it's been pretty effortless to configure.. read/write units are really the only thing we've had to configure (other than the base index) The rest of it has been easy. It has about a 100 million entries that are read or written pretty quickly.
- Jach 5y agoI remember talking to someone who was playing with AWS stuff for the first time and they had a similar architecture, using Dynamo for a lookup store. It still seems a bit odd to me though. It's been a long time since I've worked with the S3 API, so maybe it just doesn't support the same sort of thing, but wouldn't it be nicer to just query S3 with some key and get back either the path/URL to render a link, or the content itself? Why the Dynamo intermediary? (And on the other side, if you don't need to render a link to serve the content, why not use Dynamo as the actual document store and skip S3? Storage cost?)
- cmollis 5y agoYou're not going to get O(1) lookup using the S3 API. The APIs are basically O(n). So we we use DynamoDB to store the S3 URL.
- cmollis 5y agoMoreover, storing a large binary file (in this case) in Dynamo is probably not the best use-case for it..most likely would have to convert to base64 in and out of it.