3 ms·
This, we love S3. What we did is add a SQL tier for some of the data we are storing there in case we want to do some more structured operations.
by xtracto 10y ago
This, we love S3. What we did is add a SQL tier for some of the data we are storing there in case we want to do some more structured operations.
- hoodoof 10y agoYes, but are you sure your database matches the underlying data store? The real problem with building a metadata index outside is that you then have the synchronization validation - yuk.
- illumin8 10y agoYou can always do a full scan of your S3 namespace every week or so and synchronize the index. This gives your consumers low latency access to the object store, as index lookups are extremely fast, it minimizes the cost of lookup events on S3.
- hoodoof 10y agoSo my database is up to a week wrong? Errr.....
- posterboy 10y agoin that it stores undeleted files, until weekly clean-up the DB is only incomplete for as long it takes to commit to the SQL layer after storing successfully.
- otterley 10y agoThis was just introduced last November: http://docs.aws.amazon.com/AmazonS3/latest/dev/storage-inventory.html http://docs.aws.amazon.com/AmazonS3/latest/dev/storage-inven...