4 ms·
How much does it cost to use on S3? For example, how many GetObject and other misc non-free API calls does it use? And does it store data using intelligent tie
by CoolGuySteve 6y ago
How much does it cost to use on S3? For example, how many GetObject and other misc non-free API calls does it use? And does it store data using intelligent tiering?
- daviesliu 6y agoFor small files (less than 4MB), the number of Get/Put/Delete is the same comparing using S3 directly. For larger file, each object in S3 is about 4MiB. Most of S3 Client will do the similar thing to GET/PUT small parts in parallel to speed things up. Overall, JuiceFS should use the similar number of GET/PUT request comparing to use S3 directly. Second, all the List and Head request go to Redis, they are free, so you may save some cost on API costs. Third, the frequently read data will be cached in your local disks, so you will also save some cost on GET/PUT requests.
- daviesliu 6y agoThe underlying S3 bucket still have intelligent tiering, you can also put life cycle rules on it.
- corford 6y agoIf a lifecycle rule deletes an object from the bucket, does the redis metadata server gracefully handle this (i.e. bucket state drifting)?
- daviesliu 6y agoJuiceFS access objects in S3 using an unique id to generate key, so the result of lifecycle will not change the way JuiceFS accessing it, except the StorageClass Glacier or Deep Arching, which is not accessible instantly, will hurt the user experience for JuiceFS.
- corford 6y agoThanks. Sorry I wasn't being very clear, I meant: what happens if an object is permanently deleted from a bucket by an out-of-band process, invisible to JuuiceFS (e.g. by a user operating on the bucket directly or by a misconfigured life cycle rule that deletes old objects after X days/months/years)? Does JuiceFS's metadata server handle this loss of synchronisation gracefully?