It is slow, but most of the time, we don't use the raw data - we use the metadata. It worked great for our app. We got a excited and tried to implement some on

It is slow, but most of the time, we don't use the raw data - we use the metadata. It worked great for our app. We got a excited and tried to implement some on the fly data summaries where we would have to touch a bunch of files every time we wrote anything. That just didn't work because of the speed. Let's be specific on speed. Most of the time, the MB or two needed for a plot is a fraction of a second, which our customers can deal with. That retrieval is a single object from S3, the way we organize things. Having said that, the talk I linked to has some great advice - use the "folder" structure to write data so you don't search, you just use the naming scheme to do a direct object read. In addition, we can keep most reads to a single object, which is fairly fast. As is always true, you will need to test to see if it fits your speed needs. But even with just the naming scheme and meta data in the database to eliminate all searches on S3, the speed works for us.