3 ms·

Ask HN: Why don't cloud file stores like Amazon S3 have query capability?

2 points by hoodoof 12y ago

So you have an Amazon S3 bucket with millions of items in it. Seems logical there would be some sort of query language to identify files according to a range of criteria. I wonder why there's no way to do so?

wmf 12y ago
Because that would be a database and would have database prices.
- hoodoof 12y ago
  I wouldn't mind that, it's not having any way to do queries that is frustrating.
  - nl 12y ago
    Then you want to run ElasticMapReduce[1] against your S3 buckets. [1] http://aws.amazon.com/elasticmapreduce/ http://aws.amazon.com/elasticmapreduce/
eldavido 12y ago
There's no way to do so because that's not how the system is built. Queries require indexes, which require more I/O, some level of storage consistency across the system (so that the query can run against a single "snap" of the database rather than something that's constantly changing!), and a lot of other things that just aren't designed in S3. A common pattern is to store bulk data in s3 but metadata (filenames, sizes, etc.) in a database for faster queries. If you need the ability to make arbitrary queries on the data itself, S3 isn't the product to use for this. Data architecture is a fundamental part of your system; don't fuss over spending some time on it!