You can set up cloud watch events to trigger a lambda function to store meta data about the s3 file in a regular database. That way you can index it how you exp

You can set up cloud watch events to trigger a lambda function to store meta data about the s3 file in a regular database. That way you can index it how you expect to list. Very effective for our use case.