3 ms·
I could do this, and it certainly crossed my mind. But depending on the project, it could significantly increase the amount of code overall. It feels a feels a
by michalc 5y ago
I could do this, and it certainly crossed my mind. But depending on the project, it could significantly increase the amount of code overall. It feels a feels a bit... heavy to me.
I can understand the AWS auth code... while there seem to be so many layers in boto3/botocore. There's a lot of code to add a few HTTP headers to each request. (It was a few years ago, but I posted about not using boto3 https://charemza.name/blog/posts/aws/python/you-might-not-need-boto-3/ https://charemza.name/blog/posts/aws/python/you-might-not-ne...)
I might be suffering a little bit from not-invented-here syndrome admittedly... I don't know. But one of the things I really like about S3 is how much of its API is quite simple: it's just HTTP (I briefly wrote about _that_ as well https://charemza.name/blog/posts/http/s3/the-core-s3-api-is-beautiful/ https://charemza.name/blog/posts/http/s3/the-core-s3-api-is-...). My instinct is that in many cases, it's better to use a simple API directly rather than (what I see as) a more complex code base on top of it.
- orf 5y agoWhile I can see the logic behind that, what you are giving up is proper credential support (profile, env, instance, pod, etc) and a decent built in retry/exception policy. Using boto3 the entire library becomes something like: s3_client = boto3.client("s3") class S3VFS: def get_range(self, from_, to_): return s3_client.get_object(..., Range=f'bytes={from_}-{to_}')['Body'].read() with apsw.Connection(..., vfs=S3VFS()) That's drastically simpler, and safer to include from a library consumer point of view.