4 ms·

Using S3 for static media makes perfect sense, but does anyone have information on S3 when your contributors (users in the admin) are uploading lots of images f

by adamrt 13y ago

Using S3 for static media makes perfect sense, but does anyone have information on S3 when your contributors (users in the admin) are uploading lots of images for via something like django-filebrowser or a simple zipfile upload (that extracts the images into a folder)? I haven't seen anything like this in any examples.

stevenwei 13y ago
I think the standard method of dealing with this is to have your users upload the items to the local web server and then have some sort of async task or cron job responsible for moving the files to S3 later on. Of course this means you'll have to keep track of where the uploaded file is currently located, but if you ever want to scale to multiple web servers you'll need to do that anyway.
- dangoldin 13y ago
  Yea - this is how I've been doing it. The server that gets the upload does some processing and then moves the content over to S3 and adds an entry to the database that references it. It also forces you to write stateless code which is nice bonus.
obviouslygreen 13y ago
This is supported quite well via django-storages. Unless the libs you're using are doing something manual instead of using Django's built-in file handling, this will neatly override the local-filesystem defaults and send all of your content to S3. http://django-storages.readthedocs.org/en/latest/ http://django-storages.readthedocs.org/en/latest/
adrianh 13y ago
You can use S3's CORS support to allow your users to upload directly to S3 without even touching your server. Downside is that it can be hard to debug and it requires your users to use JavaScript -- but otherwise it's a big win. See http://philfreo.com/blog/how-to-allow-direct-file-uploads-from-javascript-to-amazon-s3-signed-by-python/ http://philfreo.com/blog/how-to-allow-direct-file-uploads-fr... for a Python example.

4 ms·

Using S3 for static media makes perfect sense, but does anyone have information on S3 when your contributors (users in the admin) are uploading lots of images f

by adamrt 13y ago

stevenwei 13y ago
I think the standard method of dealing with this is to have your users upload the items to the local web server and then have some sort of async task or cron job responsible for moving the files to S3 later on. Of course this means you'll have to keep track of where the uploaded file is currently located, but if you ever want to scale to multiple web servers you'll need to do that anyway.
- dangoldin 13y ago
  Yea - this is how I've been doing it. The server that gets the upload does some processing and then moves the content over to S3 and adds an entry to the database that references it. It also forces you to write stateless code which is nice bonus.
obviouslygreen 13y ago
This is supported quite well via django-storages. Unless the libs you're using are doing something manual instead of using Django's built-in file handling, this will neatly override the local-filesystem defaults and send all of your content to S3. http://django-storages.readthedocs.org/en/latest/ http://django-storages.readthedocs.org/en/latest/
adrianh 13y ago
You can use S3's CORS support to allow your users to upload directly to S3 without even touching your server. Downside is that it can be hard to debug and it requires your users to use JavaScript -- but otherwise it's a big win. See http://philfreo.com/blog/how-to-allow-direct-file-uploads-from-javascript-to-amazon-s3-signed-by-python/ http://philfreo.com/blog/how-to-allow-direct-file-uploads-fr... for a Python example.