4 ms·

What does everyone use S3 for? I'm genuinely curious. As my experiments with it have left me disappointed with its performance, I'm just not sure what I could

by erikbye 10y ago

What does everyone use S3 for? I'm genuinely curious. As my experiments with it have left me disappointed with its performance, I'm just not sure what I could use it for. Store massive amounts of data that is infrequently accessed? Well, unfortunately the upload speed I got to the standard rating one was so abysmal it would take too much time to move the data there; and then I suspect the inverse would be pretty bad as well.

harryh 10y ago
It's a very easy way to cheaply store static data to be served over HTTP. There are many many many many many many people who want to do that.
castis 10y ago
1 scenario: if you run a website that has a lot of static content (multiple GB of images, css, js, etc) and you dont want your http server to be responsible for serving that content then you give it all to s3 and let them serve it for you.
- erikbye 10y ago
  What about the performance of serving it? Sounds like I would need to cache it myself, anyway.
  - ckozlowski 10y ago
    Performance out of S3 is generally really good. However, if you're looking to say, serve up a global website and your content is in a single S3 region, then you can leverage CloudFront CDN to serve up those objects. CloudFront integrates seamlessly with S3, and you don't pay transfer charges between CloudFront and S3.
    - deleted 10y ago
      [deleted]
peu4000 10y ago
Database backups, puppet modules, a couple low traffic static sites, assets for one of our projects.
- erikbye 10y ago
  Database backups: with the upload speeds I've seen, completing a backup of a database with hundreds of GB would take a really long time. I don't want that extra load and keeping that connection open forever.
  - officelineback 10y ago
    S3 endpoints in your VPC?
ckozlowski 10y ago
S3 uploads can be very dependent on what the network looks like between your systems and the endpoint. We introduced S3 Transfer Acceleration to help address this. This uses our edge network as your endpoint, then sending your upload across our backbone rather than traversing the commercial internet. It comes with a small fee, but the fee only gets applied if it improves on what the time to transfer would be otherwise. https://aws.amazon.com/blogs/aws/aws-storage-update-amazon-s3-transfer-acceleration-larger-snowballs-in-more-regions/ https://aws.amazon.com/blogs/aws/aws-storage-update-amazon-s... If you have lots of data that needs to be uploads (TB/PB worth), then I'd take a look at AWS Snowball. https://aws.amazon.com/snowball/ https://aws.amazon.com/snowball/ Also, if you're using the AWS CLI to upload, make sure multi-part upload is enabled. Why use S3? It scales without any user interaction, is highly available (yes, I cringe saying that, but this is the first, and hopefully only time this has occurred! =D ) and extremely easy to access; it's as simple as an HTTP GET. Being able to address objects directly and not have to worry about managing file systems simplifies a lot.
foxylion 10y ago
We are storing database backups and static assets (e.g. images in mails) on S3.

by erikbye 10y ago

harryh 10y ago

It's a very easy way to cheaply store static data to be served over HTTP. There are many many many many many many people who want to do that.

castis 10y ago

1 scenario: if you run a website that has a lot of static content (multiple GB of images, css, js, etc) and you dont want your http server to be responsible for serving that content then you give it all to s3 and let them serve it for you.

erikbye 10y ago
What about the performance of serving it? Sounds like I would need to cache it myself, anyway.
- ckozlowski 10y ago
  Performance out of S3 is generally really good. However, if you're looking to say, serve up a global website and your content is in a single S3 region, then you can leverage CloudFront CDN to serve up those objects. CloudFront integrates seamlessly with S3, and you don't pay transfer charges between CloudFront and S3.
  - deleted 10y ago
    [deleted]

peu4000 10y ago

Database backups, puppet modules, a couple low traffic static sites, assets for one of our projects.

erikbye 10y ago
Database backups: with the upload speeds I've seen, completing a backup of a database with hundreds of GB would take a really long time. I don't want that extra load and keeping that connection open forever.
- officelineback 10y ago
  S3 endpoints in your VPC?

ckozlowski 10y ago

S3 uploads can be very dependent on what the network looks like between your systems and the endpoint. We introduced S3 Transfer Acceleration to help address this. This uses our edge network as your endpoint, then sending your upload across our backbone rather than traversing the commercial internet. It comes with a small fee, but the fee only gets applied if it improves on what the time to transfer would be otherwise. https://aws.amazon.com/blogs/aws/aws-storage-update-amazon-s3-transfer-acceleration-larger-snowballs-in-more-regions/ https://aws.amazon.com/blogs/aws/aws-storage-update-amazon-s... If you have lots of data that needs to be uploads (TB/PB worth), then I'd take a look at AWS Snowball. https://aws.amazon.com/snowball/ https://aws.amazon.com/snowball/ Also, if you're using the AWS CLI to upload, make sure multi-part upload is enabled. Why use S3? It scales without any user interaction, is highly available (yes, I cringe saying that, but this is the first, and hopefully only time this has occurred! =D ) and extremely easy to access; it's as simple as an HTTP GET. Being able to address objects directly and not have to worry about managing file systems simplifies a lot.

foxylion 10y ago

We are storing database backups and static assets (e.g. images in mails) on S3.