3 ms·

my favourite is "your docker registry is publicly accessible" yeah man, i know that. i made it public. i eventually had to take it down, just to stop the flo

my favourite is "your docker registry is publicly accessible" yeah man, i know that. i made it public. i eventually had to take it down, just to stop the flood of beg bounties telling me about it.

cperciva 3y ago
Oh yeah... I don't run a docker registry, but Amazon feels it necessary to remind me periodically that FreeBSD releases are public AMIs, and their filesystem images are public, and I have publicly readable data in S3 (which is mandatory in order to create an AWS Marketplace listing). So much "yes I know it's supposed to be that way".
- adastra22 3y ago
  It’s so bizarre that every time I upload something to S3 I have to jump through a hoop to make it publicly readable and Amazon displays a massive warning sign. Like the only thing I use S3 for is hosting open source software binaries. Maybe there’s a use case for restricting access, but I don’t even know what that would be.
  - toomuchtodo 3y ago
    Inadvertent public buckets leading to data loss is what created those hoops. Trying to take the ammo out of the footgun. https://www.theregister.com/2022/12/14/aws_simple_storage_service_simplified/ https://www.theregister.com/2022/12/14/aws_simple_storage_se...
    - adastra22 3y ago
      But why would they upload private data to S3 in the first place? I’m just not understanding the context here.
      - ehnto 3y ago
        S3 is just a general storage bucket, anything you could use a hard disk for, some app is using S3 for that instead. It's particularly common in serverless app backend architectures where no permanent storage exists at all, so volumes are not an option. A good deal of hosting is done on ephemeral VM instances that get rebuilt, so you choose between volumes and S3, and S3 is a bit more flexible.
        deleted 3y ago
        [deleted]
      - GauntletWizard 3y ago
        Many applications use s3 as a data store for... Whatever. Uploaded or generated files, intermediate outputs, as a key-value store for large blobs. You can put all sorts of things on s3. The PDF containing your bank statement, your medical test results, whatever - S3 is plenty secure enough for that with sane configuration, and even by default with sane account management.
      - erikerikson 3y ago
        Because half the world is below average.
        WesolyKubeczek 3y ago
        Below median, not average
        erikerikson 3y ago
        Indeed, thank you for the correction.
      - jrmcauliffe 3y ago
        A fair chunk of modern corporate data ends up in OLAP systems that are now more often or not stored using s3 (or their MS/Google equivalents) in 'Data Lakes'. The concept of 'private data' whilst using cloud providers is an interesting one, but there has been enough work done by all parties involved to ensure that all but the most sensitive data is now created, stored and analysed using the systems provided by "Big Cloud".
  - electroly 3y ago
    You really need to think carefully about whether you want to expose an S3 bucket publicly. There are probably some valid reasons out there, but if you're not an AWS expert, it's likely that you're making a mistake. If I find out the name of your bucket I could cost you thousands of dollars of egress tonight before you wake up in the morning. It's _especially_ likely to happen to hosters of open source binaries because of people absent-mindedly downloading artifacts from CI jobs. No malice required. They make the mistake but you pay the bill.
    - adastra22 3y ago
      Hosting a file on a url anyone can access from anywhere is the whole point…
      - electroly 3y ago
        Public S3 buckets are not a good choice for that because of the "anyone could bankrupt you" reason above. That is generally NOT the use case for S3, and it's the reason why private is the default and there are alerts for public buckets. For public access, within AWS, you'd want to put CloudFront in front of the bucket and only allow external users to access CloudFront. However, there's a better option... Outside of AWS, Cloudflare has a service called R2 which is just like S3, except you DON'T pay egress! It's the same thing, but without the "anyone could bankrupt you" aspect. You pay for the storage but you don't pay per download. I highly recommend this for hosting open source binaries. You can still keep a copy in a private S3 bucket for safe keeping if you don't trust Cloudflare long-term.
        adastra22 3y ago
        Thank you I’ll look into that.
  - DylanSp 3y ago
    Given the amount of data breaches that have happened from S3 buckets being accidentally publicly accessible, it definitely makes sense to me.

my favourite is "your docker registry is publicly accessible" yeah man, i know that. i made it public. i eventually had to take it down, just to stop the flood of beg bounties telling me about it.

cperciva 3y ago

Oh yeah... I don't run a docker registry, but Amazon feels it necessary to remind me periodically that FreeBSD releases are public AMIs, and their filesystem images are public, and I have publicly readable data in S3 (which is mandatory in order to create an AWS Marketplace listing). So much "yes I know it's supposed to be that way".

adastra22 3y ago
It’s so bizarre that every time I upload something to S3 I have to jump through a hoop to make it publicly readable and Amazon displays a massive warning sign. Like the only thing I use S3 for is hosting open source software binaries. Maybe there’s a use case for restricting access, but I don’t even know what that would be.
- toomuchtodo 3y ago
  Inadvertent public buckets leading to data loss is what created those hoops. Trying to take the ammo out of the footgun. https://www.theregister.com/2022/12/14/aws_simple_storage_service_simplified/ https://www.theregister.com/2022/12/14/aws_simple_storage_se...
  - adastra22 3y ago
    But why would they upload private data to S3 in the first place? I’m just not understanding the context here.
    - ehnto 3y ago
      S3 is just a general storage bucket, anything you could use a hard disk for, some app is using S3 for that instead. It's particularly common in serverless app backend architectures where no permanent storage exists at all, so volumes are not an option. A good deal of hosting is done on ephemeral VM instances that get rebuilt, so you choose between volumes and S3, and S3 is a bit more flexible.
      - deleted 3y ago
        [deleted]
    - GauntletWizard 3y ago
      Many applications use s3 as a data store for... Whatever. Uploaded or generated files, intermediate outputs, as a key-value store for large blobs. You can put all sorts of things on s3. The PDF containing your bank statement, your medical test results, whatever - S3 is plenty secure enough for that with sane configuration, and even by default with sane account management.
    - erikerikson 3y ago
      Because half the world is below average.
      - WesolyKubeczek 3y ago
        Below median, not average
        erikerikson 3y ago
        Indeed, thank you for the correction.
    - jrmcauliffe 3y ago
      A fair chunk of modern corporate data ends up in OLAP systems that are now more often or not stored using s3 (or their MS/Google equivalents) in 'Data Lakes'. The concept of 'private data' whilst using cloud providers is an interesting one, but there has been enough work done by all parties involved to ensure that all but the most sensitive data is now created, stored and analysed using the systems provided by "Big Cloud".
- electroly 3y ago
  You really need to think carefully about whether you want to expose an S3 bucket publicly. There are probably some valid reasons out there, but if you're not an AWS expert, it's likely that you're making a mistake. If I find out the name of your bucket I could cost you thousands of dollars of egress tonight before you wake up in the morning. It's _especially_ likely to happen to hosters of open source binaries because of people absent-mindedly downloading artifacts from CI jobs. No malice required. They make the mistake but you pay the bill.
  - adastra22 3y ago
    Hosting a file on a url anyone can access from anywhere is the whole point…
    - electroly 3y ago
      Public S3 buckets are not a good choice for that because of the "anyone could bankrupt you" reason above. That is generally NOT the use case for S3, and it's the reason why private is the default and there are alerts for public buckets. For public access, within AWS, you'd want to put CloudFront in front of the bucket and only allow external users to access CloudFront. However, there's a better option... Outside of AWS, Cloudflare has a service called R2 which is just like S3, except you DON'T pay egress! It's the same thing, but without the "anyone could bankrupt you" aspect. You pay for the storage but you don't pay per download. I highly recommend this for hosting open source binaries. You can still keep a copy in a private S3 bucket for safe keeping if you don't trust Cloudflare long-term.
      - adastra22 3y ago
        Thank you I’ll look into that.
- DylanSp 3y ago
  Given the amount of data breaches that have happened from S3 buckets being accidentally publicly accessible, it definitely makes sense to me.