5 ms·

OP here. The warning doesn't actually have to do with what resulted in the additional charge, unless I'm missing something big.

duckfang 6y ago
Please read the orange triangle warning you scrolled past. When you did the glacier operation, the GUI was telling you that it was COPYING the files to glacier. Those files also existed elsewhere, in normal S3 space. There's ways to cure this issue at the command line, primarily a list of all attributes per file - and there you can see the respective storage plans. Secondly, do you have billing alerts set up, at 1.25x per average day's accrual? (30$/mo would be around $1.25/day, as not to email-bomb you). Billing alerts would NOT have solved the issue of improper storage class, but you would have seen a massive spike in costs. It probably would have cost you maybe $100-$200 to solve it early on. Thirdly, how much of the 5 Pillars of Well Architected Framework are you following? ( https://aws.amazon.com/blogs/apn/the-5-pillars-of-the-aws-well-architected-framework/ https://aws.amazon.com/blogs/apn/the-5-pillars-of-the-aws-we... ) Admittedly some of these are "pay AWS double and get disaster prevention" style rules. However... pillar 5 is all about cost optimization. An ounce of prevention is worth a pound (or $7k) of cure.
- mikeyla85 6y ago
  Ah that's interesting. In my support conversations with Amazon they never brought up copies of objects as a reason billing could be high, so I didn't look into it. They told me the issue was incomplete multipart uploads, so I just believed that. My assumption when reading that message was that it would appear to update the storage class, but actually create a copy of objects with a different storage class and delete the old objects. It doesn't make sense that it would just duplicate my objects, and I still can't really imagine anyone wanting that. Why would a box called "Edit storage class" create a copy and then not destroy it? Why not make a dialog box that just...edited the storage class? So maybe I had a previous version of all my files sitting in regular S3, with the current version sitting in Deep Glacier, and I may have just happened to delete previous versions of objects when I created the lifecycle rule. To answer your question -- I don't think we're following any of the pillars of well architected frameworks. I'm a filmmaker who knows Javascript and PHP and not system architecture, and I was looking for a cheap way to not lose our movie to the ravages of time. I'm definitely not the only one.
  - e_proxus 6y ago
    Not dismissing your other concerns, but > create a copy of objects with a different storage class and delete the old objects. It doesn't make sense that it would just duplicate my objects Isn’t that exactly what 'copy' means in pretty much all IT contexts?
    - mikeyla85 6y ago
      Not really in S3... It seems that some "move" operations are "copy" operations, which is the context in which I understood this warning: https://forums.aws.amazon.com/thread.jspa?messageID=455101#jive-message-455306 https://forums.aws.amazon.com/thread.jspa?messageID=455101#j...
      - e_proxus 6y ago
        Seems to me that there are no “move” operations in S3, only copy and delete (which mean exactly what they say). So copy still means copy, and if you implement your own “move” you have to make sure it does what you want.
        mikeyla85 6y ago
        Yep, which is what makes "creates a copy" in this warning seem like "copy and delete". Because just creating a copy and not deleting the original is nothing at all like editing storage class.

duckfang 6y ago

Please read the orange triangle warning you scrolled past. When you did the glacier operation, the GUI was telling you that it was COPYING the files to glacier. Those files also existed elsewhere, in normal S3 space. There's ways to cure this issue at the command line, primarily a list of all attributes per file - and there you can see the respective storage plans. Secondly, do you have billing alerts set up, at 1.25x per average day's accrual? (30$/mo would be around $1.25/day, as not to email-bomb you). Billing alerts would NOT have solved the issue of improper storage class, but you would have seen a massive spike in costs. It probably would have cost you maybe $100-$200 to solve it early on. Thirdly, how much of the 5 Pillars of Well Architected Framework are you following? ( https://aws.amazon.com/blogs/apn/the-5-pillars-of-the-aws-well-architected-framework/ https://aws.amazon.com/blogs/apn/the-5-pillars-of-the-aws-we... ) Admittedly some of these are "pay AWS double and get disaster prevention" style rules. However... pillar 5 is all about cost optimization. An ounce of prevention is worth a pound (or $7k) of cure.

mikeyla85 6y ago
Ah that's interesting. In my support conversations with Amazon they never brought up copies of objects as a reason billing could be high, so I didn't look into it. They told me the issue was incomplete multipart uploads, so I just believed that. My assumption when reading that message was that it would appear to update the storage class, but actually create a copy of objects with a different storage class and delete the old objects. It doesn't make sense that it would just duplicate my objects, and I still can't really imagine anyone wanting that. Why would a box called "Edit storage class" create a copy and then not destroy it? Why not make a dialog box that just...edited the storage class? So maybe I had a previous version of all my files sitting in regular S3, with the current version sitting in Deep Glacier, and I may have just happened to delete previous versions of objects when I created the lifecycle rule. To answer your question -- I don't think we're following any of the pillars of well architected frameworks. I'm a filmmaker who knows Javascript and PHP and not system architecture, and I was looking for a cheap way to not lose our movie to the ravages of time. I'm definitely not the only one.
- e_proxus 6y ago
  Not dismissing your other concerns, but > create a copy of objects with a different storage class and delete the old objects. It doesn't make sense that it would just duplicate my objects Isn’t that exactly what 'copy' means in pretty much all IT contexts?
  - mikeyla85 6y ago
    Not really in S3... It seems that some "move" operations are "copy" operations, which is the context in which I understood this warning: https://forums.aws.amazon.com/thread.jspa?messageID=455101#jive-message-455306 https://forums.aws.amazon.com/thread.jspa?messageID=455101#j...
    - e_proxus 6y ago
      Seems to me that there are no “move” operations in S3, only copy and delete (which mean exactly what they say). So copy still means copy, and if you implement your own “move” you have to make sure it does what you want.
      - mikeyla85 6y ago
        Yep, which is what makes "creates a copy" in this warning seem like "copy and delete". Because just creating a copy and not deleting the original is nothing at all like editing storage class.