5 ms·

I got 3 consecutive emails warning that my budget crossed its $18 threshold. Opened it up: cost was 78 million. Thought it was a phishing attempt, logged into m

I got 3 consecutive emails warning that my budget crossed its $18 threshold. Opened it up: cost was 78 million. Thought it was a phishing attempt, logged into my actual account, and... still 78 million. EMOTIONAL DAMAGE.

SegfaultSeagull 3mo ago
Time to get a second job buddy.
rcleveng 3mo ago
I did too, those awstrack.me URL's look super suspicious and I hadn't seen this alert trigger before so didn't know what to expect. At first I was sure it was a phishing attempt. Then went to the console (not using those links) Saw there was an outage where the console was wrong (no mention of email alerts) Then I thought I was hacked - what a perfect cover up for someone to evade detection when the console was wrong. Looked at some logs, realized the incident text was just not exhaustive on the impact. Went back to my cup of coffee. Note to self- should have looked here first.
root-parent 3mo ago
Wanna bet the description of this job post will be updated by the end of the day? "Software Development Engineer II, AWS Invoicing" https://www.amazon.jobs/de/jobs/10428480/software-development-engineer-ii-aws-invoicing https://www.amazon.jobs/de/jobs/10428480/software-developmen... "...Our platforms are powered by generative AI, large language models, knowledge graphs, and agentic architectures that dynamically compose specialized agents based on context. We apply these capabilities across three reinforcing areas: intelligent launch readiness — where autonomous AI agents analyze, generate, and validate the information needed to go live in a new market; cloud-native service orchestration — where configuration-driven microservices replace per-launch bespoke engineering with centralized, reusable capabilities so that expanding into a new country becomes a zero-code configuration change rather than a development cycle; and continuous validation..."
- sebmellen 3mo ago
  That job description feels so far beyond parody that I could scarcely believe it until opening the link! What a world.
  - root-parent 3mo ago
    It gets worst: "Senior Software Development Manager, AWS Global Bill Generation" https://www.amazon.jobs/de/jobs/10471948/senior-software-development-manager-aws-global-bill-generation https://www.amazon.jobs/de/jobs/10471948/senior-software-dev... "We're transforming from monthly batch processing and manual war rooms to continuous billing, autonomous agents, and self-healing infrastructure. We believe operational burden is a technical problem, not a staffing problem" This looks clearly...a staffing problem...
    - deleted 3mo ago
      [deleted]
    - iam-TJ 3mo ago
      The best bit of that is: > In this role, you will own end-to-end bill run execution across all AWS partitions, drive the technical vision for autonomous billing operations, and build the team that ensures every customer receives an accurate cost estimated in minutes ...
    - ghurtado 3mo ago
      > This looks clearly...a staffing problem.. I think that big tech recently decided that I got 99 problems but staffing ain't one I guess Nothing is a staffing problem when you make a rule that firing people is always the solution.
    - wbl 3mo ago
      If you can make the software cover the toil you save the staff for the tough cases.
    - quickthrowman 3mo ago
      They need to fire whoever is running AP and AP software development. Vibe invoicing is ridiculous for anyone to do, let alone Amazon.
jayanmn 3mo ago
Enterprise account . We got - 3trillion and change
- chii 3mo ago
  -$3 trillion! That's the highest earning investment that has ever existed!
- idiotsecant 3mo ago
  Quick do your IPO before the books update
  - munk-a 3mo ago
    It's true, if you're spending that much money you must be worth a ton! Just look at SpaceX!
- theflyingelvis 3mo ago
  3.7 billion. Offered to pay it in monthly installments. Haven’t hears back
- rconti 3mo ago
  same. over 2t in one day.
01284a7e 3mo ago
Yes, I am taking legal action, no doubt.
- dymk 3mo ago
  …for emotional damage?
  - inigyou 3mo ago
    If you were a business maybe you could claim for the emergency on-call time spent diagnosing, but you'd probably still lose AND amazon would fire you as a customer.
    - lstodd 3mo ago
      "fire you as a customer" I love this framing
- bot403 3mo ago
  Why? What's the damages? They showed you a wrong number, then later acknowledged it and fixed it. Just because the number was "very big" to you doesn't mean you were actually aggrieved in some way.
  - amelius 3mo ago
    Big numbers can lead to stress which can lead to all kinds of disorders.
    - mito88 3mo ago
      small numbers too.... :)
      - fc417fc802 3mo ago
        We should just go with numbers in general in order to play it safe. The new guidance from legal is absolutely no numbers on invoices for liability reasons. Similar to the removal of actual math from math class we're going to let the experts figure out how to implement it.
antognini 3mo ago
To paraphrase the old joke, if you wake up with a $78,000 AWS bill, you have a problem. If you wake up with a $78 million AWS bill, Amazon has a problem.
corvad 3mo ago
Please do not open up emails directly always login to your account.
shawkinaw 3mo ago
Mine was just over $1 billion. I thought the email was a phish at first and almost trashed it, but it seemed otherwise very real, so I logged in to AWS just to check. Then I almost had a heart attack when I looked at my billing estimate. Eventually I saw a banner on the support page, but my question is why wasn’t there a banner on the page SAYING I OWE A BILLION DOLLARS?!
jerf 3mo ago
The real fun comes later when the US Federal Government taxes us on the forgiven debt as income.
- kjs3 3mo ago
  The finance fun noone knows about. I knew someone who did this as revenge. Sent some asshole a fake invoice for US$stupidmoney for a couple of months which said asshole ignored, then sent a certified mail 'I forgive asshole US$stupidmoney', and copied the IRS. IRS went to collect on the unpaid taxes, and if you've never been there, the IRS does not play when it comes to that sort of money. I don't know how it ended, but there were multiple lawyers involved, so it was expensive no matter what.

SegfaultSeagull 3mo ago

Time to get a second job buddy.

rcleveng 3mo ago

I did too, those awstrack.me URL's look super suspicious and I hadn't seen this alert trigger before so didn't know what to expect. At first I was sure it was a phishing attempt. Then went to the console (not using those links) Saw there was an outage where the console was wrong (no mention of email alerts) Then I thought I was hacked - what a perfect cover up for someone to evade detection when the console was wrong. Looked at some logs, realized the incident text was just not exhaustive on the impact. Went back to my cup of coffee. Note to self- should have looked here first.

root-parent 3mo ago

Wanna bet the description of this job post will be updated by the end of the day? "Software Development Engineer II, AWS Invoicing" https://www.amazon.jobs/de/jobs/10428480/software-development-engineer-ii-aws-invoicing https://www.amazon.jobs/de/jobs/10428480/software-developmen... "...Our platforms are powered by generative AI, large language models, knowledge graphs, and agentic architectures that dynamically compose specialized agents based on context. We apply these capabilities across three reinforcing areas: intelligent launch readiness — where autonomous AI agents analyze, generate, and validate the information needed to go live in a new market; cloud-native service orchestration — where configuration-driven microservices replace per-launch bespoke engineering with centralized, reusable capabilities so that expanding into a new country becomes a zero-code configuration change rather than a development cycle; and continuous validation..."

sebmellen 3mo ago
That job description feels so far beyond parody that I could scarcely believe it until opening the link! What a world.
- root-parent 3mo ago
  It gets worst: "Senior Software Development Manager, AWS Global Bill Generation" https://www.amazon.jobs/de/jobs/10471948/senior-software-development-manager-aws-global-bill-generation https://www.amazon.jobs/de/jobs/10471948/senior-software-dev... "We're transforming from monthly batch processing and manual war rooms to continuous billing, autonomous agents, and self-healing infrastructure. We believe operational burden is a technical problem, not a staffing problem" This looks clearly...a staffing problem...
  - deleted 3mo ago
    [deleted]
  - iam-TJ 3mo ago
    The best bit of that is: > In this role, you will own end-to-end bill run execution across all AWS partitions, drive the technical vision for autonomous billing operations, and build the team that ensures every customer receives an accurate cost estimated in minutes ...
  - ghurtado 3mo ago
    > This looks clearly...a staffing problem.. I think that big tech recently decided that I got 99 problems but staffing ain't one I guess Nothing is a staffing problem when you make a rule that firing people is always the solution.
  - wbl 3mo ago
    If you can make the software cover the toil you save the staff for the tough cases.
  - quickthrowman 3mo ago
    They need to fire whoever is running AP and AP software development. Vibe invoicing is ridiculous for anyone to do, let alone Amazon.

jayanmn 3mo ago

Enterprise account . We got - 3trillion and change

chii 3mo ago
-$3 trillion! That's the highest earning investment that has ever existed!
idiotsecant 3mo ago
Quick do your IPO before the books update
- munk-a 3mo ago
  It's true, if you're spending that much money you must be worth a ton! Just look at SpaceX!
theflyingelvis 3mo ago
3.7 billion. Offered to pay it in monthly installments. Haven’t hears back
rconti 3mo ago
same. over 2t in one day.

01284a7e 3mo ago

Yes, I am taking legal action, no doubt.

dymk 3mo ago
…for emotional damage?
- inigyou 3mo ago
  If you were a business maybe you could claim for the emergency on-call time spent diagnosing, but you'd probably still lose AND amazon would fire you as a customer.
  - lstodd 3mo ago
    "fire you as a customer" I love this framing
bot403 3mo ago
Why? What's the damages? They showed you a wrong number, then later acknowledged it and fixed it. Just because the number was "very big" to you doesn't mean you were actually aggrieved in some way.
- amelius 3mo ago
  Big numbers can lead to stress which can lead to all kinds of disorders.
  - mito88 3mo ago
    small numbers too.... :)
    - fc417fc802 3mo ago
      We should just go with numbers in general in order to play it safe. The new guidance from legal is absolutely no numbers on invoices for liability reasons. Similar to the removal of actual math from math class we're going to let the experts figure out how to implement it.

antognini 3mo ago

To paraphrase the old joke, if you wake up with a $78,000 AWS bill, you have a problem. If you wake up with a $78 million AWS bill, Amazon has a problem.

corvad 3mo ago

Please do not open up emails directly always login to your account.

shawkinaw 3mo ago

Mine was just over $1 billion. I thought the email was a phish at first and almost trashed it, but it seemed otherwise very real, so I logged in to AWS just to check. Then I almost had a heart attack when I looked at my billing estimate. Eventually I saw a banner on the support page, but my question is why wasn’t there a banner on the page SAYING I OWE A BILLION DOLLARS?!

jerf 3mo ago

The real fun comes later when the US Federal Government taxes us on the forgiven debt as income.

kjs3 3mo ago
The finance fun noone knows about. I knew someone who did this as revenge. Sent some asshole a fake invoice for US$stupidmoney for a couple of months which said asshole ignored, then sent a certified mail 'I forgive asshole US$stupidmoney', and copied the IRS. IRS went to collect on the unpaid taxes, and if you've never been there, the IRS does not play when it comes to that sort of money. I don't know how it ended, but there were multiple lawyers involved, so it was expensive no matter what.