5 ms·

GuardDuty is another example of brilliance of AWS pricing scheme and how they manage to twist your hand to pay extra which can cost quite a lot in the end of th

by m1keil 4y ago

GuardDuty is another example of brilliance of AWS pricing scheme and how they manage to twist your hand to pay extra which can cost quite a lot in the end of the month. When comparing EC2 to servers, nobody adds the added premiums of the extras. Things like CloudTrail, Support, GuardDuty, CloudWatch. All of these things have a variable cost that grows with usage and very hard to predict ahead of time. Just last week I discovered our GuardDuty bills went up from $15/month to $400/month. Inspecting closer, the issue was a small script that did AWS API call at a tight while loop in a couple of EC2 instances. So if you choose to enable GD, make sure to have your monitoring in place, gradually enable GD across the infra and establish clear baselines and alerting in place for costs.

deleted 4y ago
[deleted]
bushbaba 4y ago
Until you realize how much the alternatives cost. You think Palo Alto networks is cheap?
- albert_e 4y ago
  Is there a feature and price comparison between the native AWS offering and Palo Alto somewhere? Would be super handy for one of our projects right now. Thanks!
- Bluecobra 4y ago
  At least you can get a fixed cost if you go that route vs. your bills growing at an exponential rate. From what I have found, PA really wants you to commit to annual pricing with their VM appliances. If you go hourly you pay 73% more.
- unethical_ban 4y ago
  Do you think Palo Alto's offerings and GuardDuty are comparable?
philliphaydon 4y ago
Na. At some point you scale to the point that $400/m is peanuts for the benefits you get. We used it at previous job and realised we were under constant attack and and it reduced our cou usage by 15% in reduced requests for the amount of traffic we were getting. No more random spikes. Improved our overall security and ultimately reduced costs on the AWS bill. But if you’re gonna switch it on and walk away then you’re not really using it.
- m1keil 4y ago
  Yes of course, $400/month is peanuts so does $8000/month for certain companies. The point is not the absolute sum but how easy is it to spike your bill by 30 times. The way GD works, you are pretty much guaranteed to overpay for it sooner or later. If you can afford it.. great.
  - philliphaydon 4y ago
    One thing that we noticed was after switching it on, the EC2 instances were being hit directly, so we moved those into a private security group only accessible to the load balancer. RDS got restricted. S3 buckets fixed. Coupled with AWF to block on inregular activity, resulted in GD bill going down, not up. This is no different from programming. PHP has some awful code out in the wild, it doesn't mean PHP is shit just because people write bad code. The issue with AWS is it's far too easy for people to just spin stuff up and it works and they don't look at what they are being billed for, don't analysis their infrastructure, don't optimize. They just throw servers, containers, etc up into the wild then when the bill comes: "OH AWS BAD I got billed cos I just set it up and forgot about it, then when it worked they charged me for it, AWS is wrong, just go baremetal."
    - bnchandrapal 4y ago
      Out of curiosity, did you have any data lakes on S3? Did you find optimization techniques for the same?
      - 4y ago
ramraj07 4y ago
The biggest “hidden” cost for me was Elastic Blockstore IO charges! It started adding up quite quickly and I realized I had to think twice about doing IO intensive calculations on EC2! I switched over to lightsail (obviously these are for personal projects).
- mkesper 4y ago
  Just use gp3, shouldn't be needed to request higher IO with it (except for DBs, probably cheaper to use DynamoDB etc then).
- Bedon292 4y ago
  Yeah, provisioned IOPS can add up quick. Never had justification to spend on them myself. If I need IO I use instances with attached storage, otherwise I purely use the GP SSDs.
ransom1538 4y ago
Just make sure everything is multi-az that way if an entire zone goes down you will still be fine /s
psanford 4y ago
Its quite annoying that you can't disable parts of guardduty you don't get much value from. I think the CloudTrail monitoring is quite useful and the VPC flowlog monitoring is basically useless (to me, I have other means of doing host and network based monitoring, I'm sure that there are a lot of people who get meaningful value out of it). I'd like to be able to turn off flowlog monitoring and just use guardduty for cloudtrail monitoring, but that isn't an option. So I can either overpay for a bunch of extra things I don't get any value from or not enable guardduty at all.
JCM9 4y ago
I hear you but how is this any different than any alternative approaches? I mean yes you could throw a server under your desk and it would be cheaper. Should you run a server in prod with sensitive workloads without all these extra security bells and whistles (on prem or in the cloud)? No. No you should not. Compared to alternative security appliances and offerings this is still a good deal.
- Jensson 4y ago
  > Should you run a server in prod with sensitive workloads without all these extra security bells and whistles (on prem or in the cloud)? No. No you should not. Why not? Have people forgotten how to run servers in the past decade?
  - Spivak 4y ago
    People haven’t forgotten, companies just don’t hire infra people. “Fire all your ops people do the cloud” has made it so that organizations have forgotten how to run servers. Working in infra but not at supermassive scale does really feel like the new COBOL programmer.
Cthulhu_ 4y ago
Managing and occasionally slashing costs is why AWS consultants can make really good money - if you're into this kinda thing, it's worthwhile learning how to analyze and optimize AWS bills.
- Jensson 4y ago
  Well, before AWS big companies tended to have the same problem, but instead of paying for many AWS features they paid for a thousand different products and trying to make sense of all those bills for each product was hard. I know some people who worked as consultants to clean up all the services the company was paying for but didn't really need any longer, paying for around thousand different software products was the norm and not some unrealistically large number.
stunt 4y ago
how did you debug it? We have this weird thing where the cost of GuardDuty varies on different days of the month and stays the same from month to month. But we can't figure out what causes them. Can you actually see the events that GuardDuty has processed?

by m1keil 4y ago

deleted 4y ago

[deleted]

bushbaba 4y ago

Until you realize how much the alternatives cost. You think Palo Alto networks is cheap?

albert_e 4y ago
Is there a feature and price comparison between the native AWS offering and Palo Alto somewhere? Would be super handy for one of our projects right now. Thanks!
Bluecobra 4y ago
At least you can get a fixed cost if you go that route vs. your bills growing at an exponential rate. From what I have found, PA really wants you to commit to annual pricing with their VM appliances. If you go hourly you pay 73% more.
unethical_ban 4y ago
Do you think Palo Alto's offerings and GuardDuty are comparable?

philliphaydon 4y ago

Na. At some point you scale to the point that $400/m is peanuts for the benefits you get. We used it at previous job and realised we were under constant attack and and it reduced our cou usage by 15% in reduced requests for the amount of traffic we were getting. No more random spikes. Improved our overall security and ultimately reduced costs on the AWS bill. But if you’re gonna switch it on and walk away then you’re not really using it.

m1keil 4y ago
Yes of course, $400/month is peanuts so does $8000/month for certain companies. The point is not the absolute sum but how easy is it to spike your bill by 30 times. The way GD works, you are pretty much guaranteed to overpay for it sooner or later. If you can afford it.. great.
- philliphaydon 4y ago
  One thing that we noticed was after switching it on, the EC2 instances were being hit directly, so we moved those into a private security group only accessible to the load balancer. RDS got restricted. S3 buckets fixed. Coupled with AWF to block on inregular activity, resulted in GD bill going down, not up. This is no different from programming. PHP has some awful code out in the wild, it doesn't mean PHP is shit just because people write bad code. The issue with AWS is it's far too easy for people to just spin stuff up and it works and they don't look at what they are being billed for, don't analysis their infrastructure, don't optimize. They just throw servers, containers, etc up into the wild then when the bill comes: "OH AWS BAD I got billed cos I just set it up and forgot about it, then when it worked they charged me for it, AWS is wrong, just go baremetal."
  - bnchandrapal 4y ago
    Out of curiosity, did you have any data lakes on S3? Did you find optimization techniques for the same?
    - 4y ago

ramraj07 4y ago

The biggest “hidden” cost for me was Elastic Blockstore IO charges! It started adding up quite quickly and I realized I had to think twice about doing IO intensive calculations on EC2! I switched over to lightsail (obviously these are for personal projects).

mkesper 4y ago
Just use gp3, shouldn't be needed to request higher IO with it (except for DBs, probably cheaper to use DynamoDB etc then).
Bedon292 4y ago
Yeah, provisioned IOPS can add up quick. Never had justification to spend on them myself. If I need IO I use instances with attached storage, otherwise I purely use the GP SSDs.

ransom1538 4y ago

Just make sure everything is multi-az that way if an entire zone goes down you will still be fine /s

psanford 4y ago

Its quite annoying that you can't disable parts of guardduty you don't get much value from. I think the CloudTrail monitoring is quite useful and the VPC flowlog monitoring is basically useless (to me, I have other means of doing host and network based monitoring, I'm sure that there are a lot of people who get meaningful value out of it). I'd like to be able to turn off flowlog monitoring and just use guardduty for cloudtrail monitoring, but that isn't an option. So I can either overpay for a bunch of extra things I don't get any value from or not enable guardduty at all.

JCM9 4y ago

I hear you but how is this any different than any alternative approaches? I mean yes you could throw a server under your desk and it would be cheaper. Should you run a server in prod with sensitive workloads without all these extra security bells and whistles (on prem or in the cloud)? No. No you should not. Compared to alternative security appliances and offerings this is still a good deal.

Jensson 4y ago
> Should you run a server in prod with sensitive workloads without all these extra security bells and whistles (on prem or in the cloud)? No. No you should not. Why not? Have people forgotten how to run servers in the past decade?
- Spivak 4y ago
  People haven’t forgotten, companies just don’t hire infra people. “Fire all your ops people do the cloud” has made it so that organizations have forgotten how to run servers. Working in infra but not at supermassive scale does really feel like the new COBOL programmer.

Cthulhu_ 4y ago

Managing and occasionally slashing costs is why AWS consultants can make really good money - if you're into this kinda thing, it's worthwhile learning how to analyze and optimize AWS bills.

Jensson 4y ago
Well, before AWS big companies tended to have the same problem, but instead of paying for many AWS features they paid for a thousand different products and trying to make sense of all those bills for each product was hard. I know some people who worked as consultants to clean up all the services the company was paying for but didn't really need any longer, paying for around thousand different software products was the norm and not some unrealistically large number.

stunt 4y ago

how did you debug it? We have this weird thing where the cost of GuardDuty varies on different days of the month and stays the same from month to month. But we can't figure out what causes them. Can you actually see the events that GuardDuty has processed?