5 ms·
I just costed a datadog deployment (based on your comment) and it would cost me my yearly salary every month. No thanks. :/
by dijit 6y ago
I just costed a datadog deployment (based on your comment) and it would cost me my yearly salary every month.
No thanks. :/
- rattray 6y agoCan you say something vague about your deployment scale?
- dijit 6y ago2,500~ compute instances. Looking at the pricing page the cost $15/instance so $37,500 Middle-of-the-road developer salary is like $35k in most of Europe, outside of the capitals. Although it does say: "Volume discounts available (500+ hosts/mo). Contact us." at the bottom, so I guess 500 is a lot.
- user5994461 6y ago2500 instances could be millions per months in AWS costs. The smallest instances with some disks and bandwidth fees can push 100k a month. Spending a fraction of that to monitor that sort of infrastructure is absolutely justified. I can tell you from experience that datadog gives discount even for 100+ hosts, I don't know what they can do for 2500, but if it were me I wouldn't accept anything less than 50% off. Honestly you need to forgot about your salary, it's irrelevant when it comes to running a company. Imagine a driver in a shipping company deciding to deliver on a scooter rather than a truck because the truck is worth more than his yearly salary.
- dijit 6y agoIt's not millions, but it's the many multiples of hundreds of thousands. (and it's mainly GCP/bare metal). I guess the point I am driving at here is that there's such a thing as "business critical costs" (IE: can we ship our product or not) which is the majority of infra costs we have today, and then there's "optimisation costs". Usually when we discuss things like optimisation costs its along the lines of: "Will this product save us enough time to justify it's expense". Often, sadly, the answer is no. Terraform Enterprise is an example of a time where we said: Yes. -- because the API allows us to deploy CI/CD jobs which provision little versions of our infrastructure, saving us many man-days of time in provisioning and testing every year. As eluded to in the sibling thread, there's almost no way that we can save 3 or more peoples worth of time every year, we're 3 people right now and we have metrics collection, log tracing and alerting already. -- so it's a hard sell to the business types.
- user5994461 6y agoGreat choice on GCP! It's probably half of the price as AWS for the same thing. Monitoring and logging are business critical. It's an integral part of infrastructure and it is very normal to spend 10% there. It's really not possible to operate stably and efficiently at a large scale like that without a trove of tooling. Tools usually justify their costs by allowing to optimize the infra and helping to prevent/fix outages, though not all companies care about stability or hardware costs. And it's not a choice of free vs paid. open source software costs a lot of money too, pairs of large instances to run it don't come cheap, they're probably more than a salary too if the company wants to have any sort of redundancy or geographic distribution. May I ask what do you have for logging? I guess you must be screaming in horror at the price of elasticsearch/kibana/splunk :D
- dijit 6y agoThe community versions of ELK, Zabbix, InfluxDB and grafana. Zabbix is the weak link here for sure, but the monitoring is quite comprehensive.
- sciurus 6y agoIt could also be less than $100k a month (e.g. 2500 c5a.large with a 1-year reservation). At that point you'd wonder why your monitoring bill was 40% of your compute bill. Also, of course their salary is relevant. The cost of an engineer's time is an important factor to consider when making build vs buy decisions. Usually it's one that argues in favor of "buy", but not always.
- user5994461 6y ago2500 c5a.large is $172k per month, or $102k with one year reservation full upfront. Add 10% support fees, EBS storage for the OS, bandwidth fees and it's quite a bit more.
- aprdm 6y agoOr at that point you can have a datacenter/colo and not spend anywhere near what you're thinking :)
- cs-szazz 6y agoWait, you're saying it costs 12x as much as your salary? That doesn't seem right... the company I'm at uses it pretty heavily and we're at about 1x of a FTE salary (and it's still super worth it)
- klohto 6y agoWorld’s salaries aren’t only US based :) Datadog for a smaller Western Europe startup is going to cost more than their devs salaries.
- user5994461 6y agoWestern salaries are not that low and true costs to the employer is typically double the perceived salary. That means we're talking tens of thousands of euros, so thousands of hosts (list price is certainly negotiable at this scale). If they've got a thousand of hosts, the costs of the infrastructure itself must dwarf the salary of any developer by orders of magnitude, the salary of a developer is simply irrelevant when it comes to acquiring software/hardware.
- dijit 6y ago> true costs to the employer is typically double the perceived salary. This is true, but my comment was an offhanded way to say that my "salary" (as in, the one on my contracts and the one I "see") is less than a month of Datadog for our number of hosts. As for the rest of your comment, I wish it was true. Developer salaries outside of the capitals is quite low in Europe, and even inside the capitals only go to "near double" So, instead of 12x it becomes 6x developer costs per annum, which is a fair whack of money. For me to justify spending "3-6" peoples worth of money it had better save "3-6" peoples worth of time.
- user5994461 6y ago> For me to justify spending "3-6" peoples worth of money it had better save "3-6" peoples worth of time. Well, it does in my experience, especially if you have to handle 2000+ hosts, that's some serious infra there, need serious tooling. May I ask which country is it?
- secondcoming 6y agoWe're moving from Datadog to Prometheus for this reason