4 ms·

My r7g.16xl absolutely smokes my maxed out m2 max 16". I had to spin it up as I was sending my m2 into thermal shutdown. Task that takes 2.5 hours on the m2 max

My r7g.16xl absolutely smokes my maxed out m2 max 16". I had to spin it up as I was sending my m2 into thermal shutdown. Task that takes 2.5 hours on the m2 max takes 8 minutes on the r7g. I'm crunching about 16tb of log files streamed from s3 and collating to parquet. I even ran it from the office on Ethernet on Amazon's corp net and it was still much faster on CPU and net. I even tested it with everything locally on disk. Not even close. The r7g is cranking along at 5.2ghz all cores all day.

Nextgrid 3y ago
I don't doubt you can find an instance that smokes your average workstation. The question is how much more does it cost over just buying the equivalent machine outright. Is the markup reasonable, is it crazy, or is it an outright scam such as charging for egress data? Knowing AWS, it definitely won't be the first option, because why would it be if people are willing to pay more?
- dijit 3y ago
  Ran the numbers for you. On-Demand: $3.43/hr ($2503.90 monthly) Reserved: $2.27/hr ($1657.10 monthly) M3 Max 16" (all upgrades; to be generous): $7199.00 means about 3months on-demand or 4.5~ months reserved. Not nearly as bad as I was expecting to be perfectly honest with you. I did similar calculations recently and it ended up being 3 months for equivalent server grade hardware (IE; 256GiB ECC ram, 64 physical threads; 8TiB NVMe) -- so I bought the hardware and since then have saved more than 3x the cost. It seems not to be the case with Graviton instances I guess. :D
  - nathants 3y ago
    don’t forget spot pricing!
  - grogenaut 3y ago
    I work for an amazon subsidiary, I get hella discounts as well.
  - grogenaut 3y ago
    @dijit what server hardware did you get?
    - dijit 3y ago
      3 of these https://www.lenovo.com/us/en/p/servers-storage/servers/racks/thinksystem-sr645-v3/len21ts0008 https://www.lenovo.com/us/en/p/servers-storage/servers/racks... 4 units (its 4 machines in a single chassis) of this: https://www.gigabyte.com/Enterprise/High-Density-Server/H262-P60-rev-100 https://www.gigabyte.com/Enterprise/High-Density-Server/H262... all linked with a 10G switch: https://mikrotik.com/product/crs312_4c_8xg_rm https://mikrotik.com/product/crs312_4c_8xg_rm
- grogenaut 3y ago
  My mac cost like 7k with taxes. the R7G costs a lot less as I can turn it off and return it when i'm not using it. To do that processing job I'm paying less than a few bucks... less than that as I work for an company with aws discounts. At the rates I get for those machines it'd take several years to equal the 7k or so my mac cost when processing. And it's again absurdly faster. When you take into account actual streaming which is unfair as the data is in the same data center with the cloud instance, the job on the mac took 4 hours just to stream the data in and map it, and around 4 hours to reduce it. Both stages of that on the r7g were around 16 minutes. Again unfair as even though I have gigabit at home I'm still pulling 2-4x the throughput in the cloud instance from S3. My workloads aren't generally egressing tons of data from AWS, Im not running a file sharing service... that's not the majority of our costs. I'm not just bulk serving data from S3, I'm doing a lot of processing on data. anyway, I provided as close to I can on numbers. Take it or leave it on if you believe me.
cassianoleal 3y ago
That machine costs around US$2.5k/month. It gets expensive very quickly. In the early days of the cloud, part of the markup was justified because you wouldn't have to do maintenance, sell old to buy new, etc. At these rates though, I fail to see the appeal - at least from the point of view of cost per compute power.
- taylodl 3y ago
  Sys Admins and OS Image teams get expensive quickly. So does software licensing. When we moved to AWS we saved a fortune - and no, it wasn't in server costs.
  - Nextgrid 3y ago
    Every company I've been involved in had some full-time DevOps people wrangling YAML files. I'm not buying the argument that cloud providers reduce manpower demands. PaaS sure, but not cloud.
  - cassianoleal 3y ago
    Are you saying you don't need those because your compute is in the cloud? Who's operating those VMs? Who's hardening the OS images? What about software licensing? Whatever license is required to run a software on-prem is probably exactly the same to run it on an EC2 instance. Running in the cloud doesn't suddenly absolve you from having to pay licenses for the software. That would be piracy. This particular EC2 instance costs 2.5k/month. The MacBook in question costs what, 2x that? 3x? Let's exagerate and say it's 5x, so US$12.5k once. That machine is likely to last years in operation. Are you saying all those years at 2.5k/month for each instance is not enough to pay a team to operate them? Note that I'm not against doing things in the cloud. Many times, perhaps even most of the time, it's the right decision. That doesn't mean it's always the right one.
KaiserPro 3y ago
> My r7g.16xl absolutely smokes my maxed out m2 max 16" yes, because its apples(50-80watt TDP) to oranges(200-600watts). but also that's not the point. All those layers of microservices, or, pushing transactions to s3 slow everything down. instead of a beefy machine that takes a request and does virtually all of it locally, it need to jump out to either a local sidecar, or another service in the cluster. At best it'll be some message bus, at worse it'll be something over k8s batshit networking scheme. s3 is fine for streaming medium files, but as soon as you start modifying, boom, massive overhead. any change and you are re-writing the entire file. Than might be ok for your needs, but for others, its a pain in the balls. Lots of small files means that you are waiting for s3 io. S3 has improved significantly, and it can stream reasonably fast, but its not a _fast_ object store. now, of course, its got 30gbits a second network. great, but if you're wasitng that on writing out whole files for every single file change, you'll soon eat it up.
- grogenaut 3y ago
  The GP claimed the mac was more powerful than the cloud instance, which is what I responded to with an apples to apples equivalent. And yes it's Apples to oranges, which I'm agreeing with. Your point at the top is exactly what I'm getting at, a top end mac really can't compare to a much more powerful arm box with much more ram, much better network and much better cooling running a lot hotter. I'm not sure why you're jumping to some rant on k8s which I also dislike and don't use. The M2 Max is real fast compared to the intel macs and I live it cause it rarely runs into cooling issues as it is just a lot better. But you can make it heat up and when you do it tanks in speed to an equivalent percent as the intel boxes do. Especially when compared to a machine that's literally got a leaf blower running cooling to it.

Nextgrid 3y ago

I don't doubt you can find an instance that smokes your average workstation. The question is how much more does it cost over just buying the equivalent machine outright. Is the markup reasonable, is it crazy, or is it an outright scam such as charging for egress data? Knowing AWS, it definitely won't be the first option, because why would it be if people are willing to pay more?

dijit 3y ago
Ran the numbers for you. On-Demand: $3.43/hr ($2503.90 monthly) Reserved: $2.27/hr ($1657.10 monthly) M3 Max 16" (all upgrades; to be generous): $7199.00 means about 3months on-demand or 4.5~ months reserved. Not nearly as bad as I was expecting to be perfectly honest with you. I did similar calculations recently and it ended up being 3 months for equivalent server grade hardware (IE; 256GiB ECC ram, 64 physical threads; 8TiB NVMe) -- so I bought the hardware and since then have saved more than 3x the cost. It seems not to be the case with Graviton instances I guess. :D
- nathants 3y ago
  don’t forget spot pricing!
- grogenaut 3y ago
  I work for an amazon subsidiary, I get hella discounts as well.
- grogenaut 3y ago
  @dijit what server hardware did you get?
  - dijit 3y ago
    3 of these https://www.lenovo.com/us/en/p/servers-storage/servers/racks/thinksystem-sr645-v3/len21ts0008 https://www.lenovo.com/us/en/p/servers-storage/servers/racks... 4 units (its 4 machines in a single chassis) of this: https://www.gigabyte.com/Enterprise/High-Density-Server/H262-P60-rev-100 https://www.gigabyte.com/Enterprise/High-Density-Server/H262... all linked with a 10G switch: https://mikrotik.com/product/crs312_4c_8xg_rm https://mikrotik.com/product/crs312_4c_8xg_rm
grogenaut 3y ago
My mac cost like 7k with taxes. the R7G costs a lot less as I can turn it off and return it when i'm not using it. To do that processing job I'm paying less than a few bucks... less than that as I work for an company with aws discounts. At the rates I get for those machines it'd take several years to equal the 7k or so my mac cost when processing. And it's again absurdly faster. When you take into account actual streaming which is unfair as the data is in the same data center with the cloud instance, the job on the mac took 4 hours just to stream the data in and map it, and around 4 hours to reduce it. Both stages of that on the r7g were around 16 minutes. Again unfair as even though I have gigabit at home I'm still pulling 2-4x the throughput in the cloud instance from S3. My workloads aren't generally egressing tons of data from AWS, Im not running a file sharing service... that's not the majority of our costs. I'm not just bulk serving data from S3, I'm doing a lot of processing on data. anyway, I provided as close to I can on numbers. Take it or leave it on if you believe me.

cassianoleal 3y ago

That machine costs around US$2.5k/month. It gets expensive very quickly. In the early days of the cloud, part of the markup was justified because you wouldn't have to do maintenance, sell old to buy new, etc. At these rates though, I fail to see the appeal - at least from the point of view of cost per compute power.

taylodl 3y ago
Sys Admins and OS Image teams get expensive quickly. So does software licensing. When we moved to AWS we saved a fortune - and no, it wasn't in server costs.
- Nextgrid 3y ago
  Every company I've been involved in had some full-time DevOps people wrangling YAML files. I'm not buying the argument that cloud providers reduce manpower demands. PaaS sure, but not cloud.
- cassianoleal 3y ago
  Are you saying you don't need those because your compute is in the cloud? Who's operating those VMs? Who's hardening the OS images? What about software licensing? Whatever license is required to run a software on-prem is probably exactly the same to run it on an EC2 instance. Running in the cloud doesn't suddenly absolve you from having to pay licenses for the software. That would be piracy. This particular EC2 instance costs 2.5k/month. The MacBook in question costs what, 2x that? 3x? Let's exagerate and say it's 5x, so US$12.5k once. That machine is likely to last years in operation. Are you saying all those years at 2.5k/month for each instance is not enough to pay a team to operate them? Note that I'm not against doing things in the cloud. Many times, perhaps even most of the time, it's the right decision. That doesn't mean it's always the right one.

KaiserPro 3y ago

> My r7g.16xl absolutely smokes my maxed out m2 max 16" yes, because its apples(50-80watt TDP) to oranges(200-600watts). but also that's not the point. All those layers of microservices, or, pushing transactions to s3 slow everything down. instead of a beefy machine that takes a request and does virtually all of it locally, it need to jump out to either a local sidecar, or another service in the cluster. At best it'll be some message bus, at worse it'll be something over k8s batshit networking scheme. s3 is fine for streaming medium files, but as soon as you start modifying, boom, massive overhead. any change and you are re-writing the entire file. Than might be ok for your needs, but for others, its a pain in the balls. Lots of small files means that you are waiting for s3 io. S3 has improved significantly, and it can stream reasonably fast, but its not a _fast_ object store. now, of course, its got 30gbits a second network. great, but if you're wasitng that on writing out whole files for every single file change, you'll soon eat it up.

grogenaut 3y ago
The GP claimed the mac was more powerful than the cloud instance, which is what I responded to with an apples to apples equivalent. And yes it's Apples to oranges, which I'm agreeing with. Your point at the top is exactly what I'm getting at, a top end mac really can't compare to a much more powerful arm box with much more ram, much better network and much better cooling running a lot hotter. I'm not sure why you're jumping to some rant on k8s which I also dislike and don't use. The M2 Max is real fast compared to the intel macs and I live it cause it rarely runs into cooling issues as it is just a lot better. But you can make it heat up and when you do it tanks in speed to an equivalent percent as the intel boxes do. Especially when compared to a machine that's literally got a leaf blower running cooling to it.