4 ms·

The cloud has eaten all the hardware improvements of the last decade. It's insane that my base model Macbook routinely outperforms the (expensive!) AWS crap tha

The cloud has eaten all the hardware improvements of the last decade. It's insane that my base model Macbook routinely outperforms the (expensive!) AWS crap that most companies use as their application "servers" and yet it's the reality in many places. So to a certain extent, you are forced to over-engineer for distributed workloads because any single node has the performance of a pocket calculator. This obviously benefits the cloud provider because not only do you rent more resources from them but you now rely on their management platform to orchestrate all those workloads.

deleted 3y ago
[deleted]
agilob 3y ago
Even more crazy that single AWS server class has 30% difference in performance between instances depending on performed tasks.
grogenaut 3y ago
My r7g.16xl absolutely smokes my maxed out m2 max 16". I had to spin it up as I was sending my m2 into thermal shutdown. Task that takes 2.5 hours on the m2 max takes 8 minutes on the r7g. I'm crunching about 16tb of log files streamed from s3 and collating to parquet. I even ran it from the office on Ethernet on Amazon's corp net and it was still much faster on CPU and net. I even tested it with everything locally on disk. Not even close. The r7g is cranking along at 5.2ghz all cores all day.
- Nextgrid 3y ago
  I don't doubt you can find an instance that smokes your average workstation. The question is how much more does it cost over just buying the equivalent machine outright. Is the markup reasonable, is it crazy, or is it an outright scam such as charging for egress data? Knowing AWS, it definitely won't be the first option, because why would it be if people are willing to pay more?
  - dijit 3y ago
    Ran the numbers for you. On-Demand: $3.43/hr ($2503.90 monthly) Reserved: $2.27/hr ($1657.10 monthly) M3 Max 16" (all upgrades; to be generous): $7199.00 means about 3months on-demand or 4.5~ months reserved. Not nearly as bad as I was expecting to be perfectly honest with you. I did similar calculations recently and it ended up being 3 months for equivalent server grade hardware (IE; 256GiB ECC ram, 64 physical threads; 8TiB NVMe) -- so I bought the hardware and since then have saved more than 3x the cost. It seems not to be the case with Graviton instances I guess. :D
    - nathants 3y ago
      don’t forget spot pricing!
    - grogenaut 3y ago
      I work for an amazon subsidiary, I get hella discounts as well.
    - grogenaut 3y ago
      @dijit what server hardware did you get?
      - dijit 3y ago
        3 of these https://www.lenovo.com/us/en/p/servers-storage/servers/racks/thinksystem-sr645-v3/len21ts0008 https://www.lenovo.com/us/en/p/servers-storage/servers/racks... 4 units (its 4 machines in a single chassis) of this: https://www.gigabyte.com/Enterprise/High-Density-Server/H262-P60-rev-100 https://www.gigabyte.com/Enterprise/High-Density-Server/H262... all linked with a 10G switch: https://mikrotik.com/product/crs312_4c_8xg_rm https://mikrotik.com/product/crs312_4c_8xg_rm
  - grogenaut 3y ago
    My mac cost like 7k with taxes. the R7G costs a lot less as I can turn it off and return it when i'm not using it. To do that processing job I'm paying less than a few bucks... less than that as I work for an company with aws discounts. At the rates I get for those machines it'd take several years to equal the 7k or so my mac cost when processing. And it's again absurdly faster. When you take into account actual streaming which is unfair as the data is in the same data center with the cloud instance, the job on the mac took 4 hours just to stream the data in and map it, and around 4 hours to reduce it. Both stages of that on the r7g were around 16 minutes. Again unfair as even though I have gigabit at home I'm still pulling 2-4x the throughput in the cloud instance from S3. My workloads aren't generally egressing tons of data from AWS, Im not running a file sharing service... that's not the majority of our costs. I'm not just bulk serving data from S3, I'm doing a lot of processing on data. anyway, I provided as close to I can on numbers. Take it or leave it on if you believe me.
- cassianoleal 3y ago
  That machine costs around US$2.5k/month. It gets expensive very quickly. In the early days of the cloud, part of the markup was justified because you wouldn't have to do maintenance, sell old to buy new, etc. At these rates though, I fail to see the appeal - at least from the point of view of cost per compute power.
  - taylodl 3y ago
    Sys Admins and OS Image teams get expensive quickly. So does software licensing. When we moved to AWS we saved a fortune - and no, it wasn't in server costs.
    - Nextgrid 3y ago
      Every company I've been involved in had some full-time DevOps people wrangling YAML files. I'm not buying the argument that cloud providers reduce manpower demands. PaaS sure, but not cloud.
    - cassianoleal 3y ago
      Are you saying you don't need those because your compute is in the cloud? Who's operating those VMs? Who's hardening the OS images? What about software licensing? Whatever license is required to run a software on-prem is probably exactly the same to run it on an EC2 instance. Running in the cloud doesn't suddenly absolve you from having to pay licenses for the software. That would be piracy. This particular EC2 instance costs 2.5k/month. The MacBook in question costs what, 2x that? 3x? Let's exagerate and say it's 5x, so US$12.5k once. That machine is likely to last years in operation. Are you saying all those years at 2.5k/month for each instance is not enough to pay a team to operate them? Note that I'm not against doing things in the cloud. Many times, perhaps even most of the time, it's the right decision. That doesn't mean it's always the right one.
- KaiserPro 3y ago
  > My r7g.16xl absolutely smokes my maxed out m2 max 16" yes, because its apples(50-80watt TDP) to oranges(200-600watts). but also that's not the point. All those layers of microservices, or, pushing transactions to s3 slow everything down. instead of a beefy machine that takes a request and does virtually all of it locally, it need to jump out to either a local sidecar, or another service in the cluster. At best it'll be some message bus, at worse it'll be something over k8s batshit networking scheme. s3 is fine for streaming medium files, but as soon as you start modifying, boom, massive overhead. any change and you are re-writing the entire file. Than might be ok for your needs, but for others, its a pain in the balls. Lots of small files means that you are waiting for s3 io. S3 has improved significantly, and it can stream reasonably fast, but its not a _fast_ object store. now, of course, its got 30gbits a second network. great, but if you're wasitng that on writing out whole files for every single file change, you'll soon eat it up.
  - grogenaut 3y ago
    The GP claimed the mac was more powerful than the cloud instance, which is what I responded to with an apples to apples equivalent. And yes it's Apples to oranges, which I'm agreeing with. Your point at the top is exactly what I'm getting at, a top end mac really can't compare to a much more powerful arm box with much more ram, much better network and much better cooling running a lot hotter. I'm not sure why you're jumping to some rant on k8s which I also dislike and don't use. The M2 Max is real fast compared to the intel macs and I live it cause it rarely runs into cooling issues as it is just a lot better. But you can make it heat up and when you do it tanks in speed to an equivalent percent as the intel boxes do. Especially when compared to a machine that's literally got a leaf blower running cooling to it.
BackBlast 3y ago
The cloud was principally designed 2005-2013 era where it was pretty reasonable for an average website to need to go multi system to support traffic spikes. CPU hardware kind of stagnated 2014-2018, while SSDs took off. But the last 5 years we've seen some pretty massive improvements everywhere. How many sites need more than 500 threads, 6TB of ram, and 100Gbit egress? Which you can get in a single system now. Some do, certainly, but then we're talking about the tiny narrow edge on the upper end of the curve. Almost every new web system built today can fit on a single box, and can become massively profitable on that single box unless you unicorn out and need the new architecture which you can now afford. And... that should be the new default. But... with the cloud, we're stuck with complex massively parallel services or people starting up on tiny rented instances that are, as you say, pocket calculator equivalents.
- PH95VuimJjqBqy 3y ago
  yep, at most you might want 2 boxes for HA but you definitely don't have to start with that and it also depends on what you're doing. I know some are going to argue that it's a slippery slope from that simple HA to DR and all the other complexities, and there's some truth in that, but that complexity can be controlled with explicit decision making. It's not obvious that a simple HA/DR story has to end up with modern day cloud architectures.
RhysU 3y ago
Aye. I love the following three-step solution for compute workloads: 1) Make something right. 2) Reduce RAM requirements. 3) Apply GNU Parallel. It's magical how just much batch work can be done one manycore system when one keeps it hot.

deleted 3y ago

[deleted]

agilob 3y ago

Even more crazy that single AWS server class has 30% difference in performance between instances depending on performed tasks.

grogenaut 3y ago

My r7g.16xl absolutely smokes my maxed out m2 max 16". I had to spin it up as I was sending my m2 into thermal shutdown. Task that takes 2.5 hours on the m2 max takes 8 minutes on the r7g. I'm crunching about 16tb of log files streamed from s3 and collating to parquet. I even ran it from the office on Ethernet on Amazon's corp net and it was still much faster on CPU and net. I even tested it with everything locally on disk. Not even close. The r7g is cranking along at 5.2ghz all cores all day.

Nextgrid 3y ago
I don't doubt you can find an instance that smokes your average workstation. The question is how much more does it cost over just buying the equivalent machine outright. Is the markup reasonable, is it crazy, or is it an outright scam such as charging for egress data? Knowing AWS, it definitely won't be the first option, because why would it be if people are willing to pay more?
- dijit 3y ago
  Ran the numbers for you. On-Demand: $3.43/hr ($2503.90 monthly) Reserved: $2.27/hr ($1657.10 monthly) M3 Max 16" (all upgrades; to be generous): $7199.00 means about 3months on-demand or 4.5~ months reserved. Not nearly as bad as I was expecting to be perfectly honest with you. I did similar calculations recently and it ended up being 3 months for equivalent server grade hardware (IE; 256GiB ECC ram, 64 physical threads; 8TiB NVMe) -- so I bought the hardware and since then have saved more than 3x the cost. It seems not to be the case with Graviton instances I guess. :D
  - nathants 3y ago
    don’t forget spot pricing!
  - grogenaut 3y ago
    I work for an amazon subsidiary, I get hella discounts as well.
  - grogenaut 3y ago
    @dijit what server hardware did you get?
    - dijit 3y ago
      3 of these https://www.lenovo.com/us/en/p/servers-storage/servers/racks/thinksystem-sr645-v3/len21ts0008 https://www.lenovo.com/us/en/p/servers-storage/servers/racks... 4 units (its 4 machines in a single chassis) of this: https://www.gigabyte.com/Enterprise/High-Density-Server/H262-P60-rev-100 https://www.gigabyte.com/Enterprise/High-Density-Server/H262... all linked with a 10G switch: https://mikrotik.com/product/crs312_4c_8xg_rm https://mikrotik.com/product/crs312_4c_8xg_rm
- grogenaut 3y ago
  My mac cost like 7k with taxes. the R7G costs a lot less as I can turn it off and return it when i'm not using it. To do that processing job I'm paying less than a few bucks... less than that as I work for an company with aws discounts. At the rates I get for those machines it'd take several years to equal the 7k or so my mac cost when processing. And it's again absurdly faster. When you take into account actual streaming which is unfair as the data is in the same data center with the cloud instance, the job on the mac took 4 hours just to stream the data in and map it, and around 4 hours to reduce it. Both stages of that on the r7g were around 16 minutes. Again unfair as even though I have gigabit at home I'm still pulling 2-4x the throughput in the cloud instance from S3. My workloads aren't generally egressing tons of data from AWS, Im not running a file sharing service... that's not the majority of our costs. I'm not just bulk serving data from S3, I'm doing a lot of processing on data. anyway, I provided as close to I can on numbers. Take it or leave it on if you believe me.
cassianoleal 3y ago
That machine costs around US$2.5k/month. It gets expensive very quickly. In the early days of the cloud, part of the markup was justified because you wouldn't have to do maintenance, sell old to buy new, etc. At these rates though, I fail to see the appeal - at least from the point of view of cost per compute power.
- taylodl 3y ago
  Sys Admins and OS Image teams get expensive quickly. So does software licensing. When we moved to AWS we saved a fortune - and no, it wasn't in server costs.
  - Nextgrid 3y ago
    Every company I've been involved in had some full-time DevOps people wrangling YAML files. I'm not buying the argument that cloud providers reduce manpower demands. PaaS sure, but not cloud.
  - cassianoleal 3y ago
    Are you saying you don't need those because your compute is in the cloud? Who's operating those VMs? Who's hardening the OS images? What about software licensing? Whatever license is required to run a software on-prem is probably exactly the same to run it on an EC2 instance. Running in the cloud doesn't suddenly absolve you from having to pay licenses for the software. That would be piracy. This particular EC2 instance costs 2.5k/month. The MacBook in question costs what, 2x that? 3x? Let's exagerate and say it's 5x, so US$12.5k once. That machine is likely to last years in operation. Are you saying all those years at 2.5k/month for each instance is not enough to pay a team to operate them? Note that I'm not against doing things in the cloud. Many times, perhaps even most of the time, it's the right decision. That doesn't mean it's always the right one.
KaiserPro 3y ago
> My r7g.16xl absolutely smokes my maxed out m2 max 16" yes, because its apples(50-80watt TDP) to oranges(200-600watts). but also that's not the point. All those layers of microservices, or, pushing transactions to s3 slow everything down. instead of a beefy machine that takes a request and does virtually all of it locally, it need to jump out to either a local sidecar, or another service in the cluster. At best it'll be some message bus, at worse it'll be something over k8s batshit networking scheme. s3 is fine for streaming medium files, but as soon as you start modifying, boom, massive overhead. any change and you are re-writing the entire file. Than might be ok for your needs, but for others, its a pain in the balls. Lots of small files means that you are waiting for s3 io. S3 has improved significantly, and it can stream reasonably fast, but its not a _fast_ object store. now, of course, its got 30gbits a second network. great, but if you're wasitng that on writing out whole files for every single file change, you'll soon eat it up.
- grogenaut 3y ago
  The GP claimed the mac was more powerful than the cloud instance, which is what I responded to with an apples to apples equivalent. And yes it's Apples to oranges, which I'm agreeing with. Your point at the top is exactly what I'm getting at, a top end mac really can't compare to a much more powerful arm box with much more ram, much better network and much better cooling running a lot hotter. I'm not sure why you're jumping to some rant on k8s which I also dislike and don't use. The M2 Max is real fast compared to the intel macs and I live it cause it rarely runs into cooling issues as it is just a lot better. But you can make it heat up and when you do it tanks in speed to an equivalent percent as the intel boxes do. Especially when compared to a machine that's literally got a leaf blower running cooling to it.

BackBlast 3y ago

The cloud was principally designed 2005-2013 era where it was pretty reasonable for an average website to need to go multi system to support traffic spikes. CPU hardware kind of stagnated 2014-2018, while SSDs took off. But the last 5 years we've seen some pretty massive improvements everywhere. How many sites need more than 500 threads, 6TB of ram, and 100Gbit egress? Which you can get in a single system now. Some do, certainly, but then we're talking about the tiny narrow edge on the upper end of the curve. Almost every new web system built today can fit on a single box, and can become massively profitable on that single box unless you unicorn out and need the new architecture which you can now afford. And... that should be the new default. But... with the cloud, we're stuck with complex massively parallel services or people starting up on tiny rented instances that are, as you say, pocket calculator equivalents.

PH95VuimJjqBqy 3y ago
yep, at most you might want 2 boxes for HA but you definitely don't have to start with that and it also depends on what you're doing. I know some are going to argue that it's a slippery slope from that simple HA to DR and all the other complexities, and there's some truth in that, but that complexity can be controlled with explicit decision making. It's not obvious that a simple HA/DR story has to end up with modern day cloud architectures.

RhysU 3y ago

Aye. I love the following three-step solution for compute workloads: 1) Make something right. 2) Reduce RAM requirements. 3) Apply GNU Parallel. It's magical how just much batch work can be done one manycore system when one keeps it hot.