19 ms·

> I found the AWS vs GPU cost comparison to be fun. 10x the cost of the hardware to run the collisions in the same time window. Crazy. The real benefit of the

> I found the AWS vs GPU cost comparison to be fun. 10x the cost of the hardware to run the collisions in the same time window. Crazy. The real benefit of the AWS solution is that you could rent a large number of AWS nodes and run the calculations to completion right now if you wanted to. As in, literally tonight. And you wouldn’t have to source the hardware, assemble machines, install software, power it all, cool it, figure out all of the power complexities and so on.

systemvoltage 5y ago
I expect some "Is this Bitcoin mining?" flags to go off at AWS. So realistically, I would say, 2-3 business days to get things going properly.
- losteric 5y ago
  Does AWS ban Bitcoin mining? Why would they care as long as the bills get paid
  - nemothekid 5y ago
    >Why would they care as long as the bills get paid I imagine it's the cause of several support tickets along the lines of "I got hacked and racked up a $50,000 bill because of bitcoin mining"
  - nerdbaggy 5y ago
    They don’t ban it, but new accounts or even semi old accounts will have hard time getting access to more than a few instances. Most of the GPU machines are behind lock and key that go toward AWS bigger clients and clients with a proven track record. They don’t want to be stuck with an unpaid ec2 bill for a multi thousand dollar a month vm.
    - rkagerer 5y ago
      Easy fix, give the new, reputationless customers the chance to pay in advance.
      - abofh 5y ago
        Reserved instances accomplish basically that.
      - trillic 5y ago
        You can, I did, just email your support rep or the generic support line if you don’t have a good contact yet.
      - eru 5y ago
        You still have to make sure that the money you got was clean. Legally, even money you have in your account now might not stay there, if there was fraud involved. But the mined coins will be gone (and the compute time, too).
  - usr1106 5y ago
    With AWS prices that must be burning money (if you honestly pay the AWS bill). I didn't do the math, just my feeling.
  - tinus_hn 5y ago
    It’s a magnet for credit card fraud: take a stolen card and create an account, run the mining until the owner gets the bill and does the chargeback. You are untraceable and Amazon is left holding the bag.
- deleted 5y ago
  [deleted]
dannyw 5y ago
That in itself doesn't explain the 10x though. You can insta deploy your VM on Vast.ai for 1/8th the cost. Literally tonight. You won't have to source the hardware, assemble the machines, etc. People also overestimate how much effort it is to manage hardware, especially for raw compute (ie 99% SLA fine) operations. The cooling and power is handled by your datacentre. You don't need to deal with that. I've built a 200 GPU mining operation in one workweek, pay me $170k?
- H8crilA 5y ago
  I think the 99% SLA vs a few more nines can be the deal breaker. And you probably didn't even get a full 99% - with all due respect, as you likely didn't need it in the first place for a mining rig. I know plenty of serious workloads that would accept as little as 90% availability (~1 full day down every ~10 days), and plenty that need a large amount of 9s (seconds, maybe a minute of downtime per month on a normal basis, everything more is an outage).
  - semi-extrinsic 5y ago
    I don't know of any HPC facility that aims for/achieves anything beyond 99% uptime, most well-run ones are probably 98%-ish. And that is just raw uptime, as an end-user you also have to live with you jobs waiting for the scheduler to fit them into the queue. A day or two waiting time is pretty common on larger jobs.
    - H8crilA 5y ago
      Yup, exactly. But datacenters handling live user traffic aim for much much more than 98%.
b112 5y ago
Right. AWS is for those unable to budget, pre-plan, prepare.
banana_giraffe 5y ago
Given how often I've seen > There is no runtime available right now. Please change the compute type or try again later. recently, that's not always true. It's always funny to me when you run into a wall of "no more instances" in AWS. It doesn't happen often unless you're at a massive scale, but it does happen surprisingly often with the GPU types
- londons_explore 5y ago
  Happens with Google cloud too, sometimes for weeks at a time. It's pretty sad how Google/Amazon can't at least temporarily move their own workloads to free up resources for their public clouds while they install new hardware.
  - H8crilA 5y ago
    > free up resources (...) while they install new hardware That's likely not the reason. It's more likely caused by inherent spike-y ness of the traffic, given that most GPU instances are probably used for batch numerical computations (like training neural networks). The pricing incentivizes using a lot of nodes for a shorter amount of time, much like in the SHA example.
    - londons_explore 5y ago
      The pricing incentivises spot nodes, which use dynamic pricing to try to prevent this...
  - dspillett 5y ago
    They are never going to significantly prioritise “public” cloud resource over internal needs, beyond own needs having quotas to make sure they don't run amok if something gets unexpectedly out of shape. If their own public services slowed down in high demand times, it wouldn't look good. The sales channels could try to spin it as “giving you priority over us” but who would genuinely believe them. They could reduce resources available to low priority internal stuff during spikes of other activity, but that would not have a large effect considering the combined size of activity on the public cloud resources.
nunez 5y ago
also, using a spot fleet can bring that price down significantly
herpderperator 5y ago
And the other benefit is undoing it all just as quickly.
jgalt212 5y ago
The real benefit of the in-house solution is that marginal compute costs are zero. And while this current example is just for fun, I do want my researchers and engineers not worried about the compute cost of every experiment or line of inquiry. I dig the cloud for production, but for R&D I like my people to have fixed costs and close to zero marginal costs.
- eru 5y ago
  Electricity costs real money. But you are right that the marginal costs will be mostly made up of opportunity costs. Unless you have only one researcher, it doesn't matter for an individual researcher whether their marginal costs are opportunity costs of their co-workers or AWS bills. (And if you don't have any mechanism for your individual researchers to feel the opportunity costs of their co-workers; that is the same thing as saying that one researcher can hog up all the compute time. Social pressure is also a way to make them feel opportunity costs.)

systemvoltage 5y ago

I expect some "Is this Bitcoin mining?" flags to go off at AWS. So realistically, I would say, 2-3 business days to get things going properly.

losteric 5y ago
Does AWS ban Bitcoin mining? Why would they care as long as the bills get paid
- nemothekid 5y ago
  >Why would they care as long as the bills get paid I imagine it's the cause of several support tickets along the lines of "I got hacked and racked up a $50,000 bill because of bitcoin mining"
- nerdbaggy 5y ago
  They don’t ban it, but new accounts or even semi old accounts will have hard time getting access to more than a few instances. Most of the GPU machines are behind lock and key that go toward AWS bigger clients and clients with a proven track record. They don’t want to be stuck with an unpaid ec2 bill for a multi thousand dollar a month vm.
  - rkagerer 5y ago
    Easy fix, give the new, reputationless customers the chance to pay in advance.
    - abofh 5y ago
      Reserved instances accomplish basically that.
    - trillic 5y ago
      You can, I did, just email your support rep or the generic support line if you don’t have a good contact yet.
    - eru 5y ago
      You still have to make sure that the money you got was clean. Legally, even money you have in your account now might not stay there, if there was fraud involved. But the mined coins will be gone (and the compute time, too).
- usr1106 5y ago
  With AWS prices that must be burning money (if you honestly pay the AWS bill). I didn't do the math, just my feeling.
- tinus_hn 5y ago
  It’s a magnet for credit card fraud: take a stolen card and create an account, run the mining until the owner gets the bill and does the chargeback. You are untraceable and Amazon is left holding the bag.
deleted 5y ago
[deleted]

dannyw 5y ago

That in itself doesn't explain the 10x though. You can insta deploy your VM on Vast.ai for 1/8th the cost. Literally tonight. You won't have to source the hardware, assemble the machines, etc. People also overestimate how much effort it is to manage hardware, especially for raw compute (ie 99% SLA fine) operations. The cooling and power is handled by your datacentre. You don't need to deal with that. I've built a 200 GPU mining operation in one workweek, pay me $170k?

H8crilA 5y ago
I think the 99% SLA vs a few more nines can be the deal breaker. And you probably didn't even get a full 99% - with all due respect, as you likely didn't need it in the first place for a mining rig. I know plenty of serious workloads that would accept as little as 90% availability (~1 full day down every ~10 days), and plenty that need a large amount of 9s (seconds, maybe a minute of downtime per month on a normal basis, everything more is an outage).
- semi-extrinsic 5y ago
  I don't know of any HPC facility that aims for/achieves anything beyond 99% uptime, most well-run ones are probably 98%-ish. And that is just raw uptime, as an end-user you also have to live with you jobs waiting for the scheduler to fit them into the queue. A day or two waiting time is pretty common on larger jobs.
  - H8crilA 5y ago
    Yup, exactly. But datacenters handling live user traffic aim for much much more than 98%.

b112 5y ago

Right. AWS is for those unable to budget, pre-plan, prepare.

banana_giraffe 5y ago

Given how often I've seen > There is no runtime available right now. Please change the compute type or try again later. recently, that's not always true. It's always funny to me when you run into a wall of "no more instances" in AWS. It doesn't happen often unless you're at a massive scale, but it does happen surprisingly often with the GPU types

londons_explore 5y ago
Happens with Google cloud too, sometimes for weeks at a time. It's pretty sad how Google/Amazon can't at least temporarily move their own workloads to free up resources for their public clouds while they install new hardware.
- H8crilA 5y ago
  > free up resources (...) while they install new hardware That's likely not the reason. It's more likely caused by inherent spike-y ness of the traffic, given that most GPU instances are probably used for batch numerical computations (like training neural networks). The pricing incentivizes using a lot of nodes for a shorter amount of time, much like in the SHA example.
  - londons_explore 5y ago
    The pricing incentivises spot nodes, which use dynamic pricing to try to prevent this...
- dspillett 5y ago
  They are never going to significantly prioritise “public” cloud resource over internal needs, beyond own needs having quotas to make sure they don't run amok if something gets unexpectedly out of shape. If their own public services slowed down in high demand times, it wouldn't look good. The sales channels could try to spin it as “giving you priority over us” but who would genuinely believe them. They could reduce resources available to low priority internal stuff during spikes of other activity, but that would not have a large effect considering the combined size of activity on the public cloud resources.

nunez 5y ago

also, using a spot fleet can bring that price down significantly

herpderperator 5y ago

And the other benefit is undoing it all just as quickly.

jgalt212 5y ago

The real benefit of the in-house solution is that marginal compute costs are zero. And while this current example is just for fun, I do want my researchers and engineers not worried about the compute cost of every experiment or line of inquiry. I dig the cloud for production, but for R&D I like my people to have fixed costs and close to zero marginal costs.

eru 5y ago
Electricity costs real money. But you are right that the marginal costs will be mostly made up of opportunity costs. Unless you have only one researcher, it doesn't matter for an individual researcher whether their marginal costs are opportunity costs of their co-workers or AWS bills. (And if you don't have any mechanism for your individual researchers to feel the opportunity costs of their co-workers; that is the same thing as saying that one researcher can hog up all the compute time. Social pressure is also a way to make them feel opportunity costs.)