3 ms·

H100s are almost-instantly available to anyone with a credit card and access to the internet. Without even having to lift their butt from the seat. And you get

H100s are almost-instantly available to anyone with a credit card and access to the internet. Without even having to lift their butt from the seat. And you get plenty more than five minutes of compute for the price of an M4.

jsperson 1y ago
For the orgs where I've worked the important thing isn't availability of compute it's security. Using what we have on our local network is much easier from a governance and approval standpoint than whatever is available on the internet.
- Sharlin 1y ago
  Many orgs have no problems using cloud envs for most things. The usual suspects offer just as secure compute envs as everything else. Anyway, I was assuming personal use, like the messing-around experimenting that the article is about. (Or who knows, maybe it was part of the author’s job.)
potatolicious 1y ago
And yet just about any intro-to-programming tutorial gets something running on your local machine, and local machine development continues to be the default for most people, even though devving on a cloud machine is eminently reasonable. "Pull out credit card, sign up for some thing and pay a bit of money" is a non-trivial bit of friction! Extremely non-trivial! Especially in a corporate context - you have to get the expense approved. It's not clear if you can put company data onto the machine. Whereas generally running local things on corporate laptops is far less controversial. "Download this tool and run it." is still an extremely powerful pitch. Pretty much the only thing that beats it is "go to this website which you can use without any signup or payment".
- Sharlin 1y ago
  Sure, if you already have said local machine. Which I guess in HN’s context many/most do.
ekianjo 1y ago
no org will let you send their data to a random online h100...
- Sharlin 1y ago
  Many orgs happily use Google’s everything. And Google offers secure compute envs just like it offers secure cloud everything. Anyway, I thought the context was doing stuff for personal use/fun, not work.
  - sethhochberg 1y ago
    Frankly I think a lot of full-time-employed technical people are largely experimenting for fun in the context of things that might eventually be useful to their employer. AI is cool and fascinating stuff and when I have a few idle minutes at the end of my workweek I love catching up and experimenting with the latest and greatest, but with an eye towards company problems and on company time, and sometimes using company datasets. That means company vendor approval and financing of my efforts. In my personal life, when its time for fun, I close the laptop and go do some gardening.
dekhn 1y ago
While I love cloud computing, you're comparing the cost of renting a GPU for a fixed amount of time to the purchase of an asset which can be used for years. Not a useful comparison IMHO.
- sudoshred 1y ago
  Disagree, equity of access matters a lot. Not everyone benefits from exposure to the entire hardware lifecycle, the same way that buying housing is not the best financial decision for everyone regardless of affordability. I might have unlimited budget but if I only need access to state of the art hardware intermittently or under irregular circumstances the cost of renting may be efficient for my needs. Also consider the costs of supporting hardware that is fully owned, if you own the hardware but underutilize it that is inefficiency and the owner bears that cost. The unusual way that silicon depreciates mean that the value of your “asset” is not static and rapidly depreciates as silicon manufacturing improves.
  - dekhn 1y ago
    Your argument is not related to my statement. You're arguing something else.
    - deleted 1y ago
      [deleted]
victorbjorklund 1y ago
I already have an M4 so the cost of running it is tiny.
0x457 1y ago
Yeah, is a large server rack to run those H100s. But realistically, the majority of people have a PC with consumer grade GPU or more likely a laptop with...laptop grade GPU. Cloud H100 don't count because you need lawyer to review ToS and other agreements.
kragen 1y ago
Aren't there export controls in place that exclude 90% of the people who have internet access, even if they do have credit cards?

jsperson 1y ago

For the orgs where I've worked the important thing isn't availability of compute it's security. Using what we have on our local network is much easier from a governance and approval standpoint than whatever is available on the internet.

Sharlin 1y ago
Many orgs have no problems using cloud envs for most things. The usual suspects offer just as secure compute envs as everything else. Anyway, I was assuming personal use, like the messing-around experimenting that the article is about. (Or who knows, maybe it was part of the author’s job.)

potatolicious 1y ago

And yet just about any intro-to-programming tutorial gets something running on your local machine, and local machine development continues to be the default for most people, even though devving on a cloud machine is eminently reasonable. "Pull out credit card, sign up for some thing and pay a bit of money" is a non-trivial bit of friction! Extremely non-trivial! Especially in a corporate context - you have to get the expense approved. It's not clear if you can put company data onto the machine. Whereas generally running local things on corporate laptops is far less controversial. "Download this tool and run it." is still an extremely powerful pitch. Pretty much the only thing that beats it is "go to this website which you can use without any signup or payment".

Sharlin 1y ago
Sure, if you already have said local machine. Which I guess in HN’s context many/most do.

ekianjo 1y ago

no org will let you send their data to a random online h100...

Sharlin 1y ago
Many orgs happily use Google’s everything. And Google offers secure compute envs just like it offers secure cloud everything. Anyway, I thought the context was doing stuff for personal use/fun, not work.
- sethhochberg 1y ago
  Frankly I think a lot of full-time-employed technical people are largely experimenting for fun in the context of things that might eventually be useful to their employer. AI is cool and fascinating stuff and when I have a few idle minutes at the end of my workweek I love catching up and experimenting with the latest and greatest, but with an eye towards company problems and on company time, and sometimes using company datasets. That means company vendor approval and financing of my efforts. In my personal life, when its time for fun, I close the laptop and go do some gardening.

dekhn 1y ago

While I love cloud computing, you're comparing the cost of renting a GPU for a fixed amount of time to the purchase of an asset which can be used for years. Not a useful comparison IMHO.

sudoshred 1y ago
Disagree, equity of access matters a lot. Not everyone benefits from exposure to the entire hardware lifecycle, the same way that buying housing is not the best financial decision for everyone regardless of affordability. I might have unlimited budget but if I only need access to state of the art hardware intermittently or under irregular circumstances the cost of renting may be efficient for my needs. Also consider the costs of supporting hardware that is fully owned, if you own the hardware but underutilize it that is inefficiency and the owner bears that cost. The unusual way that silicon depreciates mean that the value of your “asset” is not static and rapidly depreciates as silicon manufacturing improves.
- dekhn 1y ago
  Your argument is not related to my statement. You're arguing something else.
  - deleted 1y ago
    [deleted]

victorbjorklund 1y ago

I already have an M4 so the cost of running it is tiny.

0x457 1y ago

Yeah, is a large server rack to run those H100s. But realistically, the majority of people have a PC with consumer grade GPU or more likely a laptop with...laptop grade GPU. Cloud H100 don't count because you need lawyer to review ToS and other agreements.

kragen 1y ago

Aren't there export controls in place that exclude 90% of the people who have internet access, even if they do have credit cards?