5 ms·

According to the article you can break even after only two weeks which is far from "10k cookies every day for multiple years".

woolvalley 8y ago
Tech employees and maintenance is more expensive on the margin than cookie bakers
soneil 8y ago
I think the analogy holds. The numbers in the article assume constant usage (there is a nod to the cost of static storage, but all the math is done on $3/hr gpu usage, not $0.1/mo/GB storage). So if your usage is 4 hours per week, 50 weeks a year, you hit $600/year of gpu. If your usage is 24 hours a day, you hit $27k. So it's not a clear-cut "buying metal is cheaper" - you have to decide how your usage amortizes. As pgeorgi says, if your demand is a pack of cookies a week, buying a bakery isn't the most cost-efficient way to satisfy that demand.
- v_lisivka 8y ago
  For DL time is measured in years*GPU instead of hours.
  - dagw 8y ago
    Only for niche problems run into by huge companies and research groups. Most 'normal' problems encountered by 'normal' companies don't need anywhere near that much training.
- thisisjeffchen 8y ago
  Depends on your use case. The more accurate models these days require lots of data with huge models. Machine Translation, Image Generation, Segmentation, etc. Each experiment takes a couple days/weeks to train, so we were encountering long stretches of constant usage.
da_chicken 8y ago
It's fine for exactly what the article describes: a student doing projects at home. For anything bigger than that, you'll quickly run into issues. Try talking to your ops team and telling them you want to set up a mid-range desktop PC with extra RAM and a high end graphics card as a DL workstation. I think you'll very quickly find some friction, especially once you want to go into production with it.
- pythonaut_16 8y ago
  Google went to production with Pentium desktops on shelves. IIRC their original hardware plan was based around commodity hardware and getting the most performance per dollar since they built the crawler to be distributed pretty early on. Point being, consumer grade hardware has come a long way since then, and if you're doing something cutting edge like DL it's not outlandish to expect that rolling your own might be worthwhile and totally justified.
  - da_chicken 8y ago
    That is a horrible comparison. Google's start up between 1996 and 1998 was also over twenty years ago. There were fewer than 200 million Internet users on the Internet and less than 2.5 million web sites [0] in 1998. Google was also started by two college grad students, meaning gasp it was initially just a research project. I'd also point out that while, yes, the initial production servers were cheap and used commodity hardware, this [1] is what they looked like, which is hardly the type of setup that the article is suggesting. [0]: http://www.internetlivestats.com/total-number-of-websites/ http://www.internetlivestats.com/total-number-of-websites/ [1]: https://en.wikipedia.org/wiki/History_of_Google#/media/File:Google%E2%80%99s_First_Production_Server.jpg https://en.wikipedia.org/wiki/History_of_Google#/media/File:...
- mmt 8y ago
  And that's just unnecessary friction, which is a problem of politics, not technology. I do recall a time as recently as 4 years ago where it was difficult to find, for example, rack-mountable servers that could hold a large number of these GPU cards, but substituting for a mid-range PC with 2 or 4 of them would not have been an issue, even then. Cost would be higher but not necessarily outrageously so, since large server chasses often have startlingly large PSUs. It might be an extra 50% on top of that $3k, but that's still competitive, even with the pre-builds the article mentions.

woolvalley 8y ago

Tech employees and maintenance is more expensive on the margin than cookie bakers

soneil 8y ago

I think the analogy holds. The numbers in the article assume constant usage (there is a nod to the cost of static storage, but all the math is done on $3/hr gpu usage, not $0.1/mo/GB storage). So if your usage is 4 hours per week, 50 weeks a year, you hit $600/year of gpu. If your usage is 24 hours a day, you hit $27k. So it's not a clear-cut "buying metal is cheaper" - you have to decide how your usage amortizes. As pgeorgi says, if your demand is a pack of cookies a week, buying a bakery isn't the most cost-efficient way to satisfy that demand.

v_lisivka 8y ago
For DL time is measured in years*GPU instead of hours.
- dagw 8y ago
  Only for niche problems run into by huge companies and research groups. Most 'normal' problems encountered by 'normal' companies don't need anywhere near that much training.
thisisjeffchen 8y ago
Depends on your use case. The more accurate models these days require lots of data with huge models. Machine Translation, Image Generation, Segmentation, etc. Each experiment takes a couple days/weeks to train, so we were encountering long stretches of constant usage.

da_chicken 8y ago

It's fine for exactly what the article describes: a student doing projects at home. For anything bigger than that, you'll quickly run into issues. Try talking to your ops team and telling them you want to set up a mid-range desktop PC with extra RAM and a high end graphics card as a DL workstation. I think you'll very quickly find some friction, especially once you want to go into production with it.

pythonaut_16 8y ago
Google went to production with Pentium desktops on shelves. IIRC their original hardware plan was based around commodity hardware and getting the most performance per dollar since they built the crawler to be distributed pretty early on. Point being, consumer grade hardware has come a long way since then, and if you're doing something cutting edge like DL it's not outlandish to expect that rolling your own might be worthwhile and totally justified.
- da_chicken 8y ago
  That is a horrible comparison. Google's start up between 1996 and 1998 was also over twenty years ago. There were fewer than 200 million Internet users on the Internet and less than 2.5 million web sites [0] in 1998. Google was also started by two college grad students, meaning gasp it was initially just a research project. I'd also point out that while, yes, the initial production servers were cheap and used commodity hardware, this [1] is what they looked like, which is hardly the type of setup that the article is suggesting. [0]: http://www.internetlivestats.com/total-number-of-websites/ http://www.internetlivestats.com/total-number-of-websites/ [1]: https://en.wikipedia.org/wiki/History_of_Google#/media/File:Google%E2%80%99s_First_Production_Server.jpg https://en.wikipedia.org/wiki/History_of_Google#/media/File:...
mmt 8y ago
And that's just unnecessary friction, which is a problem of politics, not technology. I do recall a time as recently as 4 years ago where it was difficult to find, for example, rack-mountable servers that could hold a large number of these GPU cards, but substituting for a mid-range PC with 2 or 4 of them would not have been an issue, even then. Cost would be higher but not necessarily outrageously so, since large server chasses often have startlingly large PSUs. It might be an extra 50% on top of that $3k, but that's still competitive, even with the pre-builds the article mentions.