5 ms·

I hope one day it will be possible to run this kind of models at home.

Akronymus 4y ago
Well, it used to be impossible to render on anything not a mainframe in a reasonable time. The day will come when we will be able to.
Hendrikto 4y ago
If you live in a datacenter, it already is!
- alexpotato 4y ago
  You already have access to thousands of machine now from your home computer. Naval Ravikant put it best here: https://twitter.com/naval/status/1002106977273565184 https://twitter.com/naval/status/1002106977273565184
lannisterstark 4y ago
I was about to comment exactly the same thing. Stuff like this makes me feel so much behind because there's no way I can run this lol.
- f6v 4y ago
  They hardware they mention can be rented from cloud providers. It’s just that it’s not very cheap.
rocgf 4y ago
When it will be possible to run this at home, the big companies will have models way bigger than this...
- rapnie 4y ago
  Or maybe the AI will own big companies that build bigger models for it. /s
albertzeyer 4y ago
If your disk has enough space to store the model, I think in theory you could run them, using the disk to store states. But it will be slow. I'm not sure how slow though, and also if anyone has implemented this. It actually should not be too difficult.
- redox99 4y ago
  Disk makes no sense considering RAM is pretty cheap. But even then RAM is way too slow (and the communication overhead way too high). You probably get like a 100x slowdown or more.
  - lostmsu 4y ago
    I think you are overestimating compute and I/O for this model. If you assume it is RAM bandwidth bound, with a single channel top DDR4 you will get inference time as a low multiple of 7 seconds (200GB/25GBs). In a workstation you can have 8 channels.
    - justinlloyd 4y ago
      12-channels in mine. 24-channels on some configurations, though I think that is the upper limit at this time, with a maximum density of 512GB per channel.
      - lostmsu 4y ago
        Is it multisocket?
irthomasthomas 4y ago
I think that unlikely. Barring some breakthrough that takes us beyond the limits of silicon.
- haswell 4y ago
  Couldn’t the same thing be said about most things we do on our phones these days? Won’t incremental advancement cover this eventually? (i.e. no major breakthrough required, just patience).
cal85 4y ago
Speaking of which... I built a gaming PC a few years ago but I never use it these days. I want to install Linux on it and start playing around with machine learning. Can anyone recommend any open source machine learning project that would be a good starting point? I want one that does something interesting (whether using text, images, whatever), but simple/efficient enough to run on a gaming PC and see some kind of results in hours, not months. I'm not sure what I want to do with ML yet, I just know I'm interested, and getting something up and running is likely to enthuse me to start playing and researching further. My spec is: GeForce RTX 2080 Ti (11GB), a 24-core AMD Ryzen Threadripper, and 128GB RAM. I'd be willing to spend on a new graphics card if it would make all the difference. I am a competent coder and familiar with Python but my experience with ML is limited to fawning over things on HN. Any recommendations gratefully received!
- schizo89 4y ago
  I would recommend auditing Stanford courses in following order: 1. CS231n Machine Vision https://www.youtube.com/playlist?list=PLkt2uSq6rBVctENoVBg1TpCC7OQi31AlC https://www.youtube.com/playlist?list=PLkt2uSq6rBVctENoVBg1T... 2. CS234 Reinforcement Learning https://www.youtube.com/watch?v=FgzM3zpZ55o&list=PLoROMvodv4rOSOPzutgyCTapiGlY2Nd8u https://www.youtube.com/watch?v=FgzM3zpZ55o&list=PLoROMvodv4... 3. CS330 Meta Learning https://www.youtube.com/watch?v=0rZtSwNOTQo&list=PLoROMvodv4rMC6zfYmnD7UG3LVvwaITY5 https://www.youtube.com/watch?v=0rZtSwNOTQo&list=PLoROMvodv4... Those will get you on track with general concepts about reasoning, AI engineering and concepts of learning itself Language models for me a bit of headache because there're in different domain on intersection with linguistics and humanities but here's a good course https://www.youtube.com/watch?v=rha64cQRLs8&list=PLoROMvodv4rPt5D0zs3YhbWSZA8Q_DyiJ https://www.youtube.com/watch?v=rha64cQRLs8&list=PLoROMvodv4... Those are all free and high-quality but require a lot of brain power

Well, it used to be impossible to render on anything not a mainframe in a reasonable time. The day will come when we will be able to.

Hendrikto 4y ago

If you live in a datacenter, it already is!

alexpotato 4y ago
You already have access to thousands of machine now from your home computer. Naval Ravikant put it best here: https://twitter.com/naval/status/1002106977273565184 https://twitter.com/naval/status/1002106977273565184

lannisterstark 4y ago

I was about to comment exactly the same thing. Stuff like this makes me feel so much behind because there's no way I can run this lol.

f6v 4y ago
They hardware they mention can be rented from cloud providers. It’s just that it’s not very cheap.

rocgf 4y ago

When it will be possible to run this at home, the big companies will have models way bigger than this...

rapnie 4y ago
Or maybe the AI will own big companies that build bigger models for it. /s

albertzeyer 4y ago

If your disk has enough space to store the model, I think in theory you could run them, using the disk to store states. But it will be slow. I'm not sure how slow though, and also if anyone has implemented this. It actually should not be too difficult.

redox99 4y ago
Disk makes no sense considering RAM is pretty cheap. But even then RAM is way too slow (and the communication overhead way too high). You probably get like a 100x slowdown or more.
- lostmsu 4y ago
  I think you are overestimating compute and I/O for this model. If you assume it is RAM bandwidth bound, with a single channel top DDR4 you will get inference time as a low multiple of 7 seconds (200GB/25GBs). In a workstation you can have 8 channels.
  - justinlloyd 4y ago
    12-channels in mine. 24-channels on some configurations, though I think that is the upper limit at this time, with a maximum density of 512GB per channel.
    - lostmsu 4y ago
      Is it multisocket?

irthomasthomas 4y ago

I think that unlikely. Barring some breakthrough that takes us beyond the limits of silicon.

haswell 4y ago
Couldn’t the same thing be said about most things we do on our phones these days? Won’t incremental advancement cover this eventually? (i.e. no major breakthrough required, just patience).

cal85 4y ago

Speaking of which... I built a gaming PC a few years ago but I never use it these days. I want to install Linux on it and start playing around with machine learning. Can anyone recommend any open source machine learning project that would be a good starting point? I want one that does something interesting (whether using text, images, whatever), but simple/efficient enough to run on a gaming PC and see some kind of results in hours, not months. I'm not sure what I want to do with ML yet, I just know I'm interested, and getting something up and running is likely to enthuse me to start playing and researching further. My spec is: GeForce RTX 2080 Ti (11GB), a 24-core AMD Ryzen Threadripper, and 128GB RAM. I'd be willing to spend on a new graphics card if it would make all the difference. I am a competent coder and familiar with Python but my experience with ML is limited to fawning over things on HN. Any recommendations gratefully received!

schizo89 4y ago
I would recommend auditing Stanford courses in following order: 1. CS231n Machine Vision https://www.youtube.com/playlist?list=PLkt2uSq6rBVctENoVBg1TpCC7OQi31AlC https://www.youtube.com/playlist?list=PLkt2uSq6rBVctENoVBg1T... 2. CS234 Reinforcement Learning https://www.youtube.com/watch?v=FgzM3zpZ55o&list=PLoROMvodv4rOSOPzutgyCTapiGlY2Nd8u https://www.youtube.com/watch?v=FgzM3zpZ55o&list=PLoROMvodv4... 3. CS330 Meta Learning https://www.youtube.com/watch?v=0rZtSwNOTQo&list=PLoROMvodv4rMC6zfYmnD7UG3LVvwaITY5 https://www.youtube.com/watch?v=0rZtSwNOTQo&list=PLoROMvodv4... Those will get you on track with general concepts about reasoning, AI engineering and concepts of learning itself Language models for me a bit of headache because there're in different domain on intersection with linguistics and humanities but here's a good course https://www.youtube.com/watch?v=rha64cQRLs8&list=PLoROMvodv4rPt5D0zs3YhbWSZA8Q_DyiJ https://www.youtube.com/watch?v=rha64cQRLs8&list=PLoROMvodv4... Those are all free and high-quality but require a lot of brain power