6 ms·

Yup, thank the hobbyists. Pytorch is allowing other hardware. Stable diffusion working on m chips, intel arc, and Amd. Now what I'd like to see is real benchma

Yup, thank the hobbyists. Pytorch is allowing other hardware. Stable diffusion working on m chips, intel arc, and Amd. Now what I'd like to see is real benchmarks for compute power. Might even get a few startups to compete in this new area.

mattnewton 3y ago
Re: startups, Geohotz raised a few million for this already. https://tinygrad.org/ https://tinygrad.org/
- IntelMiner 3y ago
  Didn't he do what he always does. Rake in a ton of money, fart around and then cash out exclaiming it's everyone else's fault? The way he stole Fail0verflow's work with the PS3 security leak after failing to find a hypervisor exploit for months absolutely soured any respect I had for him at the time
  - throwitawayfam 3y ago
    Yep, did exactly that. IMO he threw a fit, even though AMD was working with him squashing bugs. https://github.com/RadeonOpenCompute/ROCm/issues/2198#issuecomment-1574383483 https://github.com/RadeonOpenCompute/ROCm/issues/2198#issuec...
    - aeyes 3y ago
      He's back on it after getting AMD's CEO to commit resources to this: https://twitter.com/realGeorgeHotz/status/1669803464082489347 https://twitter.com/realGeorgeHotz/status/166980346408248934... https://twitter.com/LisaSu/status/1669848494637735936 https://twitter.com/LisaSu/status/1669848494637735936
    - deleted 3y ago
      [deleted]
    - nomel 3y ago
      To be fair, kernel crashes from running an AMD provided demo loop isn’t something he should have to work with them on. That’s borderline incompetence. His perspective was around integration into his product, where every AMD bug is a bug in his product. They deserve criticism, and responded accordingly (actual resources to get their shit together). It’s not like GPU accelerated ML is some new thing.
    - JonChesterfield 3y ago
      That's a tough issue to read through, thanks for the link. 'Your demo code on a system setup exactly as you describe dereferences null in the kernel and falls over'. Fuzz testing + a vaguely reasonable kernel debugging workflow should make things like that much harder to find.
  - adastra22 3y ago
    Wow, TIL
  - kinematikk 3y ago
    Do you have a source on the stealing part? A quick Google search didn't result in anything
    - IntelMiner 3y ago
      Marcan (of Asahi Linux fame) has talked about it many times before. But an abridged version Fail0verflow demoed how they were able to derive the private signing keys for the Sony Playstation 3 console at I believe CCC Geohot after watching the livestream raced into action to demo a "hello world!" jailbreak application and absolutely stole their thunder without giving any credit
      - aftbit 3y ago
        This apparently worked pretty well for him, as I still remember him primarily as "that guy who hacked PS3". Some people let someone else do the hard technical core, then do all the other easy but boring stuff and claim 100% credit.
        22c 3y ago
        I remember geohot as being one of the people who developed a fairly successful jailbreak for iPhone. I understand that iPhone jailbreaking is often standing on the shoulders of predecessors, but I believe he does deserve significant credit for at least one popular iPhone jailbreak.
      - ryanjshaw 3y ago
        If they demod something then they released it publically and it was fair game? In any case he absolutely did credit them, it's easily verifiable: https://web.archive.org/web/20110104040706/http://geohot.com/ https://web.archive.org/web/20110104040706/http://geohot.com... Sony sued them both, afterall!
  - ShamelessC 3y ago
    > The way he stole Fail0verflow's work with the PS3 security leak after failing to find a hypervisor exploit for months absolutely soured any respect I had for him at the time That sounds interesting. I tried googling about it but can't really find much other than that failoverflow found a key and didn't release it, and then geohot released his own subsequently. I'd love to hear more about how directly he "stole" the work from the Fail0verflow team. edit: Reading some sibling comments here, it seems you are either mistaken and/or were exaggerating your claim about the "theft" here. As far as I can tell, he simply took their findings and made his own version of an exploit that they had detailed publicly. That may be in poor taste in this particular community but it's certainly not theft. I do agree that his behavior there was lacking in decency, but not to the degree implied here where I was thinking he _literally_ stole their exploit by hacking them, or something similar to that.
    - cyrux004 3y ago
      People here generally try to bash people who are much smarter than them, throwing shade at their background. They will say that he abandoned his first company, gave up on tiny grad but both of them are very much alive projects
      - deleted 3y ago
        [deleted]
  - deleted 3y ago
    [deleted]
  - georgehotz 3y ago
    Are you interested in being factually correct, or are you interested in hating? If it's the former, I think you should do some research. If it's the latter, :salute: I had GPT-4 do some research for you, hopefully you will incorporate it in future comments you make about me. https://chat.openai.com/share/d0fa24e9-3ed7-4b17-8497-24bfdd2efde0 https://chat.openai.com/share/d0fa24e9-3ed7-4b17-8497-24bfdd...
- nomel 3y ago
  Obligatory Lex Fridman podcast, where he discusses it: https://youtu.be/dNrTrx42DGQ?t=2408 https://youtu.be/dNrTrx42DGQ?t=2408
jauntywundrkind 3y ago
Pytorch is just using Google's OpenXLA now, & OpenXLA is the actual cross platform thing, no? I'm not very well versed in this area, so pardon if mistaken. https://pytorch.org/blog/pytorch-2.0-xla-path-forward/ https://pytorch.org/blog/pytorch-2.0-xla-path-forward/
- fotcorn 3y ago
  You can use OpenXLA, but it's not the default. The main use-case for OpenXLA is running PyTorch on Google TPUs. OpenXLA also supports GPUs, but I am not sure how many people use that. Afaik JAX uses OpenXLA as backend to run on GPUs. If you use model.compile() in PyTorch, you use TorchInductor and OpenAIs Triton by default.
  - jauntywundrkind 3y ago
    Thank you for saying something useful here. I was vaguely under the impression that pytorch 2.0 had fully flipped to defaulting to openxla. That seems to not be the case. Good to hear more than a cheap snub. OpenAI Triton as the reason other GPUs work is a real non-shit answer, it seems. And interesting to hear JAX too. Thank you for being robustly useful & informative.
- mathisfun123 3y ago
  > Pytorch is just using Google's OpenXLA now this is so far from accurate it should be considered libelous; from the link > PyTorch/XLA is set to migrate to the open source OpenXLA so PyTorch on the XLA backend is set to migrate to use OpenXLA instead of XLA. but basically everyone moved from XLA to OpenXLA because there is no more OSS XLA. so that's it. in general, PyTorch has several backends, including plenty of homegrown CUDA and CPU kernels. in fact the majority of your PyTorch code runs through PyTorch's own kernels.
- voz_ 3y ago
  Wrong.
mandevil 3y ago
It isn't the hobbyists who are making sure that PyTorch and other frameworks runs well on these chips, but teams of engineers who work for NVIDIA, AMD, Intel, etc. who are doing this as their primary assigned jobs, in exchange for money from their employer, who are paying those salaries because they want to sell chips into the enormous demand for running PyTorch faster. Hobbyist and open-source are definitely not synonyms.
- Eisenstein 3y ago
  People don't usually get employed to make things with no demand, and people who work for companies with a budget line don't really care how much the nVidia tax is. You can thank hobbyists for creating a lot of demand for compatability with other cards.
  - kiratp 3y ago
    There are so many billions of dollar being spent on this hardware that everyone other than Nvidia is doing everything they can to make competition happen. Eg: https://www.intel.com/content/www/us/en/developer/videos/optimize-dl-workloads-intel-optimized-pytorch.html https://www.intel.com/content/www/us/en/developer/videos/opt... https://www.intel.com/content/www/us/en/developer/tools/oneapi/optimization-for-tensorflow.html https://www.intel.com/content/www/us/en/developer/tools/onea... https://developer.apple.com/metal/tensorflow-plugin/ https://developer.apple.com/metal/tensorflow-plugin/ Large scale opensource is, outside of a few exceptions, built by engineers paid to build it.
  - roenxi 3y ago
    There is huge demand for AMD cards that can efficiently multiply matrices together. The issue is that while there are currently isolated cases where people can make them do that, it doesn't seem to be possible at the scale that it needs to happen at. AMD are being dragged along by the market. Willingly, they aren't fighting it, but their focus has been on other areas.
    - viewtransform 3y ago

mattnewton 3y ago

Re: startups, Geohotz raised a few million for this already. https://tinygrad.org/ https://tinygrad.org/

IntelMiner 3y ago
Didn't he do what he always does. Rake in a ton of money, fart around and then cash out exclaiming it's everyone else's fault? The way he stole Fail0verflow's work with the PS3 security leak after failing to find a hypervisor exploit for months absolutely soured any respect I had for him at the time
- throwitawayfam 3y ago
  Yep, did exactly that. IMO he threw a fit, even though AMD was working with him squashing bugs. https://github.com/RadeonOpenCompute/ROCm/issues/2198#issuecomment-1574383483 https://github.com/RadeonOpenCompute/ROCm/issues/2198#issuec...
  - aeyes 3y ago
    He's back on it after getting AMD's CEO to commit resources to this: https://twitter.com/realGeorgeHotz/status/1669803464082489347 https://twitter.com/realGeorgeHotz/status/166980346408248934... https://twitter.com/LisaSu/status/1669848494637735936 https://twitter.com/LisaSu/status/1669848494637735936
  - deleted 3y ago
    [deleted]
  - nomel 3y ago
    To be fair, kernel crashes from running an AMD provided demo loop isn’t something he should have to work with them on. That’s borderline incompetence. His perspective was around integration into his product, where every AMD bug is a bug in his product. They deserve criticism, and responded accordingly (actual resources to get their shit together). It’s not like GPU accelerated ML is some new thing.
  - JonChesterfield 3y ago
    That's a tough issue to read through, thanks for the link. 'Your demo code on a system setup exactly as you describe dereferences null in the kernel and falls over'. Fuzz testing + a vaguely reasonable kernel debugging workflow should make things like that much harder to find.
- adastra22 3y ago
  Wow, TIL
- kinematikk 3y ago
  Do you have a source on the stealing part? A quick Google search didn't result in anything
  - IntelMiner 3y ago
    Marcan (of Asahi Linux fame) has talked about it many times before. But an abridged version Fail0verflow demoed how they were able to derive the private signing keys for the Sony Playstation 3 console at I believe CCC Geohot after watching the livestream raced into action to demo a "hello world!" jailbreak application and absolutely stole their thunder without giving any credit
    - aftbit 3y ago
      This apparently worked pretty well for him, as I still remember him primarily as "that guy who hacked PS3". Some people let someone else do the hard technical core, then do all the other easy but boring stuff and claim 100% credit.
      - 22c 3y ago
        I remember geohot as being one of the people who developed a fairly successful jailbreak for iPhone. I understand that iPhone jailbreaking is often standing on the shoulders of predecessors, but I believe he does deserve significant credit for at least one popular iPhone jailbreak.
    - ryanjshaw 3y ago
      If they demod something then they released it publically and it was fair game? In any case he absolutely did credit them, it's easily verifiable: https://web.archive.org/web/20110104040706/http://geohot.com/ https://web.archive.org/web/20110104040706/http://geohot.com... Sony sued them both, afterall!
- ShamelessC 3y ago
  > The way he stole Fail0verflow's work with the PS3 security leak after failing to find a hypervisor exploit for months absolutely soured any respect I had for him at the time That sounds interesting. I tried googling about it but can't really find much other than that failoverflow found a key and didn't release it, and then geohot released his own subsequently. I'd love to hear more about how directly he "stole" the work from the Fail0verflow team. edit: Reading some sibling comments here, it seems you are either mistaken and/or were exaggerating your claim about the "theft" here. As far as I can tell, he simply took their findings and made his own version of an exploit that they had detailed publicly. That may be in poor taste in this particular community but it's certainly not theft. I do agree that his behavior there was lacking in decency, but not to the degree implied here where I was thinking he _literally_ stole their exploit by hacking them, or something similar to that.
  - cyrux004 3y ago
    People here generally try to bash people who are much smarter than them, throwing shade at their background. They will say that he abandoned his first company, gave up on tiny grad but both of them are very much alive projects
    - deleted 3y ago
      [deleted]
- deleted 3y ago
  [deleted]
- georgehotz 3y ago
  Are you interested in being factually correct, or are you interested in hating? If it's the former, I think you should do some research. If it's the latter, :salute: I had GPT-4 do some research for you, hopefully you will incorporate it in future comments you make about me. https://chat.openai.com/share/d0fa24e9-3ed7-4b17-8497-24bfdd2efde0 https://chat.openai.com/share/d0fa24e9-3ed7-4b17-8497-24bfdd...
nomel 3y ago
Obligatory Lex Fridman podcast, where he discusses it: https://youtu.be/dNrTrx42DGQ?t=2408 https://youtu.be/dNrTrx42DGQ?t=2408

jauntywundrkind 3y ago

Pytorch is just using Google's OpenXLA now, & OpenXLA is the actual cross platform thing, no? I'm not very well versed in this area, so pardon if mistaken. https://pytorch.org/blog/pytorch-2.0-xla-path-forward/ https://pytorch.org/blog/pytorch-2.0-xla-path-forward/

fotcorn 3y ago
You can use OpenXLA, but it's not the default. The main use-case for OpenXLA is running PyTorch on Google TPUs. OpenXLA also supports GPUs, but I am not sure how many people use that. Afaik JAX uses OpenXLA as backend to run on GPUs. If you use model.compile() in PyTorch, you use TorchInductor and OpenAIs Triton by default.
- jauntywundrkind 3y ago
  Thank you for saying something useful here. I was vaguely under the impression that pytorch 2.0 had fully flipped to defaulting to openxla. That seems to not be the case. Good to hear more than a cheap snub. OpenAI Triton as the reason other GPUs work is a real non-shit answer, it seems. And interesting to hear JAX too. Thank you for being robustly useful & informative.
mathisfun123 3y ago
> Pytorch is just using Google's OpenXLA now this is so far from accurate it should be considered libelous; from the link > PyTorch/XLA is set to migrate to the open source OpenXLA so PyTorch on the XLA backend is set to migrate to use OpenXLA instead of XLA. but basically everyone moved from XLA to OpenXLA because there is no more OSS XLA. so that's it. in general, PyTorch has several backends, including plenty of homegrown CUDA and CPU kernels. in fact the majority of your PyTorch code runs through PyTorch's own kernels.
voz_ 3y ago
Wrong.

mandevil 3y ago

It isn't the hobbyists who are making sure that PyTorch and other frameworks runs well on these chips, but teams of engineers who work for NVIDIA, AMD, Intel, etc. who are doing this as their primary assigned jobs, in exchange for money from their employer, who are paying those salaries because they want to sell chips into the enormous demand for running PyTorch faster. Hobbyist and open-source are definitely not synonyms.

Eisenstein 3y ago
People don't usually get employed to make things with no demand, and people who work for companies with a budget line don't really care how much the nVidia tax is. You can thank hobbyists for creating a lot of demand for compatability with other cards.
- kiratp 3y ago
  There are so many billions of dollar being spent on this hardware that everyone other than Nvidia is doing everything they can to make competition happen. Eg: https://www.intel.com/content/www/us/en/developer/videos/optimize-dl-workloads-intel-optimized-pytorch.html https://www.intel.com/content/www/us/en/developer/videos/opt... https://www.intel.com/content/www/us/en/developer/tools/oneapi/optimization-for-tensorflow.html https://www.intel.com/content/www/us/en/developer/tools/onea... https://developer.apple.com/metal/tensorflow-plugin/ https://developer.apple.com/metal/tensorflow-plugin/ Large scale opensource is, outside of a few exceptions, built by engineers paid to build it.
- roenxi 3y ago
  There is huge demand for AMD cards that can efficiently multiply matrices together. The issue is that while there are currently isolated cases where people can make them do that, it doesn't seem to be possible at the scale that it needs to happen at. AMD are being dragged along by the market. Willingly, they aren't fighting it, but their focus has been on other areas.
  - viewtransform 3y ago