7 ms·

Tinygrad 0.9.0

241 points by wozeparrot 2y ago

dhruvdh 2y ago
What's the point of the 8000 LOC limit? Has anyone worked in a project with a LOC limit? Why was the limit in place?
- torlok 2y ago
  It's just a way to keep the code size in check, make sure it can be read and understood relatively easily. Don't overthink it. I doubt much, if any, research went into picking the limit. The line width is over 120 in many places, and the code inevitably ends up looking like cache_key = (device, st, dtype, op, arg, tuple(ref(x) for x in srcs)) if base is None else (st, ref(base))
  - kthxb 2y ago
    Seems the code sample contradicts your first statement
    - torlok 2y ago
      Indeed, I was making a point.
    - Lerc 2y ago
      I think this might be an example of https://en.m.wikipedia.org/wiki/Goodhart's_law https://en.m.wikipedia.org/wiki/Goodhart's_law The line count probably does still act as a limit on complexity overall but perhaps less than hoped for.
  - kveykva 2y ago
    This is truly depressing because the aspirations of tinygrad are so appealing in terms of being concise, effective and maintainable. Then, instead, they throw comprehensibility entirely out of the window.
- ofou 2y ago
  To stay Tiny
- jorlow 2y ago
  To keep it "tiny". (IIRC geohot started it because he thought pytorch and others were bloated and a simple ml framework would be inherently better)
- vinkelhake 2y ago
  Right now there doesn't seem to be much point. IIRC they had a 1000 LOC limit on the core part of the code when the project was early. The README no longer mentions the limit and it looks like they just raise it whenever needed. Three months ago it was bumped to 6500 LOC. One month ago it was bumped to 8000 lines.
  - rifty 2y ago
    A tech debt ceiling so to speak then. There might be some use to it. It's still inevitably increased, but only after debate, discussion, and a lot of time in-between really considering the form and impact of the code being entered to fit within the constraint
- gkbrk 2y ago
  To compare, the PyTorch repo has ~400k lines of C, ~850k lines of C++ and more than 1.5 million lines of Python code. PyTorch does more than tinygrad, but does it really do 343x more things?
  - danielmarkbruce 2y ago
    Probably.
  - adolph 2y ago
    If PyTorch does the 1-2 things you need and Tinygrad doesn't do, then what are you going to use? The Python source distribution has long maintained the philosophy of “batteries included” – having a rich and versatile standard library which is immediately available, without making the user download separate packages. https://peps.python.org/pep-0206/ https://peps.python.org/pep-0206/ OTOH: Simple is better than complex. Complex is better than complicated. https://peps.python.org/pep-0020/ https://peps.python.org/pep-0020/
    - jononor 2y ago
      PyTorch of course. Or alternatively a lib or custom code on top of TinyGrad. Is that a problem?
  - jejeyyy77 2y ago
    uh, ya? lol
  - Q6T46nT668w6i3m 2y ago
    Easily
  - threecheese 2y ago
    geohot explained on one of this streams, and per my terrible memory: “tiny” is a way of expressing the architecture constraint that the system should not attempt to target [(many hardware architectures and their optimizations) * (many model, training, etc etc variants)] like PyTorch - which requires maintenance of a shit ton of code and a staff/community behind Meta. Instead, tinygrad should provide core abstractions that can be composed to accomplish a similar set of targets but for only one hardware architecture (for now I guess). He is releasing a companion hardware item which would fund the development I believe.
- matternous 2y ago
  It used to be a 1,000. I guess it’s just a reminder to be succinct.
- entrep 2y ago
  Cyclomatic complexity would be a better measurement.
- Barrin92 2y ago
  Looking at the code base right now, apparently to produce some of the most unreadable code possible (https://github.com/tinygrad/tinygrad/blob/master/tinygrad/renderer/assembly.py https://github.com/tinygrad/tinygrad/blob/master/tinygrad/re...) LOC limits have to be one of the worst incentives you can give programmers.
- dimatura 2y ago
  The only one I can think of the dwm window manager (https://dwm.suckless.org/ https://dwm.suckless.org/), that used to prominently mention a SLOC limit of 2000. Doesn't seem to be mentioned in the landing page anymore, not sure if it's still in effect.
- zzo38computer 2y ago
  There are benefits of having a low number of lines of codes, e.g. if you want to print out on a paper (and reduce the number of pages), or store on a disk with a limited storage (although number of bytes is a more useful measure, then), or if you want to read it to understand it in less time than a longer program, etc. Of course, the limit of number of characters on each line, is also necessary, then. However, that doesn't solve everything. Many things it does not accurately measure, e.g. complexity, number of stuff in one line, program speed, memory usage, etc. Those are other things to measure, and it can be helpful to reduce memory usage etc, but that is not the number of lines of codes.
- blitzar 2y ago
  No new features.
dventimihasura 2y ago
You might consider posting not the release page, but rather the repo page: https://github.com/tinygrad/tinygrad https://github.com/tinygrad/tinygrad Another good alternative would be: https://tinygrad.org/#tinygrad https://tinygrad.org/#tinygrad
- fragmede 2y ago
  That's been posed many times*. What's new here is version 0.9.0 * https://hn.algolia.com/?q=https%3A%2F%2Fgithub.com%2Ftinygrad%2Ftinygrad https://hn.algolia.com/?q=https%3A%2F%2Fgithub.com%2Ftinygra...
  - dventimihasura 2y ago
    And yet, I never encountered any of those other posts. I bet that's true for most HN readers. But, suit yourself. I just think you'll get higher engagement if you put your best food forward by helping newcomers understand what this thing is. But hey, maybe I'm wrong.
    - fragmede 2y ago
      (Not my submission) I'm biased, because I already know what tinygrad is, but there's a link to the main github page for the repo at the top of the page. There's "get higher engagement" but there's also "readers here aren't drooling morons and know what a github is and can click on the link to the repo and find the README.md". But hey, maybe I'm wrong.
      - dventimihasura 2y ago
        Is "readers here [ARE] drooling morons and [DON'T] know what a github is and [CAN'T] click on the link to the repo and find the README.MD" the only other alternative? There's no middle ground? There's no "readers here aren't drooling morons and know what a github is and can click the link to the repo and find the README.md, but they're also busy and their attention is limited and so some of them won't be bothered to take those extra steps to learn about your project if you can't be bothered to put your best foot forward, and it's not a moral failing on either side, it's just the way it is"?
Havoc 2y ago
>experimental backends for not requiring any userspace components like ROCm or CUDA. That's pretty wild
jarbus 2y ago
I see they have experimental AMD backends that don’t use ROCm. Is ROCm that bad that they wrote their own, or was there some other justification for this?
- blihp 2y ago
  The author of the library has done numerous YouTube rants on how bad he thinks the AMD compute drivers are.
  - JMiao 2y ago
    What does he get wrong?
    - KaoruAoiShiho 2y ago
      He's not wrong.
  - roenxi 2y ago
    Has he? The ones I'm aware of he was complaining about the low quality of the generic kernel drivers. AMD software had a tendency to crash when doing anything outside of standard video games (which was my experience too, but I've caved and bought Nvidia since then; average driver quality of Nvidia on linux seems to be much lower but the kernel doesn't go down which is nice. Got a lot of OOM errors where on ROCm the kernel froze requiring a full system restart). But this is interesting and probably strong evidence that the CUDA API isn't the moat people thought it was. CUDA multiplies matricies and that is close to a commodity operation. The moat actually seems to be Nvidia's higher generic software engineering standards, the difficulty in writing job scheduling/memory management infrastructure and possibly the fact that closed firmware is the norm.
    - alexbaden 2y ago
      Arguably the nvidia AI moat is PyTorch and the heavily optimized libraries behind it. The CUDA language and toolchain helped get that effort off the ground, no doubt, but PyTorch is written and optimized for CUDA first. All other backends work best with similar semantics to CUDA and have to match Cuda semantics to keep their users happy.
teleforce 2y ago
This is a tiny autograd library alternative in D language: https://code.dlang.org/packages/tiny-autodiff https://code.dlang.org/packages/tiny-autodiff Unlike other autograd libraries it utilized native D Mir GLAS linear algebra library [1]: [1] Numeric age for D: Mir GLAS is faster than OpenBLAS and Eigen http://blog.mir.dlang.io/glas/benchmark/openblas/2016/09/23/glas-gemm-benchmark.html http://blog.mir.dlang.io/glas/benchmark/openblas/2016/09/23/...

Tinygrad 0.9.0

241 points by wozeparrot 2y ago

dhruvdh 2y ago

What's the point of the 8000 LOC limit? Has anyone worked in a project with a LOC limit? Why was the limit in place?

torlok 2y ago
It's just a way to keep the code size in check, make sure it can be read and understood relatively easily. Don't overthink it. I doubt much, if any, research went into picking the limit. The line width is over 120 in many places, and the code inevitably ends up looking like cache_key = (device, st, dtype, op, arg, tuple(ref(x) for x in srcs)) if base is None else (st, ref(base))
- kthxb 2y ago
  Seems the code sample contradicts your first statement
  - torlok 2y ago
    Indeed, I was making a point.
  - Lerc 2y ago
    I think this might be an example of https://en.m.wikipedia.org/wiki/Goodhart's_law https://en.m.wikipedia.org/wiki/Goodhart's_law The line count probably does still act as a limit on complexity overall but perhaps less than hoped for.
- kveykva 2y ago
  This is truly depressing because the aspirations of tinygrad are so appealing in terms of being concise, effective and maintainable. Then, instead, they throw comprehensibility entirely out of the window.
ofou 2y ago
To stay Tiny
jorlow 2y ago
To keep it "tiny". (IIRC geohot started it because he thought pytorch and others were bloated and a simple ml framework would be inherently better)
vinkelhake 2y ago
Right now there doesn't seem to be much point. IIRC they had a 1000 LOC limit on the core part of the code when the project was early. The README no longer mentions the limit and it looks like they just raise it whenever needed. Three months ago it was bumped to 6500 LOC. One month ago it was bumped to 8000 lines.
- rifty 2y ago
  A tech debt ceiling so to speak then. There might be some use to it. It's still inevitably increased, but only after debate, discussion, and a lot of time in-between really considering the form and impact of the code being entered to fit within the constraint
gkbrk 2y ago
To compare, the PyTorch repo has ~400k lines of C, ~850k lines of C++ and more than 1.5 million lines of Python code. PyTorch does more than tinygrad, but does it really do 343x more things?
- danielmarkbruce 2y ago
  Probably.
- adolph 2y ago
  If PyTorch does the 1-2 things you need and Tinygrad doesn't do, then what are you going to use? The Python source distribution has long maintained the philosophy of “batteries included” – having a rich and versatile standard library which is immediately available, without making the user download separate packages. https://peps.python.org/pep-0206/ https://peps.python.org/pep-0206/ OTOH: Simple is better than complex. Complex is better than complicated. https://peps.python.org/pep-0020/ https://peps.python.org/pep-0020/
  - jononor 2y ago
    PyTorch of course. Or alternatively a lib or custom code on top of TinyGrad. Is that a problem?
- jejeyyy77 2y ago
  uh, ya? lol
- Q6T46nT668w6i3m 2y ago
  Easily
- threecheese 2y ago
  geohot explained on one of this streams, and per my terrible memory: “tiny” is a way of expressing the architecture constraint that the system should not attempt to target [(many hardware architectures and their optimizations) * (many model, training, etc etc variants)] like PyTorch - which requires maintenance of a shit ton of code and a staff/community behind Meta. Instead, tinygrad should provide core abstractions that can be composed to accomplish a similar set of targets but for only one hardware architecture (for now I guess). He is releasing a companion hardware item which would fund the development I believe.
matternous 2y ago
It used to be a 1,000. I guess it’s just a reminder to be succinct.
entrep 2y ago
Cyclomatic complexity would be a better measurement.
Barrin92 2y ago
Looking at the code base right now, apparently to produce some of the most unreadable code possible (https://github.com/tinygrad/tinygrad/blob/master/tinygrad/renderer/assembly.py https://github.com/tinygrad/tinygrad/blob/master/tinygrad/re...) LOC limits have to be one of the worst incentives you can give programmers.
dimatura 2y ago
The only one I can think of the dwm window manager (https://dwm.suckless.org/ https://dwm.suckless.org/), that used to prominently mention a SLOC limit of 2000. Doesn't seem to be mentioned in the landing page anymore, not sure if it's still in effect.
zzo38computer 2y ago
There are benefits of having a low number of lines of codes, e.g. if you want to print out on a paper (and reduce the number of pages), or store on a disk with a limited storage (although number of bytes is a more useful measure, then), or if you want to read it to understand it in less time than a longer program, etc. Of course, the limit of number of characters on each line, is also necessary, then. However, that doesn't solve everything. Many things it does not accurately measure, e.g. complexity, number of stuff in one line, program speed, memory usage, etc. Those are other things to measure, and it can be helpful to reduce memory usage etc, but that is not the number of lines of codes.
blitzar 2y ago
No new features.

dventimihasura 2y ago

You might consider posting not the release page, but rather the repo page: https://github.com/tinygrad/tinygrad https://github.com/tinygrad/tinygrad Another good alternative would be: https://tinygrad.org/#tinygrad https://tinygrad.org/#tinygrad

fragmede 2y ago
That's been posed many times*. What's new here is version 0.9.0 * https://hn.algolia.com/?q=https%3A%2F%2Fgithub.com%2Ftinygrad%2Ftinygrad https://hn.algolia.com/?q=https%3A%2F%2Fgithub.com%2Ftinygra...
- dventimihasura 2y ago
  And yet, I never encountered any of those other posts. I bet that's true for most HN readers. But, suit yourself. I just think you'll get higher engagement if you put your best food forward by helping newcomers understand what this thing is. But hey, maybe I'm wrong.
  - fragmede 2y ago
    (Not my submission) I'm biased, because I already know what tinygrad is, but there's a link to the main github page for the repo at the top of the page. There's "get higher engagement" but there's also "readers here aren't drooling morons and know what a github is and can click on the link to the repo and find the README.md". But hey, maybe I'm wrong.
    - dventimihasura 2y ago
      Is "readers here [ARE] drooling morons and [DON'T] know what a github is and [CAN'T] click on the link to the repo and find the README.MD" the only other alternative? There's no middle ground? There's no "readers here aren't drooling morons and know what a github is and can click the link to the repo and find the README.md, but they're also busy and their attention is limited and so some of them won't be bothered to take those extra steps to learn about your project if you can't be bothered to put your best foot forward, and it's not a moral failing on either side, it's just the way it is"?

Havoc 2y ago

>experimental backends for not requiring any userspace components like ROCm or CUDA. That's pretty wild

jarbus 2y ago

I see they have experimental AMD backends that don’t use ROCm. Is ROCm that bad that they wrote their own, or was there some other justification for this?

blihp 2y ago
The author of the library has done numerous YouTube rants on how bad he thinks the AMD compute drivers are.
- JMiao 2y ago
  What does he get wrong?
  - KaoruAoiShiho 2y ago
    He's not wrong.
- roenxi 2y ago
  Has he? The ones I'm aware of he was complaining about the low quality of the generic kernel drivers. AMD software had a tendency to crash when doing anything outside of standard video games (which was my experience too, but I've caved and bought Nvidia since then; average driver quality of Nvidia on linux seems to be much lower but the kernel doesn't go down which is nice. Got a lot of OOM errors where on ROCm the kernel froze requiring a full system restart). But this is interesting and probably strong evidence that the CUDA API isn't the moat people thought it was. CUDA multiplies matricies and that is close to a commodity operation. The moat actually seems to be Nvidia's higher generic software engineering standards, the difficulty in writing job scheduling/memory management infrastructure and possibly the fact that closed firmware is the norm.
  - alexbaden 2y ago
    Arguably the nvidia AI moat is PyTorch and the heavily optimized libraries behind it. The CUDA language and toolchain helped get that effort off the ground, no doubt, but PyTorch is written and optimized for CUDA first. All other backends work best with similar semantics to CUDA and have to match Cuda semantics to keep their users happy.

teleforce 2y ago

This is a tiny autograd library alternative in D language: https://code.dlang.org/packages/tiny-autodiff https://code.dlang.org/packages/tiny-autodiff Unlike other autograd libraries it utilized native D Mir GLAS linear algebra library [1]: [1] Numeric age for D: Mir GLAS is faster than OpenBLAS and Eigen http://blog.mir.dlang.io/glas/benchmark/openblas/2016/09/23/glas-gemm-benchmark.html http://blog.mir.dlang.io/glas/benchmark/openblas/2016/09/23/...