4 ms·

This is really great, thank you. I would love to see a real "from scratch" that doesn't use torch.py et. al., though.

by pottspotts 4y ago

javajosh 4y ago
Same, but one that doesn't assume python or an operating system.
- tysam_and 4y ago
  Same, but I would honestly rather we get away from silicon or anything resembling what we call these "computers" these days. Things are too complicated and we need to truly get back to the basics: rocks on a hill.
  - bitL 4y ago
    Abacus-driven deep learning with trained monkeys flipping gradients.
    - qup 4y ago
      Outside tools are not required. We'll selectively breed humans until one can just compute it in...meat[0]. https://www.youtube.com/watch?v=7tScAyNaRdQ https://www.youtube.com/watch?v=7tScAyNaRdQ
      - tysam_and 4y ago
        in mitro computing
    - captn3m0 4y ago
      You don't need to flip gradients, just need to train the monkeys to operate matchboxes: https://en.wikipedia.org/wiki/Matchbox_Educable_Noughts_and_Crosses_Engine https://en.wikipedia.org/wiki/Matchbox_Educable_Noughts_and_...
    - trosi 4y ago
      Reminds of that part in the Three-Body Problem novel where the emperor of the videogame builds a CPU with soldiers as bits
- zerd 4y ago
  Sounds like a project for Ben Eater https://eater.net/ https://eater.net/
ironrabbit 4y ago
Andrej's "Building Makemore" series is exactly this, it includes a wonderful lecture where he computes all the gradients for a simple network by hand and compares them against the values produced by torch's autograd.
- tysam_and 4y ago
  I find Makemore to be my favorite neural-net-based (w)rapper.
bitL 4y ago
That would be next to impossible to do even in a semester long course...
- amelius 4y ago
  Here you go: https://www.youtube.com/watch?v=VMj-3S1tku0 https://www.youtube.com/watch?v=VMj-3S1tku0
deleted 4y ago
[deleted]
bialpio 4y ago
https://youtu.be/VMj-3S1tku0 https://youtu.be/VMj-3S1tku0 The entire series is at https://karpathy.ai/zero-to-hero.html https://karpathy.ai/zero-to-hero.html, first video is literally from scratch (i.e. just Python, the only external dependency is for plotting graphs if I recall correctly).
1vuio0pswjnm7 4y ago
Does one really need Python. What about using Lua. https://news.ycombinator.com/item?id=7928738 https://news.ycombinator.com/item?id=7928738

This is really great, thank you. I would love to see a real "from scratch" that doesn't use torch.py et. al., though.

javajosh 4y ago

Same, but one that doesn't assume python or an operating system.

tysam_and 4y ago
Same, but I would honestly rather we get away from silicon or anything resembling what we call these "computers" these days. Things are too complicated and we need to truly get back to the basics: rocks on a hill.
- bitL 4y ago
  Abacus-driven deep learning with trained monkeys flipping gradients.
  - qup 4y ago
    Outside tools are not required. We'll selectively breed humans until one can just compute it in...meat[0]. https://www.youtube.com/watch?v=7tScAyNaRdQ https://www.youtube.com/watch?v=7tScAyNaRdQ
    - tysam_and 4y ago
      in mitro computing
  - captn3m0 4y ago
    You don't need to flip gradients, just need to train the monkeys to operate matchboxes: https://en.wikipedia.org/wiki/Matchbox_Educable_Noughts_and_Crosses_Engine https://en.wikipedia.org/wiki/Matchbox_Educable_Noughts_and_...
  - trosi 4y ago
    Reminds of that part in the Three-Body Problem novel where the emperor of the videogame builds a CPU with soldiers as bits
zerd 4y ago
Sounds like a project for Ben Eater https://eater.net/ https://eater.net/

ironrabbit 4y ago

Andrej's "Building Makemore" series is exactly this, it includes a wonderful lecture where he computes all the gradients for a simple network by hand and compares them against the values produced by torch's autograd.

tysam_and 4y ago
I find Makemore to be my favorite neural-net-based (w)rapper.

bitL 4y ago

That would be next to impossible to do even in a semester long course...

amelius 4y ago
Here you go: https://www.youtube.com/watch?v=VMj-3S1tku0 https://www.youtube.com/watch?v=VMj-3S1tku0

deleted 4y ago

[deleted]

bialpio 4y ago

https://youtu.be/VMj-3S1tku0 https://youtu.be/VMj-3S1tku0 The entire series is at https://karpathy.ai/zero-to-hero.html https://karpathy.ai/zero-to-hero.html, first video is literally from scratch (i.e. just Python, the only external dependency is for plotting graphs if I recall correctly).

1vuio0pswjnm7 4y ago

Does one really need Python. What about using Lua. https://news.ycombinator.com/item?id=7928738 https://news.ycombinator.com/item?id=7928738