5 ms·

> Thinking in terms of differentiable programming is a given and I think certainly counts as "from scratch" in this case. I have to disagree on that being an o

by re 3y ago

> Thinking in terms of differentiable programming is a given and I think certainly counts as "from scratch" in this case. I have to disagree on that being an obvious assumption for the meaning of "from scratch", especially given that the book description says that readers only need to know Python. It feels like if I read "Crafting Interpreters" only to find that step one is to download Lex and Yacc because everyone working in the space already knows how parsers work. > I suspect the don't really understand what's happening under the hood or how contemporary machine learning works. Everyone has to start somewhere. I thought I would be interested in a book like this precisely because I don't already fully understand what's happening under the hood, but it sounds like it might not actually be a good starting point for my idea of "from scratch."

rasbt 3y ago
On that note, I have a relative comprehensive intro to PyTorch in the Appendix (~40 pages) that go over automatic differentiation etc. The alternative, if you want to build something truly from scratch, would be to implement everything in CUDA, but that would not be a very accessible book.
- antonvs 3y ago
  “If you wish to make an apple pie from scratch, you must first invent the universe.” —- Carl Sagan
nerdponx 3y ago
It depends on which hood you want to look under. Let's say you wanted to write your own SSH client as a learning exercise. Is it cheating if you use OpenSSL? Is it cheating if you use Python? Is it cheating if you use a C compiler?
- fragmede 3y ago
  Oh, I see you're using an existing ISA and not creating your own for this. And also, where do you get off using existing integrated circuits. From scratch means you have to start from sand and make your own nand gates and get to an adder and a latch and then a cpu and write an operating system for it before you can get to using language that you invented for this purpose.

by re 3y ago

rasbt 3y ago

On that note, I have a relative comprehensive intro to PyTorch in the Appendix (~40 pages) that go over automatic differentiation etc. The alternative, if you want to build something truly from scratch, would be to implement everything in CUDA, but that would not be a very accessible book.

antonvs 3y ago
“If you wish to make an apple pie from scratch, you must first invent the universe.” —- Carl Sagan

nerdponx 3y ago

It depends on which hood you want to look under. Let's say you wanted to write your own SSH client as a learning exercise. Is it cheating if you use OpenSSL? Is it cheating if you use Python? Is it cheating if you use a C compiler?

fragmede 3y ago
Oh, I see you're using an existing ISA and not creating your own for this. And also, where do you get off using existing integrated circuits. From scratch means you have to start from sand and make your own nand gates and get to an adder and a latch and then a cpu and write an operating system for it before you can get to using language that you invented for this purpose.