2 ms·
Pushback notwithstanding, this article is 100% correct in all PyTorch criticisms. PyTorch was a platform for fast experimentation with eager evaluation, now the
by 0x19241217 2y ago
Pushback notwithstanding, this article is 100% correct in all PyTorch criticisms. PyTorch was a platform for fast experimentation with eager evaluation, now they shoehorn "compilers" into it. "compilers", because a lot of the work is done by g++ and Triton.
It is a messy and quickly expanding codebase with many surprises like segfaults and leaks.
Is scientific experimentation really sped up by these frameworks? Everyone uses the Transformer model and uses the same algorithms over and over again.
If researchers wrote directly in C or Fortran, perhaps they'd get new ideas. The core inference (see Karparthy's llama.c) is ridiculously small. Core training does not seem much larger either.
- dunefox 2y ago> If researchers wrote directly in C or Fortran... ... then they would get nothing done.
- pklausler 2y agoWhy not?
- dunefox 2y ago1. data cleaning and other tasks for which Python excels, 2. C and Fortran don't come close to the developer speed of Python. Ideas don't come from using languages like C, but are rather enabled by implementing, prototyping, and iterating quickly and with little resistance from the language, using existing libraries, etc. Python is simply much better in these regards.
- TheRealKing 2y agoFortran cannot be placed with C in the same category of low programming productivity.
- pjmlp 2y agoApparently we could do research work before Python was invented.