3 ms·
Sure - there are indeed millions of lines of code if you include the operating system, the Python interpreter, PyTorch, CUDA etc. But... you don't have to writ
by simonw 3y ago
Sure - there are indeed millions of lines of code if you include the operating system, the Python interpreter, PyTorch, CUDA etc.
But... you don't have to write that code. A research group constructing a brand new LLM really does only need a few hundred lines of code for the core training and inference. They need to spend their time on the data instead.
That's what makes LLMs "easy" to build. You don't need to write millions of lines of custom code to build an "AI", which I think is pretty surprising.