I disagree. I mean it's not trivial but it is completely within reach of a single person. The only part you'd really need to lean on libraries for would be data

I disagree. I mean it's not trivial but it is completely within reach of a single person. The only part you'd really need to lean on libraries for would be data loading (e.g. jpeg). The core neural net stuff really is not that complex, even in the latest architectures like transformers or diffusion models. Look at stuff like George Hotz's tinygrad or Andrej Karpathy's makemore.