Neural nets are great at language and image processing, compression, search and probabilistic prediction problems. In many ways they open up new possibilities f

Neural nets are great at language and image processing, compression, search and probabilistic prediction problems. In many ways they open up new possibilities for what software can do at all. But replace existing solutions? How on earth do you create a single neural net architecture in say pytorch that turns into a CRUD app with oauth, email verification, css+html, support for parsing some ancient xml format uploaded by the user and stored in S3 along with various database connections? I can't even imagine what a labelled or unlabelled training set for that would look like. (Unless you're talking about making GPT write the code that does all of the above – I'm sure people are working on that – but unless I'm completely misreading Karpathy's article that's not at all what he intended; IIUC he's talking about programmers making datasets for specific problems + simple neural net architectures to train on that dataset. If ChatGPT wrote some Java code for you, you can't make it go faster by removing half the nodes, which nodes would those be? It definitely won't go faster if you send the same prompt to a lobotomised ChatGPT)