3 ms·
How come models can be so small now? I don't know a lot about AI, but is there an ELI5 for a software engineer that knows a bit about AI? For context: I've mad
by mettamage 2y ago
How come models can be so small now? I don't know a lot about AI, but is there an ELI5 for a software engineer that knows a bit about AI?
For context: I've made some simple neural nets with backprop. I read [1].
[1] http://neuralnetworksanddeeplearning.com/ http://neuralnetworksanddeeplearning.com/
- blharr 2y agoYou can find the phi-4 technical report [here](https://www.microsoft.com/en-us/research/uploads/prod/2024/12/P4TechReport.pdf https://www.microsoft.com/en-us/research/uploads/prod/2024/1...) The brief of it is by curating a smaller synthetic dataset of high quality from textbooks, problem sets, etc. instead of dumping a massive dataset with tons of information.