3 ms·
AI trains AI already, agents are happy to spin up real training pipelines for deep learning or regression models or whatever you want right? I guess the advanta
by callamdelaney 3mo ago
AI trains AI already, agents are happy to spin up real training pipelines for deep learning or regression models or whatever you want right? I guess the advantage to your project is that it provides a framework to allow the agent to access extra compute?
- Danau5tin 3mo agoYes I'd heard the labs (Anthropic mostly) speaking about LLMs training LLMs, so I wanted to make things a little more concrete and test it out myself! Essentially you are correct though, my framework allows the agent access to compute, but also the agent itself is being trained to become better at training models with that compute.
- vorticalbox 3mo agoi remeber reading in one of the release blog posts that that version was the "first that codex helped train"
- Danau5tin 3mo agoYes exactly, and also there is auto-research by Karpathy, Fable-5 on parameter golf etc which point to the direction of highly-assisted / autonomous AI training as the future. Wild.
- amelius 3mo agoI guess more important than LLMs training LLMs (which is mostly the same code over and over) is LLMs cleaning and curating training data.
- Danau5tin 3mo agoVery good point. Sifting through noisy data and creating curated datasets could be of great value. Perhaps worth a project by itself!
- throwa356262 3mo agoNot sure how much of it was marketing but M2.7 supposedly trained itself https://www.minimax.io/blog/minimax-m27 https://www.minimax.io/blog/minimax-m27
- Danau5tin 3mo agoVery cool! Thank you