3 ms·
Data sharing agreements permitting, today's inference runs can be tomorrow's training data. Presumably the models are good enough at labeling promising chains o
by lxgr 8mo ago
Data sharing agreements permitting, today's inference runs can be tomorrow's training data. Presumably the models are good enough at labeling promising chains of thought already.
I could totally imagine "free" inference for researchers under the condition that the reasoning traces get to be used as future training data.
- mccoyb 8mo agoAgreed, there's no doubt this will happen. It's likely already happening (it feels safe to assume that Anthropic is curating data from the data they record from Claude Code?) As far as I understand RL scaling (we've already maxxed out RLVR), these machines only get better as long as they have expert reasoner traces available. Having an expert work with an LLM and successfully solve a problem is high signal data, it may be the only path forward? My prior is that these companies will take this data without asking you as much as they can.
- lxgr 8mo agoExactly, or functionally equivalently, asking you in paragraph 37 of a 120-page PDF (bonus points: in an agreement update). And importantly, this can be cross-lab/model too. I suspect there's a reason why e.g. Google has been offering me free Claude inference in Google Antigravity on a free plan...
- the_af 8mo ago> Data sharing agreements permitting, today's inference runs can be tomorrow's training data. Presumably the models are good enough at labeling promising chains of thought already. Wouldn't this lead to model collapse?
- littlestymaar 8mo agoNot necessarily, as exhibited by the massive success of artificial data.
- the_af 8mo agoCould you elaborate?
- nhecker 8mo agoEDIT: probably not relevant, after re-re-reading the comment in question. Presumably littlestymaar is talking about all the LLM-generated output that's publicly available on the Internet (in various qualities but significant quantity) and there for the scraping.
- littlestymaar 8mo agoNo, I'm talking about generating data for the purpose of training. The latest HF paper on the subject offers a great intro to the technique: https://huggingface.co/spaces/HuggingFaceFW/finephrase https://huggingface.co/spaces/HuggingFaceFW/finephrase
- littlestymaar 8mo agoFor what we know, most AI labs have used a majority of artificially data since 2023. I had a discussion about a year ago with a researcher at Kyutai and they told me their lab was spending an order of magnitude more compute in artificial data generation than what they spent in training proper. I can't tell if that ratio applies to the industry as a whole, but artificial datasets are the cornerstone of modern AI training.
- suddenlybananas 8mo agoI find this very surprising, do you have any papers on the kinds of techniques that they use?
- littlestymaar 8mo agoThe most well know early paper is probably Textbooks are all you need. The latest paper from HF on synthetic datasets is a good starting point I think: https://huggingface.co/spaces/HuggingFaceFW/finephrase https://huggingface.co/spaces/HuggingFaceFW/finephrase The intro contains a few paper that may interest you as well.
- nhecker 8mo agoThe site arena.ai does exactly this already, as far as I can tell. (In addition to the whole ranking thing.)