3 ms·
The idea would to generate a false "reality" for the LLM to learn about. You would randomly generate a system of rules, use those rules to generate text, and th
by f33d5173 2y ago
The idea would to generate a false "reality" for the LLM to learn about. You would randomly generate a system of rules, use those rules to generate text, and then train the llm to predict the text. The goal would be to get it to stop encoding the reality proper in its weights, and focus on learning to pick up what reality looks like very quickly from text.
- sillysaurusx 2y agoBonus points for one of the most delightfully creative ideas I’ve heard in some time. I don’t think it will work (the space of "not reality" is superexponentially larger than the space of "this describes reality") but I’m just happy to be thinking about nonstandard ML again. (I’ve dubbed this sort of thing “nonstandard ML" since, like you, I have a fondness for thinking of unorthodox solutions that seem plausible.)