4 ms·
We don't need more data, we need independent LLMs with slight randomness.
by trueismywork 2y ago
We don't need more data, we need independent LLMs with slight randomness.
- dahart 2y agoAren’t independent LLMs just more data? Some AI researchers are arguing that more data is the primary thing that got us this far, and the primary thing that will improve AI from here. Here’s an example (and a very good talk whether you believe my summary) https://www.youtube.com/live/a13aqr07tJ4?si=FZO5m_XzrfpDhyQP https://www.youtube.com/live/a13aqr07tJ4?si=FZO5m_XzrfpDhyQP As a part-time generative artist for several decades, a user of Monte-Carlo methods on a daily basis, and author of some papers on the topic, I personally believe that randomness is not a good answer to anything creative. Randomness is boring, and average. Randomness only helps you when you have a high quality Markov model constraining what your RNG chooses between, and that’s more or less all LLMs actually are. Adding more randomness to creative works in general makes creative works muddy and lowers quality, it needs to be guided. Randomness is a useful tool, but is widely misunderstood IMO and not very effective at brainstorming style exploration; brainstorming is about solving problems in interesting ways, i.e., there is reasoning behind it, not slightly more random word salad.