4 ms·
My question is: how could this idea be adapted to our current ML models? For example, an LLM just tries to predict the next token when requested. How does one
by warning26 4y ago
My question is: how could this idea be adapted to our current ML models?
For example, an LLM just tries to predict the next token when requested. How does one go from "next token please" to something that "dreams"?
- jgilias 4y agoLoks like the current iteration of LLMs are very good when it comes to being able to hallucinate!
- midlightdenight 4y agoI’m not sure if dreaming is the right word here, but the recent ML model Dreamer v3 [0] “dreams”. This is more akin to thinking ahead since it’s really just predicting outcomes using its world model. It’s possible they could try training the model on some hallucinated scenarios to avoid over fitting, but I’m no ML researcher. [0] https://danijar.com/project/dreamerv3/ https://danijar.com/project/dreamerv3/
- jonnycomputer 4y agoWell, the idea comes from ML in the first place, and has been incorporated into some theories of brain function.