5 ms·
I believe in search, but it only works if you have an appropriate search space. Chess has a well-defined space but the everyday world does not. The trick is ena
by jmugan 2y ago
I believe in search, but it only works if you have an appropriate search space. Chess has a well-defined space but the everyday world does not. The trick is enabling an algorithm to learn its own search space through active exploration and reading about our world. I'm working on that.
- jhawleypeters 2y agoOh nice! The one thing that confused me about this article was what search space the author envisioned adding to language models.
- kragen 2y agothat's interesting; are you building a sort of 'digital twin' of the world it's explored, so that it can dream about exploring it in ways that are too slow or dangerous to explore in reality?
- jmugan 2y agoThe goal is to enable it to model the world at different levels of abstraction based on the question it wants to answer. You can model car as an object that travels fast and carries people, or you can model it down to the level of engine parts. The system should be able to pick the level of abstraction and put the right model together based on its goals.
- kragen 2y agoso then you can search over configurations of engine parts to figure out how to rebuild the engine? i may be misunderstanding what you're doing
- jmugan 2y agoYeah, you could. Or you could search for shapes of different parts that would maximize the engine efficiency. The goal is to simultaneously build a representation space and a simulator so that anything that could be represented could be simulated.
- paraschopra 2y agoHave you written about this anywhere? I’m also very interested in this. I’m at the stage where I’m exploring how to represent such a model/simulator. The world isn’t brittle, so representing it as a code / graph probably won’t work.
- jmugan 2y agoRight. You have to represent it as something that can write code/graphs. I say a little bit here https://www.jonathanmugan.com/WritingAndPress/presentations/2023_data_day_someone_to_talk_to.pdf https://www.jonathanmugan.com/WritingAndPress/presentations/...