3 ms·
the recent episode of Machine Learning Street Talk on control theory for LLMs sounds like it's thinking in this direction. Say you have 100k agents searching th
by cpill 2y ago
the recent episode of Machine Learning Street Talk on control theory for LLMs sounds like it's thinking in this direction.
Say you have 100k agents searching through research papers, and then trying every combination of them, 100k^2, to see if there is any synergy of ideas, and you keep doing this for all the successful combos... some of these might give the researchers some good ideas to try out.
I can see it happening, if they can fine tune a model that becomes good at idea synergy.
but then again real creativity is hard
- _lyxd 2y agoHow would one finetune for "idea synergy"?
- cpill 2y agoYou'd need an example set, like all FT'ing
- _lyxd 2y agoIdea synergy is Ill defined, creating a consistent, high quality dataset of the scale required is not easy at all. What is the data, pairs of syngeristic papers? Or are you embedding each paper and finding the closest in "idea space"? LLMs are apt for RAG/fuzzy search. Learning a relevant embedding space that has the fidelity where distances have an interpretation of "idea symergy" is not like "semantic meaning" (I.e. RAG embedding space), otherwise you'd simply get similar papers on the same topics as "synergistic" Maybe I'm confused as to what you mean by idea synergy.