3 ms·
I've wondered that too. If we trained an LLM on only the scientific content that predates Einstein could it come up with general relativity? How much coaxing
by Teever 2y ago
I've wondered that too. If we trained an LLM on only the scientific content that predates Einstein could it come up with general relativity?
How much coaxing would it require to get it there? What would that process be like?
Can we learn from that and get an LLM trained on all scientific material pre Einstein and post to discover new physics stuff from what we learn from that process?
- staticman2 2y ago>> Can we learn from that and get an LLM trained on all scientific material pre Einstein and post to discover new physics stuff from what we learn from that process? The parts of an LLM that teach it language is not disconnected from the parts that teach it facts. Good luck teaching it language with only pre Einstein data.
- Teever 2y agoWhy can't you just omit all modern content related to physics? There's lots of training data that doesn't mention any of Einstein's work.
- staticman2 2y agoMaybe you could, but any influence these concepts had on general culture might still leak in.
- Teever 2y agoTo me that's an interesting feature, not necessarily a bug. How much would have to leak in to guide the LLM to the right conclusions? Is it a matter of quantity or quality? Can we successfully eliminate all leakage?