5 ms·

I'd be curious what an LLM would be able to accomplish if it was given all pre-modern texts we have access to. Would it be able to come up with ideas we know as

by uludag 2y ago

I'd be curious what an LLM would be able to accomplish if it was given all pre-modern texts we have access to. Would it be able to come up with ideas we know as modern (political theories, philosophical ideas, scientific theories, etc.)? I have a hard time believing that if we fed an LLM all prehistoric speech uttered from humans that no matter what, it would never escape the paradigms of those people. If that is the case, than would relying on LLMs just get us stuck in our own paradigms and prevent true "progress"?

Teever 2y ago
I've wondered that too. If we trained an LLM on only the scientific content that predates Einstein could it come up with general relativity? How much coaxing would it require to get it there? What would that process be like? Can we learn from that and get an LLM trained on all scientific material pre Einstein and post to discover new physics stuff from what we learn from that process?
- staticman2 2y ago
  >> Can we learn from that and get an LLM trained on all scientific material pre Einstein and post to discover new physics stuff from what we learn from that process? The parts of an LLM that teach it language is not disconnected from the parts that teach it facts. Good luck teaching it language with only pre Einstein data.
  - Teever 2y ago
    Why can't you just omit all modern content related to physics? There's lots of training data that doesn't mention any of Einstein's work.
    - staticman2 2y ago
      Maybe you could, but any influence these concepts had on general culture might still leak in.
      - Teever 2y ago
        To me that's an interesting feature, not necessarily a bug. How much would have to leak in to guide the LLM to the right conclusions? Is it a matter of quantity or quality? Can we successfully eliminate all leakage?
deisteve 2y ago
i just read this paper on arxiv and im still trying to wrap my head around the implications so the authors got over 100 nlp researchers to write down novel research ideas and then had them review ideas generated by a large language model (llm) without knowing which ones were human generated and which ones were from the llm and the results are pretty fascinating the llm generated ideas were actually judged to be more novel than the human ones p value < 0.05 but slightly weaker in terms of feasibility i mean whats the point of having a novel idea if its not feasible right but still its pretty cool that the llm can come up with stuff that humans havent thought of before and the authors are saying that this study highlights some of the open problems in building research agents that can generate novel ideas like the llm was bad at self evaluation it couldnt tell which of its own ideas were good or bad and they also found that the llm generated ideas that were too similar to each other lacking diversity i mean thats not surprising right llms are trained on huge datasets but theyre still just pattern recognition machines they dont really understand the context or the implications of what theyre generating but heres the thing novelty is hard to judge even for experts i mean how do you even define novelty is it just something that nobody has thought of before or is it something that challenges our current understanding of the world and the authors are proposing a follow up study where they actually have researchers execute these ideas into full projects to see if the novelty and feasibility judgements actually translate into meaningful differences in research outcomes which is a great idea i mean thats the only way we can really know if these llms are useful for accelerating scientific discovery or not anyway im rambling on now but i just think this is a really interesting area of research and im excited to see where it goes can we really use llms to accelerate scientific discovery and what are the limitations of these models and how can we overcome them etc etc
- karmakurtisaani 2y ago
  Your output would benefit from proper punctuation. Are you using some kind of speech-to-text application? Certainly looks like it. Interesting comment nevertheless, if not somewhat difficult to parse.
onemoresoop 2y ago
LLM are just large language models, they're not capable of thinking or coming up with novel ideas. They could enable people to do so by providing food for brainstorming.
- mkaic 2y ago
  > "...they're not capable of thinking or coming up with novel ideas." This claim is unfalsifiable given common definitions of "thinking" and "coming up with novel ideas".
  - jonathaneunice 2y ago
    But it is verifiable. We could quibble over "who judges novelty," but I bet if there were regular examples of it doing so, and there were some community agreement the ideas were indeed suitably novel, we'd pretty quickly shout "existence proof!" and be done.
  - lostmsu 2y ago
    No, it is very much demonstrably false. Here's a simple program that does not even do LLMs that will trivially enumerate all ideas (broken UTF8 handling omitted for brevity): for (var numeral = new BigInteger(0); ; numeral++) WriteLine(UTF8.GetString(numeral.ToByteArray()); For any given idea in English LLMs will certainly get it faster.
- janalsncm 2y ago
  Anytime I see an argument of the form X is just Y an alarm bell goes off in my head. While it is descriptive of the components, these arguments ignore the possibility of emergent complexity. Sometimes things are more than just the sum of their parts. The sun is just a bunch of hydrogen. A computer is just a bunch of transistors. Humans are just a bunch of cells. An artificial neural network is just a bunch of matrix multiplications. I personally think LLMs are extremely limited and overhyped. But this form of argument seems incorrect to me since it can be used to argue that LLMs cannot do things we already know they can do.
  - 2y ago
mensetmanusman 2y ago
An LLM could come up with an infinite amount of new theories and ideas, of which >99% would be meaningless, if given enough time and energy. Human’s ability is in guiding the prompt (of AIs, and the mind) on what is worth knowing and which hypotheses are worth testing. Choosing an action from the countable infinite number of actions is the general framework of free will.
- ninetyninenine 2y ago
  I think the amazing thing about the LLM is that it's not a random text generator so it's not 99% meaningless. I give it only 50% meaningless.
  - mensetmanusman 2y ago
    With nearly unlimited energy, if you set up an indefinite while-loop asking it to continue to expand and make new theories on the same token vector it would approach >99% meaninglessness.
    - ninetyninenine 2y ago
      Yeah that's because the only thing changing is some random seed. It's like looking at f(0) = 1 and f(0.00001) = 1.00001 and saying that the function can't produce anything novel because the answer is always less than 2. Hint: try f(99999). Of course it's all meaningless because most of the input and output is virtually identical. Vary the input heavily and then you will approach 50%. Of course this is assuming the token vector is truly random in terms of subject matter.
      - mensetmanusman 2y ago
        It’s not just a random seed though, this would be attempting to squeeze everything out of the LLM network with all the seeds :)
        2y ago
eagerpace 2y ago
Is this how our simulation started?
fasa99 2y ago
It's a brilliant idea, but LLMs are predicated upon the massive data of the internet. That being said, you could take a pre-2018 dataset and test it for discoveries/insights circa 2019 or 2020 and see how well it performs.

by uludag 2y ago

Teever 2y ago

I've wondered that too. If we trained an LLM on only the scientific content that predates Einstein could it come up with general relativity? How much coaxing would it require to get it there? What would that process be like? Can we learn from that and get an LLM trained on all scientific material pre Einstein and post to discover new physics stuff from what we learn from that process?

staticman2 2y ago
>> Can we learn from that and get an LLM trained on all scientific material pre Einstein and post to discover new physics stuff from what we learn from that process? The parts of an LLM that teach it language is not disconnected from the parts that teach it facts. Good luck teaching it language with only pre Einstein data.
- Teever 2y ago
  Why can't you just omit all modern content related to physics? There's lots of training data that doesn't mention any of Einstein's work.
  - staticman2 2y ago
    Maybe you could, but any influence these concepts had on general culture might still leak in.
    - Teever 2y ago
      To me that's an interesting feature, not necessarily a bug. How much would have to leak in to guide the LLM to the right conclusions? Is it a matter of quantity or quality? Can we successfully eliminate all leakage?

deisteve 2y ago

i just read this paper on arxiv and im still trying to wrap my head around the implications so the authors got over 100 nlp researchers to write down novel research ideas and then had them review ideas generated by a large language model (llm) without knowing which ones were human generated and which ones were from the llm and the results are pretty fascinating the llm generated ideas were actually judged to be more novel than the human ones p value < 0.05 but slightly weaker in terms of feasibility i mean whats the point of having a novel idea if its not feasible right but still its pretty cool that the llm can come up with stuff that humans havent thought of before and the authors are saying that this study highlights some of the open problems in building research agents that can generate novel ideas like the llm was bad at self evaluation it couldnt tell which of its own ideas were good or bad and they also found that the llm generated ideas that were too similar to each other lacking diversity i mean thats not surprising right llms are trained on huge datasets but theyre still just pattern recognition machines they dont really understand the context or the implications of what theyre generating but heres the thing novelty is hard to judge even for experts i mean how do you even define novelty is it just something that nobody has thought of before or is it something that challenges our current understanding of the world and the authors are proposing a follow up study where they actually have researchers execute these ideas into full projects to see if the novelty and feasibility judgements actually translate into meaningful differences in research outcomes which is a great idea i mean thats the only way we can really know if these llms are useful for accelerating scientific discovery or not anyway im rambling on now but i just think this is a really interesting area of research and im excited to see where it goes can we really use llms to accelerate scientific discovery and what are the limitations of these models and how can we overcome them etc etc

karmakurtisaani 2y ago
Your output would benefit from proper punctuation. Are you using some kind of speech-to-text application? Certainly looks like it. Interesting comment nevertheless, if not somewhat difficult to parse.

onemoresoop 2y ago

LLM are just large language models, they're not capable of thinking or coming up with novel ideas. They could enable people to do so by providing food for brainstorming.

mkaic 2y ago
> "...they're not capable of thinking or coming up with novel ideas." This claim is unfalsifiable given common definitions of "thinking" and "coming up with novel ideas".
- jonathaneunice 2y ago
  But it is verifiable. We could quibble over "who judges novelty," but I bet if there were regular examples of it doing so, and there were some community agreement the ideas were indeed suitably novel, we'd pretty quickly shout "existence proof!" and be done.
- lostmsu 2y ago
  No, it is very much demonstrably false. Here's a simple program that does not even do LLMs that will trivially enumerate all ideas (broken UTF8 handling omitted for brevity): for (var numeral = new BigInteger(0); ; numeral++) WriteLine(UTF8.GetString(numeral.ToByteArray()); For any given idea in English LLMs will certainly get it faster.
janalsncm 2y ago
Anytime I see an argument of the form X is just Y an alarm bell goes off in my head. While it is descriptive of the components, these arguments ignore the possibility of emergent complexity. Sometimes things are more than just the sum of their parts. The sun is just a bunch of hydrogen. A computer is just a bunch of transistors. Humans are just a bunch of cells. An artificial neural network is just a bunch of matrix multiplications. I personally think LLMs are extremely limited and overhyped. But this form of argument seems incorrect to me since it can be used to argue that LLMs cannot do things we already know they can do.
- 2y ago

mensetmanusman 2y ago

An LLM could come up with an infinite amount of new theories and ideas, of which >99% would be meaningless, if given enough time and energy. Human’s ability is in guiding the prompt (of AIs, and the mind) on what is worth knowing and which hypotheses are worth testing. Choosing an action from the countable infinite number of actions is the general framework of free will.

ninetyninenine 2y ago
I think the amazing thing about the LLM is that it's not a random text generator so it's not 99% meaningless. I give it only 50% meaningless.
- mensetmanusman 2y ago
  With nearly unlimited energy, if you set up an indefinite while-loop asking it to continue to expand and make new theories on the same token vector it would approach >99% meaninglessness.
  - ninetyninenine 2y ago
    Yeah that's because the only thing changing is some random seed. It's like looking at f(0) = 1 and f(0.00001) = 1.00001 and saying that the function can't produce anything novel because the answer is always less than 2. Hint: try f(99999). Of course it's all meaningless because most of the input and output is virtually identical. Vary the input heavily and then you will approach 50%. Of course this is assuming the token vector is truly random in terms of subject matter.
    - mensetmanusman 2y ago
      It’s not just a random seed though, this would be attempting to squeeze everything out of the LLM network with all the seeds :)
      - 2y ago

eagerpace 2y ago

Is this how our simulation started?

fasa99 2y ago

It's a brilliant idea, but LLMs are predicated upon the massive data of the internet. That being said, you could take a pre-2018 dataset and test it for discoveries/insights circa 2019 or 2020 and see how well it performs.