4 ms·

> I still don't think if you trained an LLM with every pre-Newton/Liebniz algebra/geometry/trig text available, it could create calculus. (I'm open to being pro

> I still don't think if you trained an LLM with every pre-Newton/Liebniz algebra/geometry/trig text available, it could create calculus. (I'm open to being proven wrong.) The experiment is feasible. If it were performed and produced a positive result, what would it imply/change about how you see LLMs?

sumeno 5mo ago
How are you going to train a frontier level llm with no references to post 1700 mathematics?
- bjt 5mo ago
  "frontier level" is doing a lot of work there, but the idea would be to only feed it earlier sources. There are people working on this. e.g. https://github.com/haykgrigo3/TimeCapsuleLLM https://github.com/haykgrigo3/TimeCapsuleLLM
  - lovecg 5mo ago
    The problem is the amount of data with that cutoff is really minuscule to produce anything powerful. You might be able to generate a lot of 1700s sounding data, you’d have to be careful not to introduce newer concepts or ways of thinking in that synthetic data though. A lot of modern texts talk about rates of change and the like in ways that are probably influenced by preexisting knowledge of calculus.
    - NewJazz 5mo ago
      Doesn't it prove GP's point then, that LLMs themselves simply aren't capable of creating/proving new theories and axioms?
      - codebje 5mo ago
        Without passing opinion on GP's point, I think that just proves it's hard to establish a data set that doesn't bias toward the result you're hoping to find.
- kelseyfrog 5mo ago
  Time cutoff LLMs are regularly posted to HN. It takes just one success to prove feasibility. Besides, we can forecast our thoughts and actions to imagined scenarios unconditioned on their possibility. Something doesn't have to be possible for us to imagine our reactions.
- anthk 5mo ago
  Archimede was close.
pegasus 5mo ago
GP was stating that they don't believe this would happen (I don't either), but also to make the point that it's a falsifiable view. (At least in theory. In practice, there probably won't even be enough historical text to train an LLM on). No, I don't think it would be falsified. Asking what if I'm wrong is kind of redundant. If I'm wrong, I'm wrong, duh.
nathan_compton 5mo ago
I don't think its really feasible - there just isn't enough training data before calculus. I would guess all the mathematical and philosophical texts available to Newton and Leibniz would fit on a CD-ROM with loads of space to spare.

sumeno 5mo ago

How are you going to train a frontier level llm with no references to post 1700 mathematics?

bjt 5mo ago
"frontier level" is doing a lot of work there, but the idea would be to only feed it earlier sources. There are people working on this. e.g. https://github.com/haykgrigo3/TimeCapsuleLLM https://github.com/haykgrigo3/TimeCapsuleLLM
- lovecg 5mo ago
  The problem is the amount of data with that cutoff is really minuscule to produce anything powerful. You might be able to generate a lot of 1700s sounding data, you’d have to be careful not to introduce newer concepts or ways of thinking in that synthetic data though. A lot of modern texts talk about rates of change and the like in ways that are probably influenced by preexisting knowledge of calculus.
  - NewJazz 5mo ago
    Doesn't it prove GP's point then, that LLMs themselves simply aren't capable of creating/proving new theories and axioms?
    - codebje 5mo ago
      Without passing opinion on GP's point, I think that just proves it's hard to establish a data set that doesn't bias toward the result you're hoping to find.
kelseyfrog 5mo ago
Time cutoff LLMs are regularly posted to HN. It takes just one success to prove feasibility. Besides, we can forecast our thoughts and actions to imagined scenarios unconditioned on their possibility. Something doesn't have to be possible for us to imagine our reactions.
anthk 5mo ago
Archimede was close.

pegasus 5mo ago

GP was stating that they don't believe this would happen (I don't either), but also to make the point that it's a falsifiable view. (At least in theory. In practice, there probably won't even be enough historical text to train an LLM on). No, I don't think it would be falsified. Asking what if I'm wrong is kind of redundant. If I'm wrong, I'm wrong, duh.

nathan_compton 5mo ago

I don't think its really feasible - there just isn't enough training data before calculus. I would guess all the mathematical and philosophical texts available to Newton and Leibniz would fit on a CD-ROM with loads of space to spare.