4 ms·
> In 10 years what will the LLMs train on? Why do people think this is necessary? When you learn new things, like bicycling, you don't start with relearning ho
by spiderfarmer 10mo ago
> In 10 years what will the LLMs train on?
Why do people think this is necessary? When you learn new things, like bicycling, you don't start with relearning how to walk.
- okaleniuk 10mo agoWalking doesn't change as often as C++ standard.
- jakeydus 10mo agoThe issue here is borrowing from the future to pay for the present. The bicycle analogy (unless I'm missing something huge here) does not seem relevant at all.
- spiderfarmer 10mo agoWhy do you think source code and documentation are not enough for an LLM to train on?
- xeromal 10mo agoI think they are positing that LLMs do not produce new thought. If a new framework (super magic new framework) is released, current LLMs will not be able to help.
- ndriscoll 10mo agoWhy wouldn't the LLM just read the source of the framework to answer questions directly? That's how I do things as a human. Given the appropriate background knowledge (which current LLMs are already extremely capable with), it should be pretty easy to understand what it's doing, and if it's not easy to understand the source, it's probably a bad framework. I don't expect an LLM to have deep inbuilt knowledge of libraries. I expect it to be able to use a language server to find the right definitions and load them into context as needed. I expect it to have very deep inbuilt knowledge of computer science and architecture to make sense of everything it sees.
- miningape 10mo agoBecause LLMs do not work like that - there's no "understanding" the source and answering questions, it simply "finds" similar results in its training data (matching it with the context) and regurgitates (some part of) it (+ other "noise"). Meaning as technology evolves and does things in novel ways, without explainers annotating it the LLM won't have anything to draw on - reducing the quality of answers. Which brings us full circle, what will companies use as training data without answers in places like SO?
- spiderfarmer 10mo agoYou’re talking about ChatGPT 2.0 and severely underestimate the capabilities of today’s models.
- miningape 10mo agoIt's well known that even current LLMs do not perform well on logic games when you change the names / language used. i.e. try asking it to swap the meanings of the words red and green and ask it to describe the colors in a painting and analyse it with color theory - notice how quickly the results degrade, often attributing "green" qualities to "red" since it's now calling it "green". What this shows us is that training data (where the associations are made) plays a significant role in the level of answer an LLM can give, no matter how good your context is (at overriding the associations / training data). This demonstrates that training data is more important (for "novel" work) than context is.
- Eisenstein 10mo agoWrite "This sentence is green." in red sharpie and "This sentence is red" in green sharpie on a piece of paper. Show it to someone briefly and then hide it. Ask them what color the first sentence said it was and what color the second sentence was written in. Another one: ask a person to say 'silk' 5 times, then ask them what cows drink. Exploiting such quirks only tells you that you can trick people, not what their capabilities are.
- alistairSH 10mo agoNot sure I understand... How will CharGPT/CoPilot/whatever learn about the next great front-end framework? The LLMs know about existing frameworks by learning on existing content (from StackOverflow and elsewhere). If StackOverflow (and elsewhere) go away, there's nothing to provide a training material.