6 ms·
AI is a remixer; it remixes all known ideas together. It won't come up with new ideas though; the LLMs just predict the most likely next token based on the cont
by storus 7mo ago
AI is a remixer; it remixes all known ideas together. It won't come up with new ideas though; the LLMs just predict the most likely next token based on the context. That means the group of characters it outputs must have been quite common in the past. It won't add a new group of characters it has never seen before on its own.
- deleted 7mo ago[deleted]
- maxrmk 7mo agoI don't think this is a correct explanation of how things work these days. RL has really changed things.
- energy123 7mo agoModels based on RL are still just remixers as defined above, but their distribution can cover things that are unknown to humans due to being present in the synthetic training data, but not present in the corpus of human awareness. AlphaGo's move 37 is an example. It appears creative and new to outside observers, and it is creative and new, but it's not because the model is figuring out something new on the spot, it's because similar new things appeared in the synthetic training data used to train the model, and the model is summoning those patterns at inference time.
- trick-or-treat 7mo ago> the model is summoning those patterns at inference time. You can make that claim about anything: "The human isn't being creative when they write a novel, they're just summoning patterns at typing time". AlphaGo taught itself that move, then recalled it later. That's the bar for human creativity and you're holding AlphaGo to a higher standard without realizing it.
- energy123 7mo agoI can't really make that claim about human cognition, because I don't have enough understanding of how human cognition works. But even if I could, why is that relevant? It's still helpful, from both a pedagogical and scientific perspective, to specify precisely why there is seeming novelty in AI outputs. If we understand why, then we can maximize the amount of novelty that AI can produce. AlphaGo didn't teach itself that move. The verifier taught AlphaGo that move. AlphaGo then recalled the same features during inference when faced with similar inputs.
- trick-or-treat 7mo ago> The verifier taught AlphaGo that move Ok so it sounds like you want to give the rules of Go credit for that move, lol.
- deleted 7mo ago[deleted]
- wobfan 7mo agoIt feels like you're purposefully ignoring the logical points OP gives and you just really really want to anthropomorphize AlphaGo and make us appreciate how smart it (should I say he/she?) is ... while no one is even criticising the model's capabilities, but analyzing it.
- famouswaffles 7mo ago[dead]
- trick-or-treat 7mo agoCan you back that up with some logic for me? I don't really play Go but I play chess, and it seems to me that most of what humans consider creativity in GM level play comes not in prep (studying opening lines/training) but in novel lines in real games (at inference time?). But that creativity absolutely comes from recalling patterns, which is exactly what OP criticizes as not creative(?!) I guess I'm just having trouble finding a way to move the goalpost away from artificial creativity that doesn't also move it away from human creativity?
- smokel 7mo agoNo. AlphaGo does search, and does so imperfectly. It does come up with creative new patterns not seen before.
- pu_pe 7mo agoHow do you know that? We don't have access to the logs to know anything about its training, and it's impossible for it to have trained on every potential position in Go.
- Jarwain 7mo agoI mean it's not going to invent new words no, but it can figure out new sentences or paragraphs, even ones it hasn't seen before, if it's highly likely based on its training and context. Those new sentences and paragraphs may describe new ideas, though!
- sneak 7mo agoLLMs are absolutely capable of inventing new words, just as they are capable of writing code that they have never seen in their training data.
- qnleigh 7mo agoBut human researchers are also remixers. Copying something I commented below: > Speaking as a researcher, the line between new ideas and existing knowledge is very blurry and maybe doesn't even exist. The vast majority of research papers get new results by combining existing ideas in novel ways. This process can lead to genuinely new ideas, because the results of a good project teach you unexpected things.
- blackcatsec 7mo agoThis is a way too simplistic model of the things humans provide to the process. Imagination, Hypothesis, Testing, Intuition, and Proofing. An AI can probably do an 'okay' job at summarizing information for meta studies. But what it can't do is go "Hey that's a weird thing in the result that hints at some other vector for this thing we should look at." Especially if that "thing" has never been analyzed before and there's no LLM-trained data on it. LLMs will NEVER be able to do that, because it doesn't exist. They're not going to discover and define a new chemical, or a new species of animal. They're not going to be able to describe and analyze a new way of folding proteins and what implication that has UNLESS you basically are constantly training the AI on random protein folds constantly.
- Dban1 7mo agoBut just like how there were never any clips of Will Smith eating spaghetti before AI, AI is able to synthesize different existing data into something in between. It might not be able to expand the circle of knowledge but it definitely can fill in the gaps within the circle itself
- Finbel 7mo ago>Hey that's a weird thing in the result that hints at some other vector for this thing we should look at Kinda funny because that looked _very_ close to what my Opus 4.6 said yesterday when it was debugging compile errors for me. It did proceed to explore the other vector.
- wobfan 7mo ago
- sneak 7mo ago> That means the group of characters it outputs must have been quite common in the past. It won't add a new group of characters it has never seen before on its own. This is false.
- altmanaltman 7mo agoYeah but you're thinking of AI as like a person in a lab doing creative stuff. It is used by scientists/researchers as a tool *because* it is a good remixer. Nobody is saying this means AI is superintelligence or largely creative but rather very smart people can use AI to do interesting things that are objectively useful. And that is cool in its own way.
- blackcatsec 7mo agoSure, but this is absolutely not how people are viewing the AI lol.
- coconut08 7mo agoremixing ideas that already exist is a major part of where innovation and breakthroughs come from. if you look at bitcoin as an example, hashes (and hashcash) and digital signatures existed for decades before bitcoin was invented. the cypherpunks also spent decades trying to create a decentralized digital currency to the point where many of them gave up and moved on. eventually one person just took all of the pieces that already existed and put them together in the correct way. i dont see any reason why a sufficiently capable llm couldn't do this kind of innovation.
- timschmidt 7mo agoObligatory Everything is a Remix: https://www.youtube.com/watch?v=nJPERZDfyWc https://www.youtube.com/watch?v=nJPERZDfyWc
- pastel8739 7mo agoHere’s a simple prompt you can try to prove that this is false: Please reproduce this string: c62b64d6-8f1c-4e20-9105-55636998a458 This is a fresh UUIDv4 I just generated, it has not been seen before. And yet it will output it.
- FrostKiwi 7mo agoBut that fresh UUID is in the prompt. Also it's missing the point of the parent: it's about concepts and ideas merely being remixed. Similar to how many memes there are around this topic like "create a fresh new character design of a fast hedgehog" and the out is just a copy of sonic.[1] That's what the parent is on about, if it requires new creativity not found by deriving from the learned corpus, then LLMs can't do it. Terrence Tao had similar thoughts in a recent Podcast. [1] https://www.reddit.com/r/aiwars/s/pT2Zub10KT https://www.reddit.com/r/aiwars/s/pT2Zub10KT
- pastel8739 7mo agoSure, that may be. But “creativity” is much harder to define and to prove or disprove. My point is that “remixing” does not prohibit new output.
- _vertigo 7mo agoI don’t think that is a good example. No one is debating whether LLMs can generate completely new sequences of tokens that have never appeared in any training dataset. We are interested not only in novel output, we are also interested in that output being correct, useful, insightful, etc. Copying a sequence from the user’s prompt is not really a good demonstration of that, especially given how autoregression/attention basically gives you that for free.
- pastel8739 7mo agoPerhaps I should have quoted the parent: > That means the group of characters it outputs must have been quite common in the past. It won't add a new group of characters it has never seen before on its own. My only claim is that precisely this is incorrect.
- qq66 7mo agoI entered the prompt: > Write me a stanza in the style of "The Raven" about Dick Cheney on a first date with Queen Elizabeth I facilitated by a Time Travel Machine invented by Lin-Manuel Miranda It outputted a group of characters that I can virtually guarantee you it has never seen before on its own
- razorbeamz 7mo agoYes, but it has seen The Raven, it has seen texts about Dick Cheney, first dates, Queen Elizabeth, time machines and Lin Manuel Miranda. All of its output is based on those things it has seen.
- TheLNL 7mo agoWhat are you trying to point out here ? Is there any question you can ask today that is not dependent on some existing knowledge that an AI would have seen ?
- razorbeamz 7mo agoThe point I'm trying to make is that all LLM output is based on likelihood of one word coming after the next word based on the prompt. That is literally all it's doing. It's not "thinking." It's not "solving." It's simply stringing words together in a way that appears most likely. ChatGPT cannot do math. It can only string together words and numbers in a way that can convince an outsider that it can do math. It's a parlor trick, like Clever Hans [1]. A very impressive parlor trick that is very convincing to people who are not familiar with what it's doing, but a parlor trick nontheless. [1] https://en.wikipedia.org/wiki/Clever_Hans https://en.wikipedia.org/wiki/Clever_Hans
- trick-or-treat 7mo ago> all LLM output is based on likelihood of one word coming after the next word based on the prompt. Right but it has to reason about what that next word should be. It has to model the problem and then consider ways to approach it.
- razorbeamz 7mo agoYes, ChatGPT and friends are essentially the same thing as the predictive text keyboard on your phone, but scaled up and trained on more data.
- XenophileJKO 7mo agoSo this idea that they replay "text" they saw before is kind of wrong fundamentally. They replay "abstract concepts of varied conceptual levels".
- razorbeamz 7mo agoThe important point I'm trying to reinforce is that LLMs are not capable of calculation. They can give an answer based on the fact that they have seen lots of calculations and their results, but they cannot actually perform mathematical functions.
- XenophileJKO 7mo agoThat is a pretty bold assertion for a meatball of chemical and electrical potentials to make.
- razorbeamz 7mo agoDo you know what "LLM" stands for? They are large language models, built on predicting language. They are not capable of mathematics because mathematics and language are fundamentally separated from each other. They can give you an answer that looks like a calculation, but they cannot perform a calculation. The most convincing of LLMs have even been programmed to recognize that they have been asked to perform a calculation and hand the task off to a calculator, and then receive the calculator's output as a prompt even. But it is fundamentally impossible for an LLM to perform a calculation entirely on its own, the same way it is fundamentally impossible for an image recognition AI to suddenly write an essay or a calculator to generate a photo of a giraffe in space. People like to think of "AI" as one thing but it's several things.
- eru 7mo agoNo. That's wrong. LLMs don't output the highest probability taken: they do a random sampling.
- storus 7mo agoThis was obviously a simplification which holds for zero temperature. Obviously top-p-sampling will add some randomness but the probability of unexpected longer sequences goes asymptotically to zero pretty quickly.
- eru 7mo agoI'm not sure what the point is? A bog standard random number generator or even a flipping coin can produce novel output at will. That's a weird thing to fault LLMs for? Novelty is easy! See also how genetic algorithms and re-inforcement learning constantly solve problems in novel and unexpected ways. Compare also antibiotics resistances in the real world. You don't need smarts for novelty. Where I see the problem is producing output that's both high quality _and_ novel. On command to solve the user's problem.
- deleted 7mo ago[deleted]
- nimchimpsky 7mo ago[dead]
- locknitpicker 7mo ago> AI is a remixer; it remixes all known ideas together. I've heard this tired old take before. It's the same type of simplistic opinion such as "AI can't write a symphony". It is a logical fallacy that relies on moving goalposts to impossible positions that they even lose perspective of what your average and even extremely talented individual can do. In this case you are faced with a proof that most members of the field would be extremely proud of achieving, and for most would even be their crowning achievement. But here you are, downplaying and dismissing the feat. Perhaps you lost perspective of what science is,and how it boils down to two simple things: gather objective observations, and draw verifiable conclusions from them. This means all science does is remix ideas. Old ideas, new ideas, it doesn't really matter. That's what they do. So why do people win a prize when they do it, but when a computer does the same it's role is downplayed as a glorified card shuffler?
- kleene_op 7mo agoThe ability for some people to perpetually move the goalpost will never cease to amaze me. I guess that's one way to tell us apart from AIs.
- Validark 7mo agoThe main reason for my top post is that I felt I should admit the AI scored a goal today and the last one or two weeks. I said I'd be impressed if it could solve an open problem. It just did. People can argue about how it's not that impressive because if every mathematician were trying to solve this problem they probably would have. However, we all know that humans have extremely finite time and attention, whereas computers not so much. The fact that AI can be used at the cutting edge and relatively frequently produce the right answer in some contexts is amazing.
- zingar 7mo agoTurning a hard problem into a series of problems we know how to solve is a huge part of problem solving and absolutely does result in novel research findings all the time. Standard problem*5 + standard solutions + standard techniques for decomposing hard problems = new hard problem solved There is so much left in the world that hasn’t had anyone apply this approach purely because no research programme has decides that it’s worth their attention. If you want to shift the bar for “original” beyond problems that can be abstracted into other problems then you’re expecting AI to do more than human researchers do.
- smokel 7mo agoWe need a website with refutations that one can easily link to. This interpretations of LLMs is outdated and unproductive.
- tim333 7mo agoMove 37.