8 ms·
Can LLMs invent better ways to train LLMs?
- bugbuddy 2y agoBetteridge's law of headlines: no
- dietr1ch 2y agoYou'll need the following law too, No clickbaity article is worth reading.
- blowski 2y agoPlease don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something. (From the guidelines)
- binary132 2y agoFine then: “don’t write articles with question headlines or people can safely assume the answer is no.”
- dietr1ch 2y agoI like the idea of respect where this comes from, but I think that titles have been perverted by SEO way too much and at some point I need writers to stop trying to lure me for my own good. I wish we can get descriptive, boring, but accurate titles back.
- blowski 2y agoDescriptive titles have their place, but so do titles that arouse curiousity and need you to read TFA.
- rpigab 2y agoI hate clickbait, but this doesn't look like this one is, because it's not misleading, it's not trying to hide something to get you to click, and does not lure you with information that's not in the article.
- mdp2021 2y agoThe uninformed preliminary answer instead is yes: loaded dice can produce better values. The article is about using LLMs in an evolutionary framework to design better algorithms for LLM advancement, with particular occasional regard to preference optimization algorithms - and guess what, it seems it worked.
- ChuckMcM 2y agoA better question is "Can LLMs invent anything?" Don't misunderstand, building systems models using existing system response as a way of analyzing those systems is a useful methodology and it makes some things otherwise tedious things not so tedious. Much like "high level" languages removed the tedium of writing in assembly code. But for the same reason that a compiler won't emit a new, more powerful, CPU instruction in its code generator, LLMs don't generate previously unseen system responses.
- mdp2021 2y ago> Can LLMs invent anything Can they propose a working novelty: -- after deep thought about idea soundness, probably not at this stage -- through cycles of trials, not knowing exactly why - probably yes After all, your hammer needs not intelligence.
- steve1977 2y agoBut my hammer is totally useless without me as a human using it and telling it exactly what to do.
- mdp2021 2y agoYes, exactly. Tools perform without "knowing" the purpose. Unintelligent yet effective. So, "perform a selection over the enumerated combinations in the solutions space" works without the process being further sophisticated. It works as much as it can - as a preparation of data until the stage in which intelligence is required. We have been doing it since a while; simulated annealing, genetic algorithms... Dumb hammers in a way, encoding an action from an intelligent operator, and providing an effective aid when under intelligent control.
- paulddraper 2y agoWhat are inventions other than combinations of existing patterns? What is the human mind if not a computer? What is the universe if not repeated regurgitations of the four fundamental forces and 12 particles?
- pineaux 2y agoSo it does seem to work. That's not clickbait then?
- deleted 2y ago[deleted]
- teo_zero 2y agoI'm sure LLMs can optimize the training of other LLMs (either by inventing new ways or fine tuning existing ones). But we can't predict whether this will result in a giant's leap in the field, or just small increments. That's the definition of singularity, isn't it?
- seydor 2y agocan LLMs optimize anything?
- spiderfarmer 2y agoThey optimised my coding speed for sure.
- cqqxo4zV46cp 2y agoYes.
- s1gsegv 2y agoAbsolutely. LLMs get a lot of hate but sincerely, GPT-4o can be given a hunk of code (one function/small class worth) and told to find optimization opportunities, and it will do a great job, especially considering the 30s it takes to ask, It’s not perfect, but it understands lock-free algorithms, branch prediction, can tell you which memory order to use for atomic operations if you’re using too strong of an ordering, AND it will catch silly bugs at the same time. I had a bounds check in a lock-free algorithm I was optimizing, which equated to if(idx < start && idx >= end) return false, and it mentioned that error while optimizing. This guy was really putting it through its paces on already highly optimized code in an esoteric architecture (Nintendo 64) and it still found a few things: https://youtu.be/20s9hWDx0Io https://youtu.be/20s9hWDx0Io
- hlkcrcck 2y agoOf course, they can invent anything. A better question is how efficient? Because even with brute force you can invent anything: https://libraryofbabel.info/ https://libraryofbabel.info/
- jasfi 2y agoThey are very useful when ideating with a human. On their own they could veer off into uncertain territory, and likely make mistakes obvious to humans.
- luke-stanley 2y agoThe project sounds quite interesting but I'm not sure running it is going to work! The code `gpt_model = "gpt4_20231230_1106preview"` is not using a valid model name as best as I can tell, so it seems unlikely to work - from https://github.com/SakanaAI/DiscoPOP/blob/main/scripts/launch_evo.py#L15 https://github.com/SakanaAI/DiscoPOP/blob/main/scripts/launc... Unusually, the issue section doesn't exist so I can't provide feedback to them that way. But luchris429's repo does have it so will do so there. Maybe it's dead code. Still, it's wrong.
- luchris429 2y agoAuthor here! Thanks for pointing that out. The correct model name is indeed "gpt-4" instead of "gpt_model = 'gpt4_20231230_1106preview'". We were previously using an Azure endpoint, which is why the model name is different. While I understand the frustration, I assure you that the rest of the code is functional. This was a simple oversight and should be a trivial fix. I appreciate your feedback and understanding.
- rulalala 2y agoWould that not be a form of self consciousness?
- brokenmachine 2y agoCan monkeys with typewriters invent better ways to train monkeys with typewriters? Yes, but you may need a lot of monkeys.