4 ms·
Wtf, this has a name now? I thought of this exact idea literally months ago but never had the time to do any experiments on it. At the time I dismissed it as p
by 2001zhaozhao 6mo ago
Wtf, this has a name now? I thought of this exact idea literally months ago but never had the time to do any experiments on it.
At the time I dismissed it as potentially being incredibly expensive for the improvement you do get, and runs into typical pitfalls of evolutionary algorithms (in the same way evolution doesn't let an organism grow a wheel, your LLM evolution algorithm will never come up with something that requires a far bigger leap than what you allow the LLM to perturb on a single step. Also the genetic algorithm will probably result in a vibecoded mess of short-sighted decisions just like evolution creates a spaghetti genome in real life.)
I'll definitely need to look into how people have improved the idea and whether it is practical now.
- beepdyboop 6mo agoThis is not a new idea at all, many many have had it, no one really can claim it
- naveen99 6mo agoYou know this doesn’t work most of the time…
- stingraycharles 6mo agoGenetic algorithms have existed since the 60s / 70s, e.g. computers learning to play a game. LLMs aren’t particularly guide at it. I think hyperparameter tuning may actually be a kind of genetic algorithm.
- janalsncm 6mo agoHyperparameter tuning could be done by genetic algorithm. I think it’s a bit of a category error to say that it is a genetic algorithm though. Hyperparam tuning is usually done by Bayesian Optimization though.
- stingraycharles 6mo agoYeah that’s correct, it could use it, but there are better alternatives for this particular problem.