4 ms·
My crude reaction to this piece is that it feels like they are describing a technique for finding a local maximum for any coding problem. It is an unsurprising
by norir 3y ago
My crude reaction to this piece is that it feels like they are describing a technique for finding a local maximum for any coding problem. It is an unsurprising result but it seems like it would be hard to avoid getting stuck in a low valley without human intervention. I also believe that for any deterministic program, the best human written program will converge on the optimal solution given the real world matetial and temporal constraints with or without LLMs. I am not sure if LLMs will be in wide use in 10 years, but they will certainly help us humans write better code if only to avoid getting stuck in the traps they introduce.
- j_maffe 3y agoFor the local maximum issue, there's plenty of optimization algorithms that deal with this issue which I think are perfectly viable for this case scenario.
- seeknotfind 3y agoHmmm, I mean there is a more general question: in the space of algorithms, how continuous are the paths between different solutions? For instance, if you start with bubble sort, I don't think it's at all clear this algorithm can take small finite steps to turn into improved to merge sort. This is the low valley you are talking about. This also aligns nicely with the results in the paper. GPT-4 improves, so it's perhaps good enough to make larger steps or transformations of an algorithm, whereas weaker models don't. The real interesting result would be if they improved an algorithm beyond what people have been able to do. Otherwise it's just interesting and a peek into the future..