super cool. Past ML progress told us that for most problems for which we manages to get sort-of okay performance, we saw a dramatic subsequent increase in accur

super cool. Past ML progress told us that for most problems for which we manages to get sort-of okay performance, we saw a dramatic subsequent increase in accuracy. The moment we have some idea how to solve it, we usually have quite a few paths forward to increase accuracy. I bet that understanding code will be similar as soon as we focus on it. It's an interesting problem. So far our models are not really coding, which would involve actually running stuff. EDIT: but GPT4, GPT5 and GPT6 will still get stuff wrong. But so do humans. I don't think the existence of errors is a problem, we just have to get accurate enough to get useful.