This brings up an interesting point, which is that it's likely that software developers overestimate the capabilities of LLMs in other domains because our own d

This brings up an interesting point, which is that it's likely that software developers overestimate the capabilities of LLMs in other domains because our own domain is so thoroughly documented on the internet. From what I've been able to gather interacting with other sectors, it seems like software is pretty unique in having a culture of sharing everything—tools, documentation, best practices, tutorials, blogs—online for free. Most professions can't be picked up by someone learning on their own from nothing but the internet in the way that software can. I strongly suspect that the result will be that LLMs (being trained on the internet) do substantially better on software related tasks than they do in other domains, but software developers may be largely blind to that difference since they're not experts in the other domain.