My ongoing hypothesis [1] is that natural language has intelligence (or to say it another way, natural language has embedded structure that harmonizes with the

My ongoing hypothesis [1] is that natural language has intelligence (or to say it another way, natural language has embedded structure that harmonizes with the structure of problems that we face[2]). If correct, there are a few relevant conclusions that you can draw. The first is that prompt engineering is a real thing. By talking "correctly" to the LLM you're providing it with additional information that will allow it to give you more legitimate answers. This might help explain these threads where I'm seeing a lot of comments from people who rave non-stop about all of the work they're able to get LLMs do for them AND also people who cannot get LLMs to do anything worth doing (I'm included in this latter group). The former know how to "talk right" to the LLM. While I think prompt engineering is real, I don't think it's necessarily important. "Talking correctly" is nebulous and resists our ability to categorize. The metric is "gets good answers from LLM", but in order to know if the answers are good you have to also know the reality. It's pretty obvious (at least to me) that if you have to choose between speaking in a useful way and knowing the reality that you would just choose knowing reality[3]. The other thing is that this means that LLMs are a dead-ish end. Because they're approximating the intelligence in natural language, they'll never be smarter than our collective words. Sure, we'll be able to make cheaper, smaller, and more specialized LLMs, but they'll need another non-LLM component to reach the next level of usefulness. [1] - And I mostly keep mentioning it to see how people react. I've yet to see a "this is definitely wrong because ..." type of response. I'm hoping to see one that actually has some meat to it. [2] - So, the idea is that word2vec allows you to do vector math of the pattern: KING - QUEEN = v; Man - V = Woman. Great, so our grammar has some sort of algebraic structures inside of it. And this sort of makes sense that this would happen. Kind of an information theory version of frequent messages should be small. Only here, it's natural language should have a structure that mirrors the structure of problems we care about. People who talk the right way have a tendency to solve problems that matter and natural language evolves in the direction that enables people to be successful. The problem then is that some problems are simply too complex to embed their structure into natural language grammar. Like rocket science, brain surgery, or OS construction. The other problem is that some problems are simply too niche to develop a jargon. And finally, some problems have a structure that we haven't seen before. In any of these cases, I expect LLMs to fail to be useful. [3] - I suppose there could exist some world in which the knowers are able to certify the talkers and then maybe that's easier than teaching unknowers to be knowers. But it feels like it's a bit risky. If the talkers ever get off base, then it's not like anyone is going to notice until it's too late.