4 ms·

My ongoing hypothesis is that LLMs are leveraging inherit structure in natural language to cheat. Apparently if you embed a large enough corpus you can do alge

by Verdex 3y ago

My ongoing hypothesis is that LLMs are leveraging inherit structure in natural language to cheat. Apparently if you embed a large enough corpus you can do algebra with the semantic value of words. King - Queen gives you a vector that when applied to Man gives you Woman. Great our language has structure. This makes sense because people are going to want to communicate in a way that mirrors the structure of the problems that we often encounter. Kind of an information theory frequent messages should be short kind of thing. Then we see all these people trying to crack prompt engineering. And we see some people raving about how they're using LLMs to do so much amazing work and others keep getting back nonsense garbage. My conclusion is that some people are just more synced with the social ziegiest, which is also embedded into LLMs. By talking to the LLM the right way, they're dumping in additional structure about the problem they want to solve. Which means I expect non-experts (or experts who don't sufficiently internalize the jargon) to get noticably worse results just because they don't know how to talk "right". Not only will non-experts be unable to interpret results. But they also can't formulate questions or requests that the LLM will be able to give good results to in the first place. [I would love to see some one run a study on this.] Also, not every problem we face has a structure amendable to natural language. Or they're too niche to have a sufficiently large corpus of natural language and jargon for LLMs to consume. It would be really interesting to see if we can sort problems in this way. But I suspect disappointment from people trying to use LLMs on certain problems even if they are experts who can talk the talk.

deleted 3y ago
[deleted]
logicprog 3y ago
As a counterexample of one, I would consider myself pretty much an "expert" in some sense at programming, and I've internalized the jargon of software engineering and computer science to the degree that my conversation is sometimes nigh incomprehensible to even other people conversational with programming, even when I'm intentionally trying not to use jargon. I'm also typically someone who writes in an extremely careful and information-dense wa designed to specify what I mean as precisely as possible and hedge out any possible misinterpretations, even to the point of writing entirely too much. And yet, even when I combine all those factors with having a fairly simple task using common technologies in mind, I typically can't get large language models to produce anything I would consider good that I couldn't have written myself just as quickly as it took me to input the prompt and get the result. I also find translating programming ideas into words instead of code an impedance barrier when code is what I have in my head. Moreover, obviously it never does things how I want them done, which is very frustrating because every choice I make about how I write my code, down to the smallest detail, is 100% intentional and conscious, and so it just feels frustrating and disappointing to use code that a large language model has generated for me, because it's not what I wanted written in the first place. And if I had given enough specification about how to go about things to the model for it to produce exactly what I wanted, I might as well have written it myself, because the translation of my thoughts and intentions into code is largely effortless as it is. The only thing that typically slows me down noticeably is referencing APIs and extremely obscure bugs typically caused by a gotcha in whatever system I'm working on — but it would take just as much time to prompt, wait for, and then read the output of a model as it would to just tab over to the docs I already have open, click on a link for a function that seems likely, and read the description of the function or method, and as for obscure bugs, I doubt a large language model could help me debug them in the first place when I can't find anything on stack overflow or anywhere else on the internet about them. I very rarely have the sort of problems or produce the sort of bugs that would be easily helped by a large language model.

by Verdex 3y ago

deleted 3y ago

[deleted]

logicprog 3y ago

As a counterexample of one, I would consider myself pretty much an "expert" in some sense at programming, and I've internalized the jargon of software engineering and computer science to the degree that my conversation is sometimes nigh incomprehensible to even other people conversational with programming, even when I'm intentionally trying not to use jargon. I'm also typically someone who writes in an extremely careful and information-dense wa designed to specify what I mean as precisely as possible and hedge out any possible misinterpretations, even to the point of writing entirely too much. And yet, even when I combine all those factors with having a fairly simple task using common technologies in mind, I typically can't get large language models to produce anything I would consider good that I couldn't have written myself just as quickly as it took me to input the prompt and get the result. I also find translating programming ideas into words instead of code an impedance barrier when code is what I have in my head. Moreover, obviously it never does things how I want them done, which is very frustrating because every choice I make about how I write my code, down to the smallest detail, is 100% intentional and conscious, and so it just feels frustrating and disappointing to use code that a large language model has generated for me, because it's not what I wanted written in the first place. And if I had given enough specification about how to go about things to the model for it to produce exactly what I wanted, I might as well have written it myself, because the translation of my thoughts and intentions into code is largely effortless as it is. The only thing that typically slows me down noticeably is referencing APIs and extremely obscure bugs typically caused by a gotcha in whatever system I'm working on — but it would take just as much time to prompt, wait for, and then read the output of a model as it would to just tab over to the docs I already have open, click on a link for a function that seems likely, and read the description of the function or method, and as for obscure bugs, I doubt a large language model could help me debug them in the first place when I can't find anything on stack overflow or anywhere else on the internet about them. I very rarely have the sort of problems or produce the sort of bugs that would be easily helped by a large language model.