6 ms·

It's also worth pointing out that if an LLM gives you code that works, but you can't understand why it works, asking it to explain what each line is doing is re

It's also worth pointing out that if an LLM gives you code that works, but you can't understand why it works, asking it to explain what each line is doing is remarkably effective.

logicprog 3y ago
But if you don't already understand the code and the concepts, you have no way of knowing if it's actually giving you accurate information or just hallucinating or confabulating. That's my fundamental problem with the entire idea of using llms to Aid with this kind of stuff, it's not easy to check it carefully enough unless you already know enough about the topic that you could have just done what you asked it to do faster if you type fast enough, especially considering the amount of time you'll probably have to spend arguing with it and refining the prompt and clarifying and thinking through the logic in detail anyway.
- Verdex 3y ago
  My ongoing hypothesis is that LLMs are leveraging inherit structure in natural language to cheat. Apparently if you embed a large enough corpus you can do algebra with the semantic value of words. King - Queen gives you a vector that when applied to Man gives you Woman. Great our language has structure. This makes sense because people are going to want to communicate in a way that mirrors the structure of the problems that we often encounter. Kind of an information theory frequent messages should be short kind of thing. Then we see all these people trying to crack prompt engineering. And we see some people raving about how they're using LLMs to do so much amazing work and others keep getting back nonsense garbage. My conclusion is that some people are just more synced with the social ziegiest, which is also embedded into LLMs. By talking to the LLM the right way, they're dumping in additional structure about the problem they want to solve. Which means I expect non-experts (or experts who don't sufficiently internalize the jargon) to get noticably worse results just because they don't know how to talk "right". Not only will non-experts be unable to interpret results. But they also can't formulate questions or requests that the LLM will be able to give good results to in the first place. [I would love to see some one run a study on this.] Also, not every problem we face has a structure amendable to natural language. Or they're too niche to have a sufficiently large corpus of natural language and jargon for LLMs to consume. It would be really interesting to see if we can sort problems in this way. But I suspect disappointment from people trying to use LLMs on certain problems even if they are experts who can talk the talk.
  - deleted 3y ago
    [deleted]
  - logicprog 3y ago
    As a counterexample of one, I would consider myself pretty much an "expert" in some sense at programming, and I've internalized the jargon of software engineering and computer science to the degree that my conversation is sometimes nigh incomprehensible to even other people conversational with programming, even when I'm intentionally trying not to use jargon. I'm also typically someone who writes in an extremely careful and information-dense wa designed to specify what I mean as precisely as possible and hedge out any possible misinterpretations, even to the point of writing entirely too much. And yet, even when I combine all those factors with having a fairly simple task using common technologies in mind, I typically can't get large language models to produce anything I would consider good that I couldn't have written myself just as quickly as it took me to input the prompt and get the result. I also find translating programming ideas into words instead of code an impedance barrier when code is what I have in my head. Moreover, obviously it never does things how I want them done, which is very frustrating because every choice I make about how I write my code, down to the smallest detail, is 100% intentional and conscious, and so it just feels frustrating and disappointing to use code that a large language model has generated for me, because it's not what I wanted written in the first place. And if I had given enough specification about how to go about things to the model for it to produce exactly what I wanted, I might as well have written it myself, because the translation of my thoughts and intentions into code is largely effortless as it is. The only thing that typically slows me down noticeably is referencing APIs and extremely obscure bugs typically caused by a gotcha in whatever system I'm working on — but it would take just as much time to prompt, wait for, and then read the output of a model as it would to just tab over to the docs I already have open, click on a link for a function that seems likely, and read the description of the function or method, and as for obscure bugs, I doubt a large language model could help me debug them in the first place when I can't find anything on stack overflow or anywhere else on the internet about them. I very rarely have the sort of problems or produce the sort of bugs that would be easily helped by a large language model.
- jruohonen 3y ago
  > But if you don't already understand the code and the concepts, you have no way of knowing if it's actually giving you accurate information or just hallucinating or confabulating. That's the issue right there, I would also argue. And this epistemology thing is at the heart of the whole (AI) post-truth phenomenon.
  - logicprog 3y ago
    Yeah. It's similar to the Gettier problem. You know, people often belittle philosophy as a useless field, and for some fields and subjects they are very right in that (witness all the arguments over finitism or existential inertia), but I think there are a few fields of philosophy that are actually really important. Not because they give answers, but because they give you a larger, richer, and consequently more nuanced tool set with which to approach certain categories of problems that are otherwise difficult to think about, or even notice as problems in the first place, by giving you access to a whole history of thought. In this respect I think philosophy of mind, epistemology, language, semiotics, and to a lesser extent ethics have crucial bearing on the things we see happening nowadays, including the advent of large language models. It's frustrating as someone who's really interested in philosophy to see people discuss these things where long history of philosophical thought is relevant without any real respect for or experience in those fields. It leaves their arguments often reductionist and/or bereft of strong argument and clear reasoning, full of implicit assumptions and ignored details, even when they're right.
    - JHonaker 3y ago
      > In this respect I think philosophy of mind, epistemology, language, semiotics, and to a lesser extent ethics have crucial bearing on the things we see happening nowadays Whenever I hit that brushes up against these, I always find myself wishing that introductions to these topics were more easily found. I’m not saying they don’t exist, just that there’s no clear entry point to start learning about the philosophy of mind (or others). It’s like in Math, where you have books like Lang’s “Algebra” which are titled in the most literal way possible. It is simultaneously exactly what’s in the book, but woefully inappropriate for people who don’t already know what it is. Do you have any recommendations?
- simonw 3y ago
  I see this argument pretty often - the idea that if it hallucinates and you have to check everything it does for you it would be faster to just do it yourself. It's still faster to use an LLM. I don't know how to make a convincing argument about this - I just know that I've been using these tools on an almost daily basis for over a year and they absolutely help me work faster even though I'm very aware of hallucinations and mistakes and constantly look out for them.
  - hackerlight 3y ago
    I also hallucinate. Turns out aggregate hallucations go down when it's the union of me + AI instead of me by myself.
- xboxnolifes 3y ago
  The same is true about all learning material.
deleted 3y ago
[deleted]
65 3y ago
At that point it seems faster to just learn how to write the logic yourself. Or get your answer from StackOverflow. To me this seems like we're trying to re-invent SQL in a weird way, where you can "write code like how you write naturally!" - except you still need to logically piece your code together. Programming is more like plumbing than it is like writing context independent functions.

It's also worth pointing out that if an LLM gives you code that works, but you can't understand why it works, asking it to explain what each line is doing is remarkably effective.

logicprog 3y ago

But if you don't already understand the code and the concepts, you have no way of knowing if it's actually giving you accurate information or just hallucinating or confabulating. That's my fundamental problem with the entire idea of using llms to Aid with this kind of stuff, it's not easy to check it carefully enough unless you already know enough about the topic that you could have just done what you asked it to do faster if you type fast enough, especially considering the amount of time you'll probably have to spend arguing with it and refining the prompt and clarifying and thinking through the logic in detail anyway.

Verdex 3y ago
My ongoing hypothesis is that LLMs are leveraging inherit structure in natural language to cheat. Apparently if you embed a large enough corpus you can do algebra with the semantic value of words. King - Queen gives you a vector that when applied to Man gives you Woman. Great our language has structure. This makes sense because people are going to want to communicate in a way that mirrors the structure of the problems that we often encounter. Kind of an information theory frequent messages should be short kind of thing. Then we see all these people trying to crack prompt engineering. And we see some people raving about how they're using LLMs to do so much amazing work and others keep getting back nonsense garbage. My conclusion is that some people are just more synced with the social ziegiest, which is also embedded into LLMs. By talking to the LLM the right way, they're dumping in additional structure about the problem they want to solve. Which means I expect non-experts (or experts who don't sufficiently internalize the jargon) to get noticably worse results just because they don't know how to talk "right". Not only will non-experts be unable to interpret results. But they also can't formulate questions or requests that the LLM will be able to give good results to in the first place. [I would love to see some one run a study on this.] Also, not every problem we face has a structure amendable to natural language. Or they're too niche to have a sufficiently large corpus of natural language and jargon for LLMs to consume. It would be really interesting to see if we can sort problems in this way. But I suspect disappointment from people trying to use LLMs on certain problems even if they are experts who can talk the talk.
- deleted 3y ago
  [deleted]
- logicprog 3y ago
  As a counterexample of one, I would consider myself pretty much an "expert" in some sense at programming, and I've internalized the jargon of software engineering and computer science to the degree that my conversation is sometimes nigh incomprehensible to even other people conversational with programming, even when I'm intentionally trying not to use jargon. I'm also typically someone who writes in an extremely careful and information-dense wa designed to specify what I mean as precisely as possible and hedge out any possible misinterpretations, even to the point of writing entirely too much. And yet, even when I combine all those factors with having a fairly simple task using common technologies in mind, I typically can't get large language models to produce anything I would consider good that I couldn't have written myself just as quickly as it took me to input the prompt and get the result. I also find translating programming ideas into words instead of code an impedance barrier when code is what I have in my head. Moreover, obviously it never does things how I want them done, which is very frustrating because every choice I make about how I write my code, down to the smallest detail, is 100% intentional and conscious, and so it just feels frustrating and disappointing to use code that a large language model has generated for me, because it's not what I wanted written in the first place. And if I had given enough specification about how to go about things to the model for it to produce exactly what I wanted, I might as well have written it myself, because the translation of my thoughts and intentions into code is largely effortless as it is. The only thing that typically slows me down noticeably is referencing APIs and extremely obscure bugs typically caused by a gotcha in whatever system I'm working on — but it would take just as much time to prompt, wait for, and then read the output of a model as it would to just tab over to the docs I already have open, click on a link for a function that seems likely, and read the description of the function or method, and as for obscure bugs, I doubt a large language model could help me debug them in the first place when I can't find anything on stack overflow or anywhere else on the internet about them. I very rarely have the sort of problems or produce the sort of bugs that would be easily helped by a large language model.
jruohonen 3y ago
> But if you don't already understand the code and the concepts, you have no way of knowing if it's actually giving you accurate information or just hallucinating or confabulating. That's the issue right there, I would also argue. And this epistemology thing is at the heart of the whole (AI) post-truth phenomenon.
- logicprog 3y ago
  Yeah. It's similar to the Gettier problem. You know, people often belittle philosophy as a useless field, and for some fields and subjects they are very right in that (witness all the arguments over finitism or existential inertia), but I think there are a few fields of philosophy that are actually really important. Not because they give answers, but because they give you a larger, richer, and consequently more nuanced tool set with which to approach certain categories of problems that are otherwise difficult to think about, or even notice as problems in the first place, by giving you access to a whole history of thought. In this respect I think philosophy of mind, epistemology, language, semiotics, and to a lesser extent ethics have crucial bearing on the things we see happening nowadays, including the advent of large language models. It's frustrating as someone who's really interested in philosophy to see people discuss these things where long history of philosophical thought is relevant without any real respect for or experience in those fields. It leaves their arguments often reductionist and/or bereft of strong argument and clear reasoning, full of implicit assumptions and ignored details, even when they're right.
  - JHonaker 3y ago
    > In this respect I think philosophy of mind, epistemology, language, semiotics, and to a lesser extent ethics have crucial bearing on the things we see happening nowadays Whenever I hit that brushes up against these, I always find myself wishing that introductions to these topics were more easily found. I’m not saying they don’t exist, just that there’s no clear entry point to start learning about the philosophy of mind (or others). It’s like in Math, where you have books like Lang’s “Algebra” which are titled in the most literal way possible. It is simultaneously exactly what’s in the book, but woefully inappropriate for people who don’t already know what it is. Do you have any recommendations?
simonw 3y ago
I see this argument pretty often - the idea that if it hallucinates and you have to check everything it does for you it would be faster to just do it yourself. It's still faster to use an LLM. I don't know how to make a convincing argument about this - I just know that I've been using these tools on an almost daily basis for over a year and they absolutely help me work faster even though I'm very aware of hallucinations and mistakes and constantly look out for them.
- hackerlight 3y ago
  I also hallucinate. Turns out aggregate hallucations go down when it's the union of me + AI instead of me by myself.
xboxnolifes 3y ago
The same is true about all learning material.

deleted 3y ago

[deleted]

65 3y ago

At that point it seems faster to just learn how to write the logic yourself. Or get your answer from StackOverflow. To me this seems like we're trying to re-invent SQL in a weird way, where you can "write code like how you write naturally!" - except you still need to logically piece your code together. Programming is more like plumbing than it is like writing context independent functions.