4 ms·

The comments about debugging make me sad. It shows that people fundamentally do not understand the tools they are using. Given a book of numbers, here are two

The comments about debugging make me sad. It shows that people fundamentally do not understand the tools they are using. Given a book of numbers, here are two tasks: 1) copy out the entire book, but replace every prime number with 7. 2) write down the list of prime numbers in the book. Which one is easier? LLMs have to generate tokens one at a time, and it’s very very difficult to perfectly generate a set of input tokens except for some tokens. Since you are almost certainly randomising the probabilities to some degree (that’s what temperature does), you’re also asking for both deterministic and random outputs. TLDR: ask LLMs what is wrong with the code. Ask for a diff. Don’t ask for an LLM to refactor, bug fix or annotate code… That’s extremely naive usage. Back to my stupid analogy: “please copy out this book, but fix the numbers which are ‘wrong’” I can hardly complain when I get terrible results can I?

jprete 3y ago
No-one fundamentally understands LLMs. It's inherent to the technology.
- wokwokwok 3y ago
  Don't be daft. If you don't understand that an LLM generates output token-by-token, and that as a result of randomizing the token output probabilities that you cannot generate an error free copy of the input you've invested so little time in understanding as to be farcical. There's 'wow, these are complicated and I don't fully understand them' ...and there's, 'What this. It shiny. Not worky. Make some random change to prompt and pray to LLM gods' Come on, make an effort.
  - jiggawatts 3y ago
    I think you're being unfairly downvoted. People generally are terrible at using tools, especially tools that they perceive as "black boxes". I've seen an IT professional type this into Google: "Why did my PC crash?" I couldn't believe that after two decades of using web search technology, he still hadn't figured out how to extract value from a text index using specific and relevant key words. Similarly, very few developers know what a database index does or whether they need one or not. My pet theory is that NoSQL databases become popular because many of them automatically index every column, making it feel like a magic black box instead of an evil black box. LLMs are not only black boxes, but they're soooo fundamentally different and new that a lot of people are really struggling to wrap their brains around it. Just in this thread, today, there are people that are complaining about the direct equivalent of "my poorly thought out Google search didn't work, so Google is bad."
    - jprete 3y ago
      The GP directly calling me "daft" was unwarranted.
      - wokwokwok 3y ago
        I apologise for calling you daft. Here's a re-wording of our conversation without that; you decide how you want to take it. me: I am sad because people are clearly using these tools without understanding them; here is a specific example and reason of why what they're trying to do doesn't work. you: these tools cannot be understood. me: not only is that is literally false, it's obviously and self evidently false, and I just gave you a specific example of how; I can't take anything you say as being in good faith when you believe that anything to do with AI is literally unfathomable, and you can't even be bothered responding to what I actually wrote. Maybe, looking into it, you would find that it's not nearly as complicated or difficult as you imagine. If not, I guess we have nothing to talk about. This kind of attitude is why I'm sad. /shrug

jprete 3y ago

No-one fundamentally understands LLMs. It's inherent to the technology.

wokwokwok 3y ago
Don't be daft. If you don't understand that an LLM generates output token-by-token, and that as a result of randomizing the token output probabilities that you cannot generate an error free copy of the input you've invested so little time in understanding as to be farcical. There's 'wow, these are complicated and I don't fully understand them' ...and there's, 'What this. It shiny. Not worky. Make some random change to prompt and pray to LLM gods' Come on, make an effort.
- jiggawatts 3y ago
  I think you're being unfairly downvoted. People generally are terrible at using tools, especially tools that they perceive as "black boxes". I've seen an IT professional type this into Google: "Why did my PC crash?" I couldn't believe that after two decades of using web search technology, he still hadn't figured out how to extract value from a text index using specific and relevant key words. Similarly, very few developers know what a database index does or whether they need one or not. My pet theory is that NoSQL databases become popular because many of them automatically index every column, making it feel like a magic black box instead of an evil black box. LLMs are not only black boxes, but they're soooo fundamentally different and new that a lot of people are really struggling to wrap their brains around it. Just in this thread, today, there are people that are complaining about the direct equivalent of "my poorly thought out Google search didn't work, so Google is bad."
  - jprete 3y ago
    The GP directly calling me "daft" was unwarranted.
    - wokwokwok 3y ago
      I apologise for calling you daft. Here's a re-wording of our conversation without that; you decide how you want to take it. me: I am sad because people are clearly using these tools without understanding them; here is a specific example and reason of why what they're trying to do doesn't work. you: these tools cannot be understood. me: not only is that is literally false, it's obviously and self evidently false, and I just gave you a specific example of how; I can't take anything you say as being in good faith when you believe that anything to do with AI is literally unfathomable, and you can't even be bothered responding to what I actually wrote. Maybe, looking into it, you would find that it's not nearly as complicated or difficult as you imagine. If not, I guess we have nothing to talk about. This kind of attitude is why I'm sad. /shrug