5 ms·

The problem is that people like this author are trying to literally treat it like a person instead of an LLM. Like honestly if you look at the linked chat convo

The problem is that people like this author are trying to literally treat it like a person instead of an LLM. Like honestly if you look at the linked chat convo early in the article, this person kind of just sucks at prompt engineering, imo. "At times I was able to get the chat agent to give me what I wanted, but I had to be very specific and I often had to scold it." You can't just half-ass a paragraph of disjointed system instructions into the user input and expect clean results, in my experience. You need to leverage the custom system instructions, give example responses if possible, and be very, very specific and direct with instructions. You need to explain the type of response you want, and you also need to describe any applicable constraints (or lack thereof) on the response content. "When you are asked something, it is crucial that you cite your sources, and always use the most authoritative sources (government agencies for example) rather that sites like Wikipedia" This is not sufficient to achieve what the author intends. It's written in a speech-like roundabout style (e.g. "it is crucial that"), and there's a typo right in the middle on an important word (than --> that). LLM can work around typos in most cases, but here it is vaguely possibly imo that this is what is causing it to continue citing wikipedia in responses. "At times, the tool was too eager to please, so I asked it to tone it down a little: “You can skip the chit chat and pleasantries.”" I have found in my experience playing with ChatGPT that this is just the completely wrong mental model to have of the tool in order to get what you want out of it. You have to treat it more like a prose-language programming tool, not like a person with emotions that you are conversing with...

JohnFen 3y ago
> The problem is that people like this author are trying to literally treat it like a person instead of an LLM. It's hard to blame them, though. The author and many others are using the LLM in exactly the way that they're told it should be used. You don't learn that this is wrong unless you're an LLM nerd.
- happytiger 3y ago
  I don’t think that’s true. I think a lot of these articles actually create these results intentionally so that they have the conclusion they want to write about: that AI isn’t ready. I have been watching Devin and really see the model they have implemented working well for these kinds of executive function step-by-step LLM tasks. It’s remarkably smart how they implemented it. https://the-decoder.com/cognition-unveils-ai-powered-software-developer-devin-for-better-programming/ https://the-decoder.com/cognition-unveils-ai-powered-softwar... If you give vague instructions to a junior level human you get poor work product just like AI. But every journalist wants to write that ringer-dinger traffic bringer about ai not being ready for prime time or whatever bad headline works…
  - JohnFen 3y ago
    You seem to be assuming malice or bad faith here, but I haven't seen any reason to suspect that. Not to say that you're wrong, of course. You might be right. I just don't see why I should suspect it. Particularly considering that I know multiple people who have made a similar error. The distortion that the extreme hype is generating is pretty widespread.
    - happytiger 3y ago
      Perhaps. But I generally would expect some level of research and professionalism beyond such simplistic concepts from someone who has a degree in journalism and works in the field. Hype is precisely what journalists are supposed to cut through. That’s why it’s called ‘reporting.’ So the idea that because many people with no background in AI are confused conflating to reasons why that’s acceptable for journalists writing about it seems rather weak to me. I do see a broad series of articles that follow the pattern, and do believe it to be clickbait ‘journalism’ of the lowest standard. Incompetence could be an explanation, but it’s article after article from professional mainstream sources: sources that should know better or at least have a consult with someone who does as part of their due diligence. That implies it’s being done because it gets views, not because it’s good journalism and is therefore intentional. But that is my conjecture you are right.
      - JohnFen 3y ago
        Well, the press has always been truly awful at reporting on technical topics. It's why you can't, and never could, take any reporting on such things at face value at all, is why all of the scientists and researchers I've worked with have disliked it when their work gets reported on, and is why the so much of what regular people think "scientists say" is incorrect. Nuance and therefore accuracy gets dropped in favor of having a clear, simple story. I don't see why LLM-related topics would be any different. But the reasons why this happens are pretty clear, and poor intentions or even laziness on the part of the reporters are rarely factors.
dragonwriter 3y ago
> The problem is that people like this author are trying to literally treat it like a person instead of an LLM. This is exactly how consumer-facing LLM-powered interfaces are marketed and promoted, though.
Tainnor 3y ago
In other words, in order to get the "massive productivity boost" often cited as the main selling point of LLMs, you need to have completed a thorough training on prompt engineering, and if you get something slightly wrong (e.g. a simple typo), you get absolutely zero feedback that your intent was misunderstood?
- wenebego 3y ago
  Yeah these people keep telling me about how cars are a "massive productivity boost" and i get in and nothing happens.
  - Grimblewald 3y ago
    Also I refused to learn basic traffic rules and now I keep ending up in accidents. I didn't need to know traffic rules when walking! I was told walking but faster with more capacity to transport cargo >:(
    - 6510 3y ago
      move fast and break things

JohnFen 3y ago

> The problem is that people like this author are trying to literally treat it like a person instead of an LLM. It's hard to blame them, though. The author and many others are using the LLM in exactly the way that they're told it should be used. You don't learn that this is wrong unless you're an LLM nerd.

happytiger 3y ago
I don’t think that’s true. I think a lot of these articles actually create these results intentionally so that they have the conclusion they want to write about: that AI isn’t ready. I have been watching Devin and really see the model they have implemented working well for these kinds of executive function step-by-step LLM tasks. It’s remarkably smart how they implemented it. https://the-decoder.com/cognition-unveils-ai-powered-software-developer-devin-for-better-programming/ https://the-decoder.com/cognition-unveils-ai-powered-softwar... If you give vague instructions to a junior level human you get poor work product just like AI. But every journalist wants to write that ringer-dinger traffic bringer about ai not being ready for prime time or whatever bad headline works…
- JohnFen 3y ago
  You seem to be assuming malice or bad faith here, but I haven't seen any reason to suspect that. Not to say that you're wrong, of course. You might be right. I just don't see why I should suspect it. Particularly considering that I know multiple people who have made a similar error. The distortion that the extreme hype is generating is pretty widespread.
  - happytiger 3y ago
    Perhaps. But I generally would expect some level of research and professionalism beyond such simplistic concepts from someone who has a degree in journalism and works in the field. Hype is precisely what journalists are supposed to cut through. That’s why it’s called ‘reporting.’ So the idea that because many people with no background in AI are confused conflating to reasons why that’s acceptable for journalists writing about it seems rather weak to me. I do see a broad series of articles that follow the pattern, and do believe it to be clickbait ‘journalism’ of the lowest standard. Incompetence could be an explanation, but it’s article after article from professional mainstream sources: sources that should know better or at least have a consult with someone who does as part of their due diligence. That implies it’s being done because it gets views, not because it’s good journalism and is therefore intentional. But that is my conjecture you are right.
    - JohnFen 3y ago
      Well, the press has always been truly awful at reporting on technical topics. It's why you can't, and never could, take any reporting on such things at face value at all, is why all of the scientists and researchers I've worked with have disliked it when their work gets reported on, and is why the so much of what regular people think "scientists say" is incorrect. Nuance and therefore accuracy gets dropped in favor of having a clear, simple story. I don't see why LLM-related topics would be any different. But the reasons why this happens are pretty clear, and poor intentions or even laziness on the part of the reporters are rarely factors.

dragonwriter 3y ago

> The problem is that people like this author are trying to literally treat it like a person instead of an LLM. This is exactly how consumer-facing LLM-powered interfaces are marketed and promoted, though.

Tainnor 3y ago

In other words, in order to get the "massive productivity boost" often cited as the main selling point of LLMs, you need to have completed a thorough training on prompt engineering, and if you get something slightly wrong (e.g. a simple typo), you get absolutely zero feedback that your intent was misunderstood?

wenebego 3y ago
Yeah these people keep telling me about how cars are a "massive productivity boost" and i get in and nothing happens.
- Grimblewald 3y ago
  Also I refused to learn basic traffic rules and now I keep ending up in accidents. I didn't need to know traffic rules when walking! I was told walking but faster with more capacity to transport cargo >:(
  - 6510 3y ago
    move fast and break things