4 ms·

Sam Altman states that "prompt engineering", in his mind, is a result of a temporary limitations of the LLMs. That he doesn't expect people to be doing what we

Sam Altman states that "prompt engineering", in his mind, is a result of a temporary limitations of the LLMs. That he doesn't expect people to be doing what we call "prompt engineering" in 5 years. You'll simply ask for what you want and if there is ambiguity it can be sorted out interactively or it will just do the 'right' or obvious thing that we would expect.

ElFitz 4y ago
So a technical solution to a human problem?
Mezzie 4y ago
I disagree with him. I've spent several years answering people's random questions (librarian) in multiple settings and my first W2 job back in 2004 was technological education. I also have a Linguistics degree and can program so I understand what LLMs can and can't do fairly well even if I'm nowhere near an expert. He's vastly underestimating the average person's awareness of their own thought process or even ability to accurately answer iterative questions in order to sort things out interactively. The weakness with such a claim isn't the AI, it's the humans' ability to provide the AI with accurate and useful information. Not to mention that there's still a large portion of the general public that just freezes up when it comes to anything technological. I've provided support to a number of different communities and even educated people can't do this type of thinking easily. It's possible that we could eventually get to that point, but it would require some massive educational efforts and a culture shift, which would also require substantial investment without a clear road to profit, and I believe America lacks the political will or even ability to implement anything like this. Since we can't act in unison, it will be down to companies, and for them it makes more sense to solve the problem by hiring prompt engineers with the savings they get from cutting elsewhere instead of training all their existing teams on how to learn new ways of thinking. tl;dr: Technologically he's probably right, but he's massively overestimating the public's ability to use interactive tools to get information. > obvious thing that we would expect Any time you rely on something being 'obvious', the public is going to somehow fuck it up.
- cloverich 4y ago
  This is similar to the situation with software engineers interacting with other parts of the business or customers. It's IME the biggest skill of software engineers, not to write code, but to understand and translate business requirement in to code, while accounting for _existing_ (prior) requirements that were often forgotten. The code as such is usually straight forward, but the requirements on either end is where the work is. This is also what programmers typically complain about when they say their PM isn't "technical" enough. What they usually mean is they don't understand the details of the business / domain well enough to meaningfully contribute to the organization / break up of the work. To that end, I'd expect these language models to eventually improve at that task too, perhaps responding with choices or diagrams to help users learn how to ask what they actually want. That's in a sense what a prompt engineer is doing in this context. In that sense I think I agree w/ your assessment although I also think there's a lot of room for these AI's to handle that task, at least partially, as they improve.
  - Mezzie 4y ago
    I agree. I'm not a SWE because I don't want to code for 40+ hours a week, but I've been picking up projects here and there for small businesses for a long time (I started helping my dad do the same when I was 5-6) and so I manage projects end to end and so much of it is the back and forth with the clients. Knowing the right questions to ask and how people think about things are some of my most important skills, and in that respect my education and library experience has been very helpful. I'm far from the best programmer - hell, I'd consider myself below average but have enough experience to still be passable, but I'm easy to work with and that counts for a lot. The issue with language models in particular is that to use them effectively you have to fundamentally understand that you're translating human-brain-language into computer-language and understand that those are in fact two different things even if their delivery happens identically. For most coders, this jump happens semi-instinctively because they know how to test something to verify 'ah yes, this follows computer logic'. The general public thinks computers are wizardry. The other group of people who will be able to understand this are those who have thought deeply about language for whatever reason and therefore know that human-brain-language has qualities that extend beyond 'We use words'. This is also very hard to impart to novices; I have a Linguistics degree and studied a cumulative 10 years of 6 languages so I went through that knowledge transfer as well. They're both hard and one or the other is necessary to interface with language models effectively and will be for likely the next 10ish years just because providing a good technical solution will require the technical people to understand much more about the social side of communication and that will also take time.
kzrdude 4y ago
The "sorted out interactively" that he mentions is exactly prompt engineering, isn't it?
- rcme 4y ago
  If you ask someone to do something for you and they ask a clarifying question, do you consider that prompt engineering?
  - kzrdude 4y ago
    If it's a language model, yes. I think if it's in the initial prompt (first message) or second message to the AI doesn't make much difference for its role in the process.

ElFitz 4y ago

So a technical solution to a human problem?

Mezzie 4y ago

I disagree with him. I've spent several years answering people's random questions (librarian) in multiple settings and my first W2 job back in 2004 was technological education. I also have a Linguistics degree and can program so I understand what LLMs can and can't do fairly well even if I'm nowhere near an expert. He's vastly underestimating the average person's awareness of their own thought process or even ability to accurately answer iterative questions in order to sort things out interactively. The weakness with such a claim isn't the AI, it's the humans' ability to provide the AI with accurate and useful information. Not to mention that there's still a large portion of the general public that just freezes up when it comes to anything technological. I've provided support to a number of different communities and even educated people can't do this type of thinking easily. It's possible that we could eventually get to that point, but it would require some massive educational efforts and a culture shift, which would also require substantial investment without a clear road to profit, and I believe America lacks the political will or even ability to implement anything like this. Since we can't act in unison, it will be down to companies, and for them it makes more sense to solve the problem by hiring prompt engineers with the savings they get from cutting elsewhere instead of training all their existing teams on how to learn new ways of thinking. tl;dr: Technologically he's probably right, but he's massively overestimating the public's ability to use interactive tools to get information. > obvious thing that we would expect Any time you rely on something being 'obvious', the public is going to somehow fuck it up.

cloverich 4y ago
This is similar to the situation with software engineers interacting with other parts of the business or customers. It's IME the biggest skill of software engineers, not to write code, but to understand and translate business requirement in to code, while accounting for _existing_ (prior) requirements that were often forgotten. The code as such is usually straight forward, but the requirements on either end is where the work is. This is also what programmers typically complain about when they say their PM isn't "technical" enough. What they usually mean is they don't understand the details of the business / domain well enough to meaningfully contribute to the organization / break up of the work. To that end, I'd expect these language models to eventually improve at that task too, perhaps responding with choices or diagrams to help users learn how to ask what they actually want. That's in a sense what a prompt engineer is doing in this context. In that sense I think I agree w/ your assessment although I also think there's a lot of room for these AI's to handle that task, at least partially, as they improve.
- Mezzie 4y ago
  I agree. I'm not a SWE because I don't want to code for 40+ hours a week, but I've been picking up projects here and there for small businesses for a long time (I started helping my dad do the same when I was 5-6) and so I manage projects end to end and so much of it is the back and forth with the clients. Knowing the right questions to ask and how people think about things are some of my most important skills, and in that respect my education and library experience has been very helpful. I'm far from the best programmer - hell, I'd consider myself below average but have enough experience to still be passable, but I'm easy to work with and that counts for a lot. The issue with language models in particular is that to use them effectively you have to fundamentally understand that you're translating human-brain-language into computer-language and understand that those are in fact two different things even if their delivery happens identically. For most coders, this jump happens semi-instinctively because they know how to test something to verify 'ah yes, this follows computer logic'. The general public thinks computers are wizardry. The other group of people who will be able to understand this are those who have thought deeply about language for whatever reason and therefore know that human-brain-language has qualities that extend beyond 'We use words'. This is also very hard to impart to novices; I have a Linguistics degree and studied a cumulative 10 years of 6 languages so I went through that knowledge transfer as well. They're both hard and one or the other is necessary to interface with language models effectively and will be for likely the next 10ish years just because providing a good technical solution will require the technical people to understand much more about the social side of communication and that will also take time.

kzrdude 4y ago

The "sorted out interactively" that he mentions is exactly prompt engineering, isn't it?

rcme 4y ago
If you ask someone to do something for you and they ask a clarifying question, do you consider that prompt engineering?
- kzrdude 4y ago
  If it's a language model, yes. I think if it's in the initial prompt (first message) or second message to the AI doesn't make much difference for its role in the process.