9 ms·

I just can't believe how idiotic such a thing is. It's like calling a person who uses a toaster a Toast Engineer with a salary of 300k.

deleted 4y ago
[deleted]
vsareto 4y ago
>Notably, chain of thought (CoT) prompting, a recent technique for eliciting complex multi-step reasoning through step-by-step answer examples, achieved the state-of-the-art performances in arithmetics and symbolic reasoning, difficult system-2 tasks that do not follow the standard scaling laws for LLMs https://arxiv.org/abs/2205.11916 https://arxiv.org/abs/2205.11916 >https://sites.google.com/view/automatic-prompt-engineer https://sites.google.com/view/automatic-prompt-engineer Not exactly a "toaster go brrrr" job, but it could be obsolete one day WaPo does need to chill though. There’s barely any Prompt Engineer jobs Edit: If anyone's curious, I've been following this for prompt stuff: https://github.com/dair-ai/Prompt-Engineering-Guide https://github.com/dair-ai/Prompt-Engineering-Guide
usea 4y ago
It's just software. It takes skill and labor to achieve the outputs you want. Is someone who uses photoshop all day not an artist? Or someone who writes text for a compiler not a programmer? You've got a huge blind spot if you think prompt engineer isn't already a thing.
- vba616 4y ago
  >You've got a huge blind spot if you think prompt engineer isn't already a thing. It may be a "thing", because generating BS is a viable business model and ChatGPT makes it more efficient. ..but I submit as a working hypothesis, that it is completely impossible to gain knowledge you do not already possess from a language model, no matter how clever your prompting. I'm very interested in counter-examples, but I have seen a few that turn out to be fake already.
  - eurasiantiger 4y ago
    Synthetic knowledge is still novel knowledge if you haven’t put the pieces together before.
    - vba616 4y ago
      Can you unpack "synthetic knowledge", cause I don't know what that really means.
      - eurasiantiger 4y ago
        New knowledge synthesized from existing knowledge. For example, you might know of A and B and maybe think that A -> B or B -> A based on their co-occurrence, but an AI might make you realize that C -> A as well as C -> B.
        vba616 4y ago
        Ok, that's straightforward, I just don't care for the idea that AI can do it or even help. You might synthesize new knowledge. When ChatGPT produces new output, it's not synthesizing new knowledge. It can't even output the knowledge it was trained with, as long as it lacks the ability to tag it in a trustworthy way. It's not that it's always BS, it's that it's almost always BS and if you don't know the answer in advance or independently, you can't distinguish it from anything within the model.
  - thfuran 4y ago
    That's so obviously false that I literally can't imagine how you could believe it. GPT-3 certainly isn't 100% accurate, but neither is it so perfectly unreliable that no one could ever get it to produce a relevant fact not in the prompt. And even if it were, it would probably still be potentially useful for learning languages.
    - vba616 4y ago
      > GPT-3 certainly isn't 100% accurate, but neither is it so perfectly unreliable that no one could ever get it to produce a relevant fact not in the prompt. I think I understand the sense in which you claim it produces relevant facts not in the prompt. It's not that we differ on easily observable behavior of the system. It's that I question if GPT-3 is "producing" these identifiable facts, and if the user is "producing" them instead, whether they can possibly be "relevant".
echelon 4y ago
> I just can't believe how idiotic such a thing is. Well, you do you. That's old world thinking for a field that's going to dramatically morph into something that barely resembles what we have today. I'm hiring a contract prompt engineer for my startup. If you want to help us achieve better "TV replacement" results, send me an email (see profile). https://fakeyou.com/news https://fakeyou.com/news (early demo, more coming soon!)
- deleted 4y ago
  [deleted]
- superdisk 4y ago
  Sounds good to me. Since the job doesn't require any skills, I think we should proceed to the negotiation phase. How does 500/hr sound?
  - echelon 4y ago
    > Since the job doesn't require any skills Oh like hell it requires no skill. You tell me how you'll generate better photos, improve dialogue coherence across multiple speakers, and control camera direction and movement (something we're using LLMs for too as we experiment with special-purposed models). All of this is not known a priori, by the way. And I won't accept building a database or lookup table as an answer. I also want to know how you'll test, benchmark, and refine. You also need to budget for inference complexity. I'm waiting :) I can do this myself, but it is a full time job. I am so busy with all other aspects of my business I'm looking for people to bring on board.
    - superdisk 4y ago
      I normally don't work for free, but just as a sample, here's one I've been crafting for a few weeks... Epic 4k HD photo, high res and epic, cool extra awesome photorealistic 5k or 6k, realistic, in the style of a really good photographer. Nah I'm just playing, your company looks pretty cool, I just think a dedicated job for coming up with prompts (which is only going to become easier anyway with better ways to control output) is silly.
reducesuffering 4y ago
Welcome to the last decade of title inflation. Everyone is a "manager" now. A "marketing manager", "product manager", "account manager". No more secretary, it's "executive assistant". It's a perk a company can offer, conferring higher status, at no expense to themselves. So the equilibrium is for other companies to do the same, otherwise the company that gives this cost-free perk outcompetes for talent. People are graduating watered-down educations, earning inflated cash, with inflated titles. It all helps people believe they're higher status, that they have a university degree and are a manager earning $80k, surely they're getting close to the top of the totem pole now. But they have a worse standard of living and education equivalent to high school in the '60s.
- voytec 4y ago
  Related: Companies save billions of dollars by giving employees fake "manager" titles[1] [1] https://news.ycombinator.com/item?id=34641549 https://news.ycombinator.com/item?id=34641549 [1] https://www.cbsnews.com/news/salary-manager-jobs-fake-titles-save-4-billion-overtime-nber/ https://www.cbsnews.com/news/salary-manager-jobs-fake-titles...
  - antibasilisk 4y ago
    'assistant regional manager'
    - impish9208 4y ago
      Assistant to the regional manager
abootstrapper 4y ago
You’re thinking of a Toast Master.
teaearlgraycold 4y ago
I worked with a prompt engineer. They're basically a Natural Language Programmer. There is skill in doing the job well.
tanto 4y ago
The naming might be flawed but it's like a batista I guess. You can spend a lot of money on the machine but without an expert to use it you will not get the best out of it.
- dmonitor 4y ago
  Just train an LLM on LLM prompts and automate the prompt engineer process.
  - p1esk 4y ago
    And how do you get this LLM to output a good prompt?
- xoac 4y ago
  Yes but you can teach most of what there is to know in a couple of weeks tops. You don't call baristas Coffee Engineers.
  - jareklupinski 4y ago
    starbucks takes notes furiously
    - jjdeveloper 4y ago
      Best comment! lol
p1esk 4y ago
No more idiotic than a lawyer - someone who knows the right words to convince a jury (model) to produce a desired outcome.
- dijit 4y ago
  I dislike that analogy greatly. Lawyers have a bad reputation, sure, but there’s a lot of education about the interpretation of our law and the absurdly large corpus of legal documentation that must be read in order to even become a lawyer is far and above anything you describe.
  - p1esk 4y ago
    To be an effective prompt engineer you need to have expertise in two different domains - large generative ML models, and in whatever it is people want to generate (e.g. art).
  - deeviant 4y ago
    So you are presenting the fact that lawyers must digest an absurdly large corpus of legal documentation, but also maintaining that is something a human lawyer has some sort of advantage over a LLM?
    - vba616 4y ago
      A LLM is a generator of misinformation that is maximally difficult to distinguish from real information. How do you use this as a lawyer? I mean, as a stereotypical evil lawyer in a world of naïve people who don't learn from experience, you could maybe use it to win cases until you destroy the justice system. But other than that...
      - deeviant 4y ago
        Speaking as somebody who spent thousands of dollars for a lawyer on a matter, with basically no results, and also who used chatGPT, personal research, and common sense to solve this same matter for free, I can only say if a LLM is "a generator of misinformation that is maximally difficult to distinguish from real information" a lawyer is simply "a human who has been trained to maximally drain your wallet without regard for any other matter", of course, neither is true and there exists far more nuance for both. Sure, there are matters I would only trust a lawyer to handle, but there are a great many I wouldn't. Further, the average quality of a human lawyer will likely remain the same tomorrow as it is today, while AI will only get better. LLM today, perhaps some hybrid stack tomorrow, it's only a matter of time before an AI lawyer is the way to go for just about any legal matter. And let me be clear, that time might be 10 years, or may be 100+, but it is coming.
cfn 4y ago
I still remeber when Google appeared there were people offering search services for a fee (maybe they were called Search Engineers but I don't recal seeing that).
TrackerFF 4y ago
Not too different from analysts writing SQL queries all day long.
- super256 4y ago
  Complex SQL requires knowledge about relational algebra (Cartesian products, set theory, domain relational calculus yada yada) and understanding of how RDBMS and their query planners work. At least, if performance is important to you. I don't see this in this prompt engineering. In my limited experience (I played a few hours with Stable Diffusion and more hours with the OAI davinci-003 model), you can get good at it within a few days.
  - p1esk 4y ago
    I spent a few hours learning SQL. I can get “good” at writing SQL queries within a few days. Do you want to hire me as someone who’s primary role is to write SQL queries?
  - grufus 4y ago
    [flagged]
    - teaearlgraycold 4y ago
      I don’t think that’s Dunning-Kruger but I get what you’re saying.
      - grufus 4y ago
        Not? "After a few hours of playing with it I give the opinion that it can't be that hard" is kinda the essence of basically not having a clue but feeling competent to express such a judgement nevertheless.
        teaearlgraycold 4y ago
        Fair enough. I think at the time I felt that just one data point wasn't enough to evoke the effect. Maybe I'm just biased against the effect since it was dis-proven: https://economicsfromthetopdown.com/2022/04/08/the-dunning-kruger-effect-is-autocorrelation/ https://economicsfromthetopdown.com/2022/04/08/the-dunning-k...
paxys 4y ago
Just about as idiotic as calling someone who types some weird words on a screen a Software Engineer with a salary of 300k.
krackers 4y ago
>This morning, I brushed and flossed, which makes me a Plaque Removal Engineer. I then used my skills as a Room Tidyness Engineer to make the bed. After that, I engineered the harness onto my dog and took her on a walk: Canine Fitness Engineer. I engineered the water to a higher temperature using the kettle, and poured it over some coffee grounds to create a chemical reaction in my morning brew: Yeah, that's right, Caffeine Engineer. After this incredibly productive morning, I got in the car and drove to my job as a computer programmer. -mlsu: https://news.ycombinator.com/item?id=34884683 https://news.ycombinator.com/item?id=34884683 At this point the word "engineer" has lost its original meaning. Until there's a formal theory of how we can interact with LLMs and you make use of that in a systematic fashion, "prompt engineering" is really closer to "prompt artist."
- grufus 4y ago
  > At this point the word "engineer" has lost its original meaning. Until there's a formal theory of how we can interact with LLMs and you make use of that in a systematic fashion, "prompt engineering" is really closer to "prompt artist." Interesting angle. Are you saying there are rarely any "software engineers" out there, that they are all merely "software artists"? Cause none of these uses a formal theory for their craft. If they were then all those highly opinionated discussions of whether to use goto in C or what are the greatest flaws of node.js would just not exist.
  - krackers 4y ago
    Correct, in my eyes "software engineering" in the sense of "person who glues libraries together to build systems" should not yet be called an engineering discipline because there isn't yet a rigorous enough theory on why something should be designed one way as opposed to another. We are still in the stage of figuring out best practices and making things more rigorous (maybe the functional programming folks will end up contributing something here, but I don't know enough about category theory). There are other narrower senses of "software engineer" such as "person who optimizes code" and to me those more qualify as engineering because we not only have a decent enough theoretical background (see Agner Fog's work) but also can experimentally verify things. On the other hand it's a lot harder to quantitatively say if one design is better than another. I think there's also some work in terms of rigorously modeling concurrent/distributed systems (Lamport's TLA+) work which I'd like to see more of.

deleted 4y ago

[deleted]

vsareto 4y ago

>Notably, chain of thought (CoT) prompting, a recent technique for eliciting complex multi-step reasoning through step-by-step answer examples, achieved the state-of-the-art performances in arithmetics and symbolic reasoning, difficult system-2 tasks that do not follow the standard scaling laws for LLMs https://arxiv.org/abs/2205.11916 https://arxiv.org/abs/2205.11916 >https://sites.google.com/view/automatic-prompt-engineer https://sites.google.com/view/automatic-prompt-engineer Not exactly a "toaster go brrrr" job, but it could be obsolete one day WaPo does need to chill though. There’s barely any Prompt Engineer jobs Edit: If anyone's curious, I've been following this for prompt stuff: https://github.com/dair-ai/Prompt-Engineering-Guide https://github.com/dair-ai/Prompt-Engineering-Guide

usea 4y ago

It's just software. It takes skill and labor to achieve the outputs you want. Is someone who uses photoshop all day not an artist? Or someone who writes text for a compiler not a programmer? You've got a huge blind spot if you think prompt engineer isn't already a thing.

vba616 4y ago
>You've got a huge blind spot if you think prompt engineer isn't already a thing. It may be a "thing", because generating BS is a viable business model and ChatGPT makes it more efficient. ..but I submit as a working hypothesis, that it is completely impossible to gain knowledge you do not already possess from a language model, no matter how clever your prompting. I'm very interested in counter-examples, but I have seen a few that turn out to be fake already.
- eurasiantiger 4y ago
  Synthetic knowledge is still novel knowledge if you haven’t put the pieces together before.
  - vba616 4y ago
    Can you unpack "synthetic knowledge", cause I don't know what that really means.
    - eurasiantiger 4y ago
      New knowledge synthesized from existing knowledge. For example, you might know of A and B and maybe think that A -> B or B -> A based on their co-occurrence, but an AI might make you realize that C -> A as well as C -> B.
      - vba616 4y ago
        Ok, that's straightforward, I just don't care for the idea that AI can do it or even help. You might synthesize new knowledge. When ChatGPT produces new output, it's not synthesizing new knowledge. It can't even output the knowledge it was trained with, as long as it lacks the ability to tag it in a trustworthy way. It's not that it's always BS, it's that it's almost always BS and if you don't know the answer in advance or independently, you can't distinguish it from anything within the model.
- thfuran 4y ago
  That's so obviously false that I literally can't imagine how you could believe it. GPT-3 certainly isn't 100% accurate, but neither is it so perfectly unreliable that no one could ever get it to produce a relevant fact not in the prompt. And even if it were, it would probably still be potentially useful for learning languages.
  - vba616 4y ago
    > GPT-3 certainly isn't 100% accurate, but neither is it so perfectly unreliable that no one could ever get it to produce a relevant fact not in the prompt. I think I understand the sense in which you claim it produces relevant facts not in the prompt. It's not that we differ on easily observable behavior of the system. It's that I question if GPT-3 is "producing" these identifiable facts, and if the user is "producing" them instead, whether they can possibly be "relevant".

echelon 4y ago

> I just can't believe how idiotic such a thing is. Well, you do you. That's old world thinking for a field that's going to dramatically morph into something that barely resembles what we have today. I'm hiring a contract prompt engineer for my startup. If you want to help us achieve better "TV replacement" results, send me an email (see profile). https://fakeyou.com/news https://fakeyou.com/news (early demo, more coming soon!)

deleted 4y ago
[deleted]
superdisk 4y ago
Sounds good to me. Since the job doesn't require any skills, I think we should proceed to the negotiation phase. How does 500/hr sound?
- echelon 4y ago
  > Since the job doesn't require any skills Oh like hell it requires no skill. You tell me how you'll generate better photos, improve dialogue coherence across multiple speakers, and control camera direction and movement (something we're using LLMs for too as we experiment with special-purposed models). All of this is not known a priori, by the way. And I won't accept building a database or lookup table as an answer. I also want to know how you'll test, benchmark, and refine. You also need to budget for inference complexity. I'm waiting :) I can do this myself, but it is a full time job. I am so busy with all other aspects of my business I'm looking for people to bring on board.
  - superdisk 4y ago
    I normally don't work for free, but just as a sample, here's one I've been crafting for a few weeks... Epic 4k HD photo, high res and epic, cool extra awesome photorealistic 5k or 6k, realistic, in the style of a really good photographer. Nah I'm just playing, your company looks pretty cool, I just think a dedicated job for coming up with prompts (which is only going to become easier anyway with better ways to control output) is silly.

reducesuffering 4y ago

Welcome to the last decade of title inflation. Everyone is a "manager" now. A "marketing manager", "product manager", "account manager". No more secretary, it's "executive assistant". It's a perk a company can offer, conferring higher status, at no expense to themselves. So the equilibrium is for other companies to do the same, otherwise the company that gives this cost-free perk outcompetes for talent. People are graduating watered-down educations, earning inflated cash, with inflated titles. It all helps people believe they're higher status, that they have a university degree and are a manager earning $80k, surely they're getting close to the top of the totem pole now. But they have a worse standard of living and education equivalent to high school in the '60s.

voytec 4y ago
Related: Companies save billions of dollars by giving employees fake "manager" titles[1] [1] https://news.ycombinator.com/item?id=34641549 https://news.ycombinator.com/item?id=34641549 [1] https://www.cbsnews.com/news/salary-manager-jobs-fake-titles-save-4-billion-overtime-nber/ https://www.cbsnews.com/news/salary-manager-jobs-fake-titles...
- antibasilisk 4y ago
  'assistant regional manager'
  - impish9208 4y ago
    Assistant to the regional manager

abootstrapper 4y ago

You’re thinking of a Toast Master.

teaearlgraycold 4y ago

I worked with a prompt engineer. They're basically a Natural Language Programmer. There is skill in doing the job well.

tanto 4y ago

The naming might be flawed but it's like a batista I guess. You can spend a lot of money on the machine but without an expert to use it you will not get the best out of it.

dmonitor 4y ago
Just train an LLM on LLM prompts and automate the prompt engineer process.
- p1esk 4y ago
  And how do you get this LLM to output a good prompt?
xoac 4y ago
Yes but you can teach most of what there is to know in a couple of weeks tops. You don't call baristas Coffee Engineers.
- jareklupinski 4y ago
  starbucks takes notes furiously
  - jjdeveloper 4y ago
    Best comment! lol

p1esk 4y ago

No more idiotic than a lawyer - someone who knows the right words to convince a jury (model) to produce a desired outcome.

dijit 4y ago
I dislike that analogy greatly. Lawyers have a bad reputation, sure, but there’s a lot of education about the interpretation of our law and the absurdly large corpus of legal documentation that must be read in order to even become a lawyer is far and above anything you describe.
- p1esk 4y ago
  To be an effective prompt engineer you need to have expertise in two different domains - large generative ML models, and in whatever it is people want to generate (e.g. art).
- deeviant 4y ago
  So you are presenting the fact that lawyers must digest an absurdly large corpus of legal documentation, but also maintaining that is something a human lawyer has some sort of advantage over a LLM?
  - vba616 4y ago
    A LLM is a generator of misinformation that is maximally difficult to distinguish from real information. How do you use this as a lawyer? I mean, as a stereotypical evil lawyer in a world of naïve people who don't learn from experience, you could maybe use it to win cases until you destroy the justice system. But other than that...
    - deeviant 4y ago
      Speaking as somebody who spent thousands of dollars for a lawyer on a matter, with basically no results, and also who used chatGPT, personal research, and common sense to solve this same matter for free, I can only say if a LLM is "a generator of misinformation that is maximally difficult to distinguish from real information" a lawyer is simply "a human who has been trained to maximally drain your wallet without regard for any other matter", of course, neither is true and there exists far more nuance for both. Sure, there are matters I would only trust a lawyer to handle, but there are a great many I wouldn't. Further, the average quality of a human lawyer will likely remain the same tomorrow as it is today, while AI will only get better. LLM today, perhaps some hybrid stack tomorrow, it's only a matter of time before an AI lawyer is the way to go for just about any legal matter. And let me be clear, that time might be 10 years, or may be 100+, but it is coming.

cfn 4y ago

I still remeber when Google appeared there were people offering search services for a fee (maybe they were called Search Engineers but I don't recal seeing that).

TrackerFF 4y ago

Not too different from analysts writing SQL queries all day long.

super256 4y ago
Complex SQL requires knowledge about relational algebra (Cartesian products, set theory, domain relational calculus yada yada) and understanding of how RDBMS and their query planners work. At least, if performance is important to you. I don't see this in this prompt engineering. In my limited experience (I played a few hours with Stable Diffusion and more hours with the OAI davinci-003 model), you can get good at it within a few days.
- p1esk 4y ago
  I spent a few hours learning SQL. I can get “good” at writing SQL queries within a few days. Do you want to hire me as someone who’s primary role is to write SQL queries?
- grufus 4y ago
  [flagged]
  - teaearlgraycold 4y ago
    I don’t think that’s Dunning-Kruger but I get what you’re saying.
    - grufus 4y ago
      Not? "After a few hours of playing with it I give the opinion that it can't be that hard" is kinda the essence of basically not having a clue but feeling competent to express such a judgement nevertheless.
      - teaearlgraycold 4y ago
        Fair enough. I think at the time I felt that just one data point wasn't enough to evoke the effect. Maybe I'm just biased against the effect since it was dis-proven: https://economicsfromthetopdown.com/2022/04/08/the-dunning-kruger-effect-is-autocorrelation/ https://economicsfromthetopdown.com/2022/04/08/the-dunning-k...

paxys 4y ago

Just about as idiotic as calling someone who types some weird words on a screen a Software Engineer with a salary of 300k.

krackers 4y ago

>This morning, I brushed and flossed, which makes me a Plaque Removal Engineer. I then used my skills as a Room Tidyness Engineer to make the bed. After that, I engineered the harness onto my dog and took her on a walk: Canine Fitness Engineer. I engineered the water to a higher temperature using the kettle, and poured it over some coffee grounds to create a chemical reaction in my morning brew: Yeah, that's right, Caffeine Engineer. After this incredibly productive morning, I got in the car and drove to my job as a computer programmer. -mlsu: https://news.ycombinator.com/item?id=34884683 https://news.ycombinator.com/item?id=34884683 At this point the word "engineer" has lost its original meaning. Until there's a formal theory of how we can interact with LLMs and you make use of that in a systematic fashion, "prompt engineering" is really closer to "prompt artist."

grufus 4y ago
> At this point the word "engineer" has lost its original meaning. Until there's a formal theory of how we can interact with LLMs and you make use of that in a systematic fashion, "prompt engineering" is really closer to "prompt artist." Interesting angle. Are you saying there are rarely any "software engineers" out there, that they are all merely "software artists"? Cause none of these uses a formal theory for their craft. If they were then all those highly opinionated discussions of whether to use goto in C or what are the greatest flaws of node.js would just not exist.
- krackers 4y ago
  Correct, in my eyes "software engineering" in the sense of "person who glues libraries together to build systems" should not yet be called an engineering discipline because there isn't yet a rigorous enough theory on why something should be designed one way as opposed to another. We are still in the stage of figuring out best practices and making things more rigorous (maybe the functional programming folks will end up contributing something here, but I don't know enough about category theory). There are other narrower senses of "software engineer" such as "person who optimizes code" and to me those more qualify as engineering because we not only have a decent enough theoretical background (see Agner Fog's work) but also can experimentally verify things. On the other hand it's a lot harder to quantitatively say if one design is better than another. I think there's also some work in terms of rigorously modeling concurrent/distributed systems (Lamport's TLA+) work which I'd like to see more of.