6 ms·

Think of it more as "prompt-fu", akin to "google-fu"--or how to google. You do have to know how to talk to an LLM, and it will be important. But it's not so muc

by iamwil 4y ago

Think of it more as "prompt-fu", akin to "google-fu"--or how to google. You do have to know how to talk to an LLM, and it will be important. But it's not so much engineering, as we come to think of it in software traditionally. I wouldn't get hung up on the name, but just see it for what it is--a bad name for something we'll need to learn to be effective. It won't go away entirely, but will be mitigated to be less important as LLMs and AI get better.

coffeebeqn 4y ago
LLMs are very quirky. They need the correct context to produce something good and it’s often surprising what that needed context is. OP just play around with midjourney and you’ll soon realize that it is a very real thing
- ZephyrBlu 4y ago
  100%. People who have not tried to interact with these models don't understand there is actually quite a lot of skill in navigating the latent space.
  - mdp2021 4y ago
    It really should be just an intermediate phase. Proper, solid control system must be created. First you have Stable Diffusion, then you create ControlNet - and this as just a beginning. This is also one of the reasons why the problem of transparency ("but why and how does it work?") is so important in Machine Learning: allowing control.
    - ZephyrBlu 4y ago
      Adding a working control system doesn't take away the fact it requires skill though. It seems like people think that one day they will be able to give a vague prompt and generate exactly what they were looking for, which I don't believe will ever happen. It feels like there will always be some skill in navigating the model.
      - mdp2021 4y ago
        > there will always be some skill There is a world of difference between the needs in governance in just having to write "Those decorations where I marked the area, make them a bit more Hans Holbein as opposed to Klimt", compared to knowing patching tricks so that if you asked the engine to draw a 'T' you were not being served a 'B'. This latter example - ask 'B', get 'T' - I have seen from Midjourney only a few hours ago, in the public showcase. I have examined several prompts and results, and very often they do not overlap but very partially - the engine is like a wild horse. That this is not intrinsic, but just a current stage, is only part of the story: the user will want to have a tool that allows the production of what the artist had exactly in mind (not just "anything nice as /inspired/ by our suggestions"). It is the normal form of the product in its "ready for shipment" state.
        ZephyrBlu 4y ago
        I disagree it will ever be that easy for people to generate specifically what they’re looking for. I believe there will always be some amount of navigation of the latent space required, and people who don’t understand how to navigate it will struggle. To use your analogy, even if the horse is tame you still need to know how to ride it.
        mdp2021 4y ago
        > that easy Can you code? Can you illustrate? Can you build? Piece by piece, you sketch, implement, hone, finetune, improve, correct, retry, go into detail, expand... The ease of the process is relative to the tools adopted. Of course there has to be knowledge behind it. Of course you have to know your tools. But "wild horse" and "diligent consultant", axe and scalpel require different degrees areas and modalities of competence. A well built tool will build on the competences already natural for the professional (e.g. a graphic designer will be able to sketch and to instruct the machine in a reasonably standard language). Of course the better you know your tool, the greater the effectiveness. But currently, the systems can be strongly unreliable - they do not respect the request. The user will have to learn to deal with a machine taking natural language as an input just like any manager has to know how to speak to personnel to obtain what is wanted. On the other hand, the implementation of "virtual personnel" which is reliable is a responsibility of the tool developers.
    - SkyPuncher 4y ago
      Somethings will be easier, but it's still a lot like programming. You often need to do a lot of setup to accomplish a task. For somethings, there's just no way around creating a specific context. In many ways, it's the difference between using a library like lodash and writing the methods yourself. You still need to understand what's being done, a library just handles more things automatically for you.
    - toss1 4y ago
      This sounds like a joke I heard decades ago when I first worked with LISP programmers - when I commented about some unexpected result, he said "ah, you're looking for the DWIM command..." "What?"... "The Do What I Mean command, they're still working on it..." Decades later, and it looks like everyone is still working on the DWIM command...
  - rasz 4y ago
    Just like you wouldnt be able to drive, let alone start Ford T.
- deleted 4y ago
  [deleted]
Ekaros 4y ago
Specially with image generation and like I can believe that there is certain amount of learning and practise to get exactly the results you want and how different inputs affect output. And then also picking up right model for right job. Not that it is engineering, but there is lot of uses where it doesn't match either...
bank6062 4y ago
Prompt engineering has been fascinating to follow at my job. People believe that they will be using "prompt engineering" as a new way to verbally interact with peers. As if they now have found the Rosetta stone for their own quirks.
pcthrowaway 4y ago
Next you're going to say social engineering isn't real engineering :P
- didgeoridoo 4y ago
  I’m saving this one, thanks.
Waterluvian 4y ago
A great example of this hit me last week when a demo for prompting pixel art pointed out you need to date the video game you’re referencing. You might know you want “final fantasy 6 pixel art style” but there are many remakes of it.
cyanydeez 4y ago
Google Fi no long works. It's closer to prompt engineering.
- deleted 4y ago
  [deleted]
eagleshou 4y ago
when software first came out, us electrical engineers don't think this is "engineering" either, basically it is all people creating a imaginary machinary for other people operating in their imaging world.

by iamwil 4y ago

coffeebeqn 4y ago

LLMs are very quirky. They need the correct context to produce something good and it’s often surprising what that needed context is. OP just play around with midjourney and you’ll soon realize that it is a very real thing

ZephyrBlu 4y ago
100%. People who have not tried to interact with these models don't understand there is actually quite a lot of skill in navigating the latent space.
- mdp2021 4y ago
  It really should be just an intermediate phase. Proper, solid control system must be created. First you have Stable Diffusion, then you create ControlNet - and this as just a beginning. This is also one of the reasons why the problem of transparency ("but why and how does it work?") is so important in Machine Learning: allowing control.
  - ZephyrBlu 4y ago
    Adding a working control system doesn't take away the fact it requires skill though. It seems like people think that one day they will be able to give a vague prompt and generate exactly what they were looking for, which I don't believe will ever happen. It feels like there will always be some skill in navigating the model.
    - mdp2021 4y ago
      > there will always be some skill There is a world of difference between the needs in governance in just having to write "Those decorations where I marked the area, make them a bit more Hans Holbein as opposed to Klimt", compared to knowing patching tricks so that if you asked the engine to draw a 'T' you were not being served a 'B'. This latter example - ask 'B', get 'T' - I have seen from Midjourney only a few hours ago, in the public showcase. I have examined several prompts and results, and very often they do not overlap but very partially - the engine is like a wild horse. That this is not intrinsic, but just a current stage, is only part of the story: the user will want to have a tool that allows the production of what the artist had exactly in mind (not just "anything nice as /inspired/ by our suggestions"). It is the normal form of the product in its "ready for shipment" state.
      - ZephyrBlu 4y ago
        I disagree it will ever be that easy for people to generate specifically what they’re looking for. I believe there will always be some amount of navigation of the latent space required, and people who don’t understand how to navigate it will struggle. To use your analogy, even if the horse is tame you still need to know how to ride it.
        mdp2021 4y ago
        > that easy Can you code? Can you illustrate? Can you build? Piece by piece, you sketch, implement, hone, finetune, improve, correct, retry, go into detail, expand... The ease of the process is relative to the tools adopted. Of course there has to be knowledge behind it. Of course you have to know your tools. But "wild horse" and "diligent consultant", axe and scalpel require different degrees areas and modalities of competence. A well built tool will build on the competences already natural for the professional (e.g. a graphic designer will be able to sketch and to instruct the machine in a reasonably standard language). Of course the better you know your tool, the greater the effectiveness. But currently, the systems can be strongly unreliable - they do not respect the request. The user will have to learn to deal with a machine taking natural language as an input just like any manager has to know how to speak to personnel to obtain what is wanted. On the other hand, the implementation of "virtual personnel" which is reliable is a responsibility of the tool developers.
  - SkyPuncher 4y ago
    Somethings will be easier, but it's still a lot like programming. You often need to do a lot of setup to accomplish a task. For somethings, there's just no way around creating a specific context. In many ways, it's the difference between using a library like lodash and writing the methods yourself. You still need to understand what's being done, a library just handles more things automatically for you.
  - toss1 4y ago
    This sounds like a joke I heard decades ago when I first worked with LISP programmers - when I commented about some unexpected result, he said "ah, you're looking for the DWIM command..." "What?"... "The Do What I Mean command, they're still working on it..." Decades later, and it looks like everyone is still working on the DWIM command...
- rasz 4y ago
  Just like you wouldnt be able to drive, let alone start Ford T.
deleted 4y ago
[deleted]

Ekaros 4y ago

Specially with image generation and like I can believe that there is certain amount of learning and practise to get exactly the results you want and how different inputs affect output. And then also picking up right model for right job. Not that it is engineering, but there is lot of uses where it doesn't match either...

bank6062 4y ago

Prompt engineering has been fascinating to follow at my job. People believe that they will be using "prompt engineering" as a new way to verbally interact with peers. As if they now have found the Rosetta stone for their own quirks.

pcthrowaway 4y ago

Next you're going to say social engineering isn't real engineering :P

didgeoridoo 4y ago
I’m saving this one, thanks.

Waterluvian 4y ago

A great example of this hit me last week when a demo for prompting pixel art pointed out you need to date the video game you’re referencing. You might know you want “final fantasy 6 pixel art style” but there are many remakes of it.

cyanydeez 4y ago

Google Fi no long works. It's closer to prompt engineering.

deleted 4y ago
[deleted]

eagleshou 4y ago

when software first came out, us electrical engineers don't think this is "engineering" either, basically it is all people creating a imaginary machinary for other people operating in their imaging world.