3 ms·

Has it? I mean, maybe the idea of people doing this as a long-time career has, but practically, I still find it a challenge to get those AIs to do exactly what

by c7b 4y ago

Has it? I mean, maybe the idea of people doing this as a long-time career has, but practically, I still find it a challenge to get those AIs to do exactly what I want. I've played around with Dreambooth-style extensions now, and that goes some way for some applications, and I'm excited to try OP's solution, but in my experience, it is still a bit of a limitation for working with those AIs right now.

rjh29 4y ago
Oh yeah it's definitely still an issue right now! But I think the power of ChatGPT's ability to understand and execute instructions has convinced most people that "prompt engineering" isn't going to be a career path in the future.
- c7b 4y ago
  Absolutely. I briefly thought about asking ChatGPT to write a prompt, but then I remembered that the training corpus is probably older than those tools (I heard that if you ask it the right way, it will tell you that its corpus ended in 21 - whether it's true or not, it sounds plausible). But that's a truly temporary issue, the respective subreddits probably have enough information to train an AI for prompt engineering already (if you start from a strong foundation like the latest GPT versions). Plus, who knows whether future models won't be able to integrate those different modes much better (along those lines https://www.deepmind.com/publications/a-generalist-agent https://www.deepmind.com/publications/a-generalist-agent).
  - rjh29 4y ago
    In the near future you can totally imagine a dialogue like that you'd have with a real designer, "can you make it pop a bit more?" or "can you move that logo to the right side?". It might some trial and error but it's only going to improve. Making the AI truly creative (which means going beyond what the client asks for, towards things the client doesn't even know they want) would be a much larger leap and potentially take a lot longer.
- TeMPOraL 4y ago
  I don't get it. Pre-ChatGPT prompt engineering was a BS exercise in guessing how a given model's front-end tokekizes and processes the prompt. ChatGPT made it only more BS. But I've seen a paper the other day, implementing more structured, formal prompt language, with working logic operators implemented one layer below - instead of adding more cleverly structured English, they were stepping the language model with variations of the prompt (as determined by the operators), and did math on probability distributions of next tokens the model returned. That, to me, sounds like valid, non-BS approach, and strictly better than doubling down on natural language.
  - c7b 4y ago
    Think about the problem in an end-to-end fashion: the user has an idea of what sort of image they want, they just need an interface to tell the machine. A combination of natural language plus optional image/video input is probably the most intuitive interface we can provide (at least until we've made far more progress on reading brain signals more directly). How exactly we get there, by adding layers like on top like language models, or adding layers below like what you described, doesn't seem like such a fundamental difference. It's engineering, you try different approaches, vary your parameters and see what works best. And from the onset, natural language does seem like a good candidate for encoding nuances like "make it pink, but not cheesy" or "has the vibes of a 50's Soviet propaganda poster, but with friendlier colors".

by c7b 4y ago

rjh29 4y ago

Oh yeah it's definitely still an issue right now! But I think the power of ChatGPT's ability to understand and execute instructions has convinced most people that "prompt engineering" isn't going to be a career path in the future.

c7b 4y ago
Absolutely. I briefly thought about asking ChatGPT to write a prompt, but then I remembered that the training corpus is probably older than those tools (I heard that if you ask it the right way, it will tell you that its corpus ended in 21 - whether it's true or not, it sounds plausible). But that's a truly temporary issue, the respective subreddits probably have enough information to train an AI for prompt engineering already (if you start from a strong foundation like the latest GPT versions). Plus, who knows whether future models won't be able to integrate those different modes much better (along those lines https://www.deepmind.com/publications/a-generalist-agent https://www.deepmind.com/publications/a-generalist-agent).
- rjh29 4y ago
  In the near future you can totally imagine a dialogue like that you'd have with a real designer, "can you make it pop a bit more?" or "can you move that logo to the right side?". It might some trial and error but it's only going to improve. Making the AI truly creative (which means going beyond what the client asks for, towards things the client doesn't even know they want) would be a much larger leap and potentially take a lot longer.
TeMPOraL 4y ago
I don't get it. Pre-ChatGPT prompt engineering was a BS exercise in guessing how a given model's front-end tokekizes and processes the prompt. ChatGPT made it only more BS. But I've seen a paper the other day, implementing more structured, formal prompt language, with working logic operators implemented one layer below - instead of adding more cleverly structured English, they were stepping the language model with variations of the prompt (as determined by the operators), and did math on probability distributions of next tokens the model returned. That, to me, sounds like valid, non-BS approach, and strictly better than doubling down on natural language.
- c7b 4y ago
  Think about the problem in an end-to-end fashion: the user has an idea of what sort of image they want, they just need an interface to tell the machine. A combination of natural language plus optional image/video input is probably the most intuitive interface we can provide (at least until we've made far more progress on reading brain signals more directly). How exactly we get there, by adding layers like on top like language models, or adding layers below like what you described, doesn't seem like such a fundamental difference. It's engineering, you try different approaches, vary your parameters and see what works best. And from the onset, natural language does seem like a good candidate for encoding nuances like "make it pink, but not cheesy" or "has the vibes of a 50's Soviet propaganda poster, but with friendlier colors".