9 ms·

Unpredictable black boxes are terrible interfaces

217 points by tim_sw 4y ago

duxup 4y ago
As someone who struggles finding good prompts when I goof around with this stuff this article makes me feel better. There’s so much “hey look at this from AI” and I look at what I get and it’s… not good/ frustrating.
- turmeric_root 4y ago
  A lot of the 'look what I made with AI' images that get shared around also don't include the creator's workflow. There's usually lots of trial-and-error, manual painting/inpainting, multiple models involved etc. and explaining all that is a lot harder than just saying 'I used stable diffusion'.
rektide 4y ago
They are against Natural Science & the enlightenment of humanity. Existence is wonderfully observable & learnable. It took human kind to create this particularly infernal information void, to have manufactured such a degrading & omnipresent local system of black holes. These are woefully against all spirit & should be torn down. More positively, there's so many positive pro-human pro-Augment-Intellect things tech could be doing; we need new powerful positive symbols that establish visible legible expansion of understandability.
- AussieWog93 4y ago
  Are you ok man?
  - turmeric_root 4y ago
    they're just microdosing it's ok
- anonymouskimmer 4y ago
  > there's so many positive pro-human pro-Augment-Intellect things tech could be doing But can it do it without baby steps?
- StrangeATractor 4y ago
  I agree. It's one thing to delegate labor to a machine, but I find myself very uncomfortable when the machine takes the role of creative expression. Art is a form of communication which transmits feeling, experience and emotion, but there's nothing a machine puts into this, there's nothing being transmitted, it's just a rendering, it's just pushing your buttons that make you go 'neato.' I fear people will start taking the beautiful and profound for granted, it'll just be content.
  - rektide 4y ago
    I dont particularly mind there being some places where tech is in the lead, where it can claim the initiative. But it just feels like humanity secedes more and more ground & understanding to humanity each month, and there's so few visible notable frontiers where humanity is staking down real wins. Rather than have advances in general purpose computing, we keep getting more and more apps, more and more far-off cloud systems.
    - rektide 4y ago
      *to tech
  - IanCal 4y ago
    On the other side, I'm astounded that I can ask a machine to convey something it wants to convey and get something out that makes me feel something. > but there's nothing a machine puts into this What makes you so sure?
    - grugagag 4y ago
      > ask a machine to convey something it wants to convey What makes you think a machine has any wants and desires?
      - IanCal 4y ago
        I'm a squishy machine and I have them, for as far as those terms have any meaning. What measure or process do you have of determining if I have them?
boringuser2 4y ago
Not too interesting to solve a "problem" that solves itself. Just going from GPT-3.5 to GPT-4 I'd say intent inference has improved by an order of magnitude. It almost feels mind-reader-ish. I've been waffling because I'm trying to cope with the absurdity of this product by highlighting the flaws (it can fail badly on tough problems), but we've gone from CleverBot to a genie that can instantly analyze your written intent and produce a highly logically coherent, well-worded analysis in a very short period of time.
- ChatGTP 4y ago
  I’d double check the code. I have a similar experience but when I actually look at what’s been returned I find strange things in there. Like it often works, but it’s rarely as correct as it seems. At least for a given use case.
  - boringuser2 4y ago
    I read all of the code it writes.
  - rjbwork 4y ago
    I agree. One of the engineers on my team came to me with a script (maybe 20 lines of bash) like "hey isn't it cool that I got ChatGPT to generate this?" and after studying it for a few minutes I was like "it is cool, but now spot the bug". It was near the end of the day so I asked them to follow up with me in the morning after they'd found and fixed the bug. I ended up having to point it out directly the next day. It was a spurious -A NUM argument that would, in a production scenario, cause it to recognize valid data as invalid because the full context of the data would be missing. All this to say, it is a very useful tool, but we should be highly suspicious of its output, and really make sure we understand what it's giving us. The ability to read and mentally parse and execute code will become extremely important, and give people with that ability a leg up.
    - boringuser2 4y ago
      Give me the snippet and I guarantee I can get GPT-4 to identify the bug with no hints and minimal prompting.
WFHRenaissance 4y ago
>>> When users cannot predict how input controls affect outputs they have to resort to trial-and-error, which is frustrating. This is a major issue when using generative AI for creating new content and it will remain an issue as long as the mapping between the input controls and outputs is unclear. But we can improve AI interfaces by enabling conversational interactions that can let users establish common ground/shared semantics with the AI, and that provide repair mechanisms when such shared semantics are missing. And this, ladies and gentlemen, is the reasoning for working on your prompt engineering. The prompt is the interface. Prompting is a communication skill, a management skill, and a technical skill. I doubt many will be hired as a "prompt engineers" in coming years (some already have been, by idiotic corps) - but being a bad prompt engineer will be a noticeable lack. Start practicing.
- kevingadd 4y ago
  Are you saying that a "good prompt" has the same properties no matter what network you're feeding it to? If so, why? It seems to me that writing a good prompt would require an understanding of the nature of the network you're using, at a minimum - for example "trending on artstation" is common in image generation prompts, and that's not going to be very useful input when generating text for a blog post. If not, what is the specific thing people should be practicing right now?
- sshumaker 4y ago
  This is really a lot less necessary with GPT-4. What required careful prompting in 3.5 often you can give it something slapdash in 4 and it can do a great job figuring out intent.
- chewxy 4y ago
  Sooner or later we'd have to be more and more precise with prompts. We'd be going from "I want a function that takes anything and returns itself" to "I : a -> a" ... and hey that looks like a programming language
  - stale2002 4y ago
    It seems like the hottest new programming language is the English language.
skepticATX 4y ago
And this is why I can't get behind LLMs as a universal GUI replacement. Not in their current form. We know that users are increasingly likely to leave a page as load time increases. Now what happens when you have an unpredictable black box that sometimes doesn't do what the user expects, or flat out refuses to cooperate? This is poor UX for many tasks.
- Gigachad 4y ago
  I can see LLMs being a better interface for the command line. Where I can just say "give me the ffmpeg command to flip a video upside down" ChatGPT gave me "ffmpeg -i input.mp4 -vf "vflip" output.mp4" and then explained what each part of the command does. I searched on google and the results were all for random tools when I specified ffmpeg, and the ones that were correct had complex walls of text, adverts everywhere, and other bloat.
  - Izkata 4y ago
    My first non-ad result on Google for "ffmpeg flip video upside-down" was [0], which includes both that and several other options including rotating 180 degrees (a different way of getting upside-down). Knowing more precisely that rotation isn't what I want, googling for "ffmpeg vertical flip video" sends me to [1] in the very first result, which is the same as what you got. Except it also specifies copying the audio without reencoding it, I'm not sure what your version would do without including that. [0] https://stackoverflow.com/questions/3937387/rotating-videos-with-ffmpeg https://stackoverflow.com/questions/3937387/rotating-videos-... [1] https://filme.imyfone.com/video-edit-tutorials/ffmpeg-flip-video/ https://filme.imyfone.com/video-edit-tutorials/ffmpeg-flip-v...
    - Gigachad 4y ago
      The first link has the entire "above the fold" content as useless bloat including ironically a huge block about SO closing the question as off topic. There is some good content in there if you dig though. The second one has a huge number of annoying popups all over the page (4 visible with adblock enabled). The ChatGPT one just leads with the answer and explains it after. It's a non destructive command so if I later decide I also want audio, I can just continue the conversation and ask for audio to be included.
lamuswawir 4y ago
I couldn't agree more, these tools are very useful, but interaction is almost just like that with humans, you use a word, they understand it as another. So you have to correct them, to explicitly tell them what you want. They are very good tools when you they and you are talking about the same thing. Indeed just like human conversation. The instant feedback is better than with humans. Like humans, these models can take one thing to mean another, but because we are still using them for tasks that require instant feedback, we can catch the errors early and correct. It becomes worse if the are to be used on longer lasting tasks. The entire project could be derailed by one misunderstood concept. Just like a human project. Natural language is a fair interface. But with lots of ambiguity. When I say 'clear', do you mean understandable, easily recognisable or clear like water. I faced that dilemma when I used that word as a design prompt because it can mean things which are opposites of each other. Some languages are so rich they avoid such ambiguities. A common language can be good. It should not be designed by a committee, it should evolve with these tools naturally.
__alias 4y ago
Amazing interfaces for particular scenarios if you look at them from another angle though. Yesterday I wanted to write a bash script to do a particular task. " I never properly learnt bash and it takes me a very long time to do something simple, furthermore, in this case, I didn't actually know the particular tools which were needed for what I wanted to write. After inputting, it correctly solves the exact problem for me which was perfect. Working backwards, I then asked it to explain what each part of the script was doing -- and learned the syntax for sed etc, along with gaining a better understanding of what what each command was doing. The point i'm trying to make here, is that it's amazing to tackle some interactions more naturally. Top down, instead of bottom up. This can help reduce the paralysis that occurs when working with the unknown. Someone very skilled with photoshop might understand the tool very well and work with particular tools to come up with a result. but Natural language can have you provide the desired end result and work down to fine details.
- taberiand 4y ago
  I've had the same experience (near exact same it seems, with writing bash scripts) - using it as an iterative guided tool provides a bright light to follow along an otherwise uncertain path, when you know where you want to go but not exactly how to get there, yet you've got enough experience to know when you're going off track.
- onion2k 4y ago
  and learned the syntax for sed etc, Did you actually learn it though, enough so you understand it and can recall it? If you did then next time you need a bash script you'll be able to quickly figure out the basics of it, or at least Google for something and modify it with your new knlwledge. I don't think it's much of a stretch to suggest that you won't do that and you'll just use GPT again instead. And next time you might not bother asking it to explain everything... That's not a criticism of you or of GPT. You have an awesome new tool that magically writes things for you. The most efficient way to use it is to let it do its magic and move on with more important things. All I'm suggesting is that most people, given such a tool, will use it and not learn from it.
  - 4y ago
cwkoss 4y ago
I feel like this is valid criticism for Dall-e, but not so much for stable diffusion. Stable diffusion (and to a lesser extent midjourney) exposes a LOT more knobs to tune. Layer on controlnet and other extensions, and you can really iteratively refine an image to get closer and closer to what you want. The key is to be able to 'lock in' a depth map, rough composition, seed, etc, and then tune the other parameters to refine what you're looking for. It's definitely not a one-shot solution, but its still miles faster than learning how to do digital painting and manually rendering things. I don't feel like it matters if you have to generate 16 images to find one to start building off of when each image can be rendered in 1/1000th the time a human could.
milchek 4y ago
What i find interesting are the discussions around good vs bad prompts and being adept at “prompt engineering” We’ve had this in the engineering world a long time in the form of Google and StackOverflow searches. The great engineers i’ve worked with know most stuff they need day-to-day, but when they run into issues it’s their ability to search for and find the right answer that gives them advantage over others who don’t know where to look (or even start).
- ChatGTP 4y ago
  Using an LLM yesterday to code, there was no way I’d be proficient with the workflow if I wasn’t an experienced programmer. I’ve never seen code look so correct, sort of work but be so strangely crap.
  - chillfox 4y ago
    I used to think that LLMs for coding would mostly be a tool to make experienced developers much more productive, but after coming across this video I think it will be used by normal people with no programming experience to make small basic programs. https://www.youtube.com/watch?v=IyKKhxYJ4U4 https://www.youtube.com/watch?v=IyKKhxYJ4U4
    - otabdeveloper4 4y ago
      Normal people with no programming experience can already make small basic programs by copy-pasting code from Github and StackOverflow.
      - lordnacho 4y ago
        This is so much harder than most people think. I would reckon that as soon as a complication shows up, 75% of non coders will get helplessly stuck. Complication meaning something slightly beyond changing variable names: you need a different if condition, the snippet doesn't contain the required import, the version has advanced slightly.
      - 4y ago
anonymouskimmer 4y ago
> Even worse, it is unclear to me how to change the prompt to move the image towards the image I want. An image program I use has sliders for things called "High", "Low ", and "Gamma". I don't know what these mean, but I can immediately see the effects they have when I use them. I can also reverse the effects back to the original image by putting them back. I don't know if AI could easily reverse back changes, at least over a couple of iterations. But I wonder if things like sliders or knobs could be added to AI interfaces to allow more gradual iteration of a text or image, or idea of a text or image, than having the AI generate an iteration based on a new plain text prompt, or even "conversational interface". You could have a ton of sliders with names that the user doesn't understand. You could link sliders together so that moving one moves them all, and then unlink individual ones as desired to see what they do. This would provide more immediate and tailorable feedback than trying to get your words just right.
- dendrite9 4y ago
  A while ago someone posted https://www.calligrapher.ai/ https://www.calligrapher.ai/ (discussion: https://news.ycombinator.com/item?id=34530011 https://news.ycombinator.com/item?id=34530011) The sliders were interesting and generally useful but I found it frustrating that they were not repeatable. Since then I've started to think I was approaching the sliders the wrong way by assuming they directly controlled the output, rather than being weights in one direction or another.
- danybittel 4y ago
  Totally agree. Something like: http://vecg.cs.ucl.ac.uk/Projects/projects_fonts/projects_fonts.html http://vecg.cs.ucl.ac.uk/Projects/projects_fonts/projects_fo... but not only for fonts.
- 8n4vidtmkvmk 4y ago
  there are sliders for things like "cfg" and "denoising" and various other things. with some trial and error you can figure out what they do and how to tune them, the issue is that image generation still takes 10 or 30 seconds so it's not a quick process. but yes, I'd still welcome more dials. more is a bigger pain to get started but useful after you figure them out
blueblimp 4y ago
The recommendation to support conversational interactions is very good. After all, if you consult with a human expert, one of the main things they'll be doing is conversing with you to figure out your requirements. I find it a bit frustrating that even ChatGPT will give you a giant answer straightaway instead of clarifying your requirements first. (Prompting can help with this somewhat, if you explicitly tell it to ask clarifying questions.)
koromak 4y ago
I feel like bad UX is the least concerning problem we're going to have to deal with.
bighoki2885000 4y ago
[dead]
vrglvrglvrgl 4y ago
[dead]
hyperthesis 4y ago
The computer problems golden-era science fiction predicted have finally come true. Also reminds me of Monty Python's Stay Here and Make Sure He Doesn't Leave https://youtube.com/watch?v=2f5MvVx8RM8 https://youtube.com/watch?v=2f5MvVx8RM8
Izkata 4y ago
The refrigerator "true system model" has the numbers and letters swapped from the instructions in the previous image.
bob1029 4y ago
We are finding that using context hints injected dynamically into ChatGPT prompts by way of separate classifiers is potentially the ultimate solution for many business needs. Having the very first thing be an explicit natural language feature parsing phase can make the downstream generative stuff way more predictable. We could actually go back and look at the traces around a user's original prompt to determine why it might have generated nonsense downstream. The challenging part of this is inverting everything and figuring out what all of your classes are and how to arrange training data to fit them. Set theory is another new dragon here. Smaller models like Ada and Babbage are ideal for this, but we are looking at 90's tech too (SVMs). If we need 10k separate models because we have so many classes, then I'd prefer to be able to train them in a few seconds/minutes on a CPU. Fine tuning gigantic models like Davinci is a dead-end at this point IMO. A fleet of microscopic models working together to feed 1 big powerful (untunable) model feels a lot more manageable.
- raldi 4y ago
  Penultimate means second-to-last; if that’s what you meant, what’s the ultimate solution to those business needs?
  - bob1029 4y ago
    You are right. Apologies. I corrected it. I am still spinning on some results. To answer the hypothetical - I do think handcrafted solutions by professionals are still the best option in a lot of places. Less weird edges and things to worry about. But, a lot of hard things are quickly reduced to routine crap once you have effectively 'done it' a few times.
opisthenar84 4y ago
Prompt engineering is a band-aid fix for the real underlying problem (one we can't solve) - all human languages are inherently vague and prone to a variety of different interpretations.
frizlab 4y ago
I think this guy found the reason why I have disliked AI from the start—and still do.
bighoki2885000 4y ago
[dead]
nilawafer 4y ago
i’m curious to know more about how the model would know what is blade runner outside of stills found online. is it able to watch and analyze YouTube videos or ““ watch movies and transcribe the text and also pull out stylistic elements from the cinematic work?
visarga 4y ago
> Unpredictable Black Boxes Are Terrible Interfaces Like, you know, managing humans?
atoav 4y ago
I am a strange guy — I like to understand things. That translates to me often using microframeworks where I can build most stuff myself. I don't like to use fully featured solutions that already do 75% of what you want but where you have to understand everything and read the whole source code and sell your first born to get the remaining 25%. So I'd rather buy wood and nails to build the hut I like than to buy some hut and hammer it into the hut I wanted. The first is a really fulfilling task, the last annoying. And so I rather drive my car to my destination than let it drive itself and have to control if it doesn't make a mistake.
noobcoder 4y ago
Collaborating with humans may be time-consuming but at least we can establish common ground and fix misunderstandings through conversation. But I feel ChatGPT is a step in the right direction, where you can question or drive the conversation. I dont think we are far from common sense and symbolic reasoning to these models. I bet AI researchers will bridge this gap soon
- clement_b 4y ago
  I was about to say. The chat paradigm solves the issue. I nearly always get what I need after rephrasing or adding more context, which is exactly how it works with humans.
thrashh 4y ago
Every time I use Siri, it reminds me of this problem. And it’s true that humans are not necessarily better. But that’s not entirely true. My closest friends know me. If I say “put up some turnstile,” they know I’m probably talking about the band because I like the band. Taking to these AIs and voice assistants is like talking to an anonymous person I’ve just met for the first time, every time, so you always need extra qualifiers.
factormeta 4y ago
google ads?
grumbel 4y ago
This seems far less than a fundamental problem with black boxes and far more a problem with image models not yet using the full power of large language models. ChatGPT almost always understands what I want. Even if it doesn't, it's easy add another prompt to put it on the right track. It's shockingly good at that. The answers it provides might not always compile, but neither would most human written code when written without access to a compiler for testing. And getting facts wrong is quite understandable as well without access to outside references. I also find that all the stuff ChatGPT fails at the most, are simply things that don't have any easy answers. It's not magic, it knows what it was feed and little beyond that. Image generation meanwhile uses a far more primitive prompting style. There are no follow questions you can ask to correct the result. And that one prompt you get is not natural language to begin with, it's just keywords used for photobashing, one that is dramatically underspecified no less. If you tell it to make a picture of New York, well, New York is big, you didn't tell where to put the camera. So of course the results will be pretty random. Worse yet, they'll change dramatically with minimum prompt changes, the moment you add a person to a photo, that person will become the primary subject and you'll have a very hard time making it just a background character by prompt. There are just a lot of image features that you can't accurately describe with a few keyword, or even if you could come up with a fitting keyword, as long as that keyword isn't in training set it's not going to help. But that's really nothing more than a current day limitation. The language understanding will get better and so will the integration ways to manipulate the image, we already have inpainting to change specific parts of the image, outpainting to change the framing, people have been playing around with projecting the images onto depthmaps or 3D geometry. In the end what matters is not that the AI is a blackbox, the inner workings are largely irrelevant, but that the output of the AI fits into my personal predictions of what it should be. And as far as ChatGPT is concerned, it's already better than any other user interface I ever touched.
yawnxyz 4y ago
Someone should create a series of prompts based in ACT and/or psych, and are essentially prompts that work on both people and LLMs

217 points by tim_sw 4y ago

As someone who struggles finding good prompts when I goof around with this stuff this article makes me feel better. There’s so much “hey look at this from AI” and I look at what I get and it’s… not good/ frustrating.

turmeric_root 4y ago
A lot of the 'look what I made with AI' images that get shared around also don't include the creator's workflow. There's usually lots of trial-and-error, manual painting/inpainting, multiple models involved etc. and explaining all that is a lot harder than just saying 'I used stable diffusion'.

rektide 4y ago

They are against Natural Science & the enlightenment of humanity. Existence is wonderfully observable & learnable. It took human kind to create this particularly infernal information void, to have manufactured such a degrading & omnipresent local system of black holes. These are woefully against all spirit & should be torn down. More positively, there's so many positive pro-human pro-Augment-Intellect things tech could be doing; we need new powerful positive symbols that establish visible legible expansion of understandability.

AussieWog93 4y ago
Are you ok man?
- turmeric_root 4y ago
  they're just microdosing it's ok
anonymouskimmer 4y ago
> there's so many positive pro-human pro-Augment-Intellect things tech could be doing But can it do it without baby steps?
StrangeATractor 4y ago
I agree. It's one thing to delegate labor to a machine, but I find myself very uncomfortable when the machine takes the role of creative expression. Art is a form of communication which transmits feeling, experience and emotion, but there's nothing a machine puts into this, there's nothing being transmitted, it's just a rendering, it's just pushing your buttons that make you go 'neato.' I fear people will start taking the beautiful and profound for granted, it'll just be content.
- rektide 4y ago
  I dont particularly mind there being some places where tech is in the lead, where it can claim the initiative. But it just feels like humanity secedes more and more ground & understanding to humanity each month, and there's so few visible notable frontiers where humanity is staking down real wins. Rather than have advances in general purpose computing, we keep getting more and more apps, more and more far-off cloud systems.
  - rektide 4y ago
    *to tech
- IanCal 4y ago
  On the other side, I'm astounded that I can ask a machine to convey something it wants to convey and get something out that makes me feel something. > but there's nothing a machine puts into this What makes you so sure?
  - grugagag 4y ago
    > ask a machine to convey something it wants to convey What makes you think a machine has any wants and desires?
    - IanCal 4y ago
      I'm a squishy machine and I have them, for as far as those terms have any meaning. What measure or process do you have of determining if I have them?

boringuser2 4y ago

Not too interesting to solve a "problem" that solves itself. Just going from GPT-3.5 to GPT-4 I'd say intent inference has improved by an order of magnitude. It almost feels mind-reader-ish. I've been waffling because I'm trying to cope with the absurdity of this product by highlighting the flaws (it can fail badly on tough problems), but we've gone from CleverBot to a genie that can instantly analyze your written intent and produce a highly logically coherent, well-worded analysis in a very short period of time.

ChatGTP 4y ago
I’d double check the code. I have a similar experience but when I actually look at what’s been returned I find strange things in there. Like it often works, but it’s rarely as correct as it seems. At least for a given use case.
- boringuser2 4y ago
  I read all of the code it writes.
- rjbwork 4y ago
  I agree. One of the engineers on my team came to me with a script (maybe 20 lines of bash) like "hey isn't it cool that I got ChatGPT to generate this?" and after studying it for a few minutes I was like "it is cool, but now spot the bug". It was near the end of the day so I asked them to follow up with me in the morning after they'd found and fixed the bug. I ended up having to point it out directly the next day. It was a spurious -A NUM argument that would, in a production scenario, cause it to recognize valid data as invalid because the full context of the data would be missing. All this to say, it is a very useful tool, but we should be highly suspicious of its output, and really make sure we understand what it's giving us. The ability to read and mentally parse and execute code will become extremely important, and give people with that ability a leg up.
  - boringuser2 4y ago
    Give me the snippet and I guarantee I can get GPT-4 to identify the bug with no hints and minimal prompting.

WFHRenaissance 4y ago

>>> When users cannot predict how input controls affect outputs they have to resort to trial-and-error, which is frustrating. This is a major issue when using generative AI for creating new content and it will remain an issue as long as the mapping between the input controls and outputs is unclear. But we can improve AI interfaces by enabling conversational interactions that can let users establish common ground/shared semantics with the AI, and that provide repair mechanisms when such shared semantics are missing. And this, ladies and gentlemen, is the reasoning for working on your prompt engineering. The prompt is the interface. Prompting is a communication skill, a management skill, and a technical skill. I doubt many will be hired as a "prompt engineers" in coming years (some already have been, by idiotic corps) - but being a bad prompt engineer will be a noticeable lack. Start practicing.

kevingadd 4y ago
Are you saying that a "good prompt" has the same properties no matter what network you're feeding it to? If so, why? It seems to me that writing a good prompt would require an understanding of the nature of the network you're using, at a minimum - for example "trending on artstation" is common in image generation prompts, and that's not going to be very useful input when generating text for a blog post. If not, what is the specific thing people should be practicing right now?
sshumaker 4y ago
This is really a lot less necessary with GPT-4. What required careful prompting in 3.5 often you can give it something slapdash in 4 and it can do a great job figuring out intent.
chewxy 4y ago
Sooner or later we'd have to be more and more precise with prompts. We'd be going from "I want a function that takes anything and returns itself" to "I : a -> a" ... and hey that looks like a programming language
- stale2002 4y ago
  It seems like the hottest new programming language is the English language.

skepticATX 4y ago

And this is why I can't get behind LLMs as a universal GUI replacement. Not in their current form. We know that users are increasingly likely to leave a page as load time increases. Now what happens when you have an unpredictable black box that sometimes doesn't do what the user expects, or flat out refuses to cooperate? This is poor UX for many tasks.

Gigachad 4y ago
I can see LLMs being a better interface for the command line. Where I can just say "give me the ffmpeg command to flip a video upside down" ChatGPT gave me "ffmpeg -i input.mp4 -vf "vflip" output.mp4" and then explained what each part of the command does. I searched on google and the results were all for random tools when I specified ffmpeg, and the ones that were correct had complex walls of text, adverts everywhere, and other bloat.
- Izkata 4y ago
  My first non-ad result on Google for "ffmpeg flip video upside-down" was [0], which includes both that and several other options including rotating 180 degrees (a different way of getting upside-down). Knowing more precisely that rotation isn't what I want, googling for "ffmpeg vertical flip video" sends me to [1] in the very first result, which is the same as what you got. Except it also specifies copying the audio without reencoding it, I'm not sure what your version would do without including that. [0] https://stackoverflow.com/questions/3937387/rotating-videos-with-ffmpeg https://stackoverflow.com/questions/3937387/rotating-videos-... [1] https://filme.imyfone.com/video-edit-tutorials/ffmpeg-flip-video/ https://filme.imyfone.com/video-edit-tutorials/ffmpeg-flip-v...
  - Gigachad 4y ago
    The first link has the entire "above the fold" content as useless bloat including ironically a huge block about SO closing the question as off topic. There is some good content in there if you dig though. The second one has a huge number of annoying popups all over the page (4 visible with adblock enabled). The ChatGPT one just leads with the answer and explains it after. It's a non destructive command so if I later decide I also want audio, I can just continue the conversation and ask for audio to be included.

lamuswawir 4y ago

I couldn't agree more, these tools are very useful, but interaction is almost just like that with humans, you use a word, they understand it as another. So you have to correct them, to explicitly tell them what you want. They are very good tools when you they and you are talking about the same thing. Indeed just like human conversation. The instant feedback is better than with humans. Like humans, these models can take one thing to mean another, but because we are still using them for tasks that require instant feedback, we can catch the errors early and correct. It becomes worse if the are to be used on longer lasting tasks. The entire project could be derailed by one misunderstood concept. Just like a human project. Natural language is a fair interface. But with lots of ambiguity. When I say 'clear', do you mean understandable, easily recognisable or clear like water. I faced that dilemma when I used that word as a design prompt because it can mean things which are opposites of each other. Some languages are so rich they avoid such ambiguities. A common language can be good. It should not be designed by a committee, it should evolve with these tools naturally.

__alias 4y ago

Amazing interfaces for particular scenarios if you look at them from another angle though. Yesterday I wanted to write a bash script to do a particular task. " I never properly learnt bash and it takes me a very long time to do something simple, furthermore, in this case, I didn't actually know the particular tools which were needed for what I wanted to write. After inputting, it correctly solves the exact problem for me which was perfect. Working backwards, I then asked it to explain what each part of the script was doing -- and learned the syntax for sed etc, along with gaining a better understanding of what what each command was doing. The point i'm trying to make here, is that it's amazing to tackle some interactions more naturally. Top down, instead of bottom up. This can help reduce the paralysis that occurs when working with the unknown. Someone very skilled with photoshop might understand the tool very well and work with particular tools to come up with a result. but Natural language can have you provide the desired end result and work down to fine details.

taberiand 4y ago
I've had the same experience (near exact same it seems, with writing bash scripts) - using it as an iterative guided tool provides a bright light to follow along an otherwise uncertain path, when you know where you want to go but not exactly how to get there, yet you've got enough experience to know when you're going off track.
onion2k 4y ago
and learned the syntax for sed etc, Did you actually learn it though, enough so you understand it and can recall it? If you did then next time you need a bash script you'll be able to quickly figure out the basics of it, or at least Google for something and modify it with your new knlwledge. I don't think it's much of a stretch to suggest that you won't do that and you'll just use GPT again instead. And next time you might not bother asking it to explain everything... That's not a criticism of you or of GPT. You have an awesome new tool that magically writes things for you. The most efficient way to use it is to let it do its magic and move on with more important things. All I'm suggesting is that most people, given such a tool, will use it and not learn from it.
- 4y ago

cwkoss 4y ago

I feel like this is valid criticism for Dall-e, but not so much for stable diffusion. Stable diffusion (and to a lesser extent midjourney) exposes a LOT more knobs to tune. Layer on controlnet and other extensions, and you can really iteratively refine an image to get closer and closer to what you want. The key is to be able to 'lock in' a depth map, rough composition, seed, etc, and then tune the other parameters to refine what you're looking for. It's definitely not a one-shot solution, but its still miles faster than learning how to do digital painting and manually rendering things. I don't feel like it matters if you have to generate 16 images to find one to start building off of when each image can be rendered in 1/1000th the time a human could.

milchek 4y ago

What i find interesting are the discussions around good vs bad prompts and being adept at “prompt engineering” We’ve had this in the engineering world a long time in the form of Google and StackOverflow searches. The great engineers i’ve worked with know most stuff they need day-to-day, but when they run into issues it’s their ability to search for and find the right answer that gives them advantage over others who don’t know where to look (or even start).

ChatGTP 4y ago
Using an LLM yesterday to code, there was no way I’d be proficient with the workflow if I wasn’t an experienced programmer. I’ve never seen code look so correct, sort of work but be so strangely crap.
- chillfox 4y ago
  I used to think that LLMs for coding would mostly be a tool to make experienced developers much more productive, but after coming across this video I think it will be used by normal people with no programming experience to make small basic programs. https://www.youtube.com/watch?v=IyKKhxYJ4U4 https://www.youtube.com/watch?v=IyKKhxYJ4U4
  - otabdeveloper4 4y ago
    Normal people with no programming experience can already make small basic programs by copy-pasting code from Github and StackOverflow.
    - lordnacho 4y ago
      This is so much harder than most people think. I would reckon that as soon as a complication shows up, 75% of non coders will get helplessly stuck. Complication meaning something slightly beyond changing variable names: you need a different if condition, the snippet doesn't contain the required import, the version has advanced slightly.
    - 4y ago

anonymouskimmer 4y ago

> Even worse, it is unclear to me how to change the prompt to move the image towards the image I want. An image program I use has sliders for things called "High", "Low ", and "Gamma". I don't know what these mean, but I can immediately see the effects they have when I use them. I can also reverse the effects back to the original image by putting them back. I don't know if AI could easily reverse back changes, at least over a couple of iterations. But I wonder if things like sliders or knobs could be added to AI interfaces to allow more gradual iteration of a text or image, or idea of a text or image, than having the AI generate an iteration based on a new plain text prompt, or even "conversational interface". You could have a ton of sliders with names that the user doesn't understand. You could link sliders together so that moving one moves them all, and then unlink individual ones as desired to see what they do. This would provide more immediate and tailorable feedback than trying to get your words just right.

dendrite9 4y ago
A while ago someone posted https://www.calligrapher.ai/ https://www.calligrapher.ai/ (discussion: https://news.ycombinator.com/item?id=34530011 https://news.ycombinator.com/item?id=34530011) The sliders were interesting and generally useful but I found it frustrating that they were not repeatable. Since then I've started to think I was approaching the sliders the wrong way by assuming they directly controlled the output, rather than being weights in one direction or another.
danybittel 4y ago
Totally agree. Something like: http://vecg.cs.ucl.ac.uk/Projects/projects_fonts/projects_fonts.html http://vecg.cs.ucl.ac.uk/Projects/projects_fonts/projects_fo... but not only for fonts.
8n4vidtmkvmk 4y ago
there are sliders for things like "cfg" and "denoising" and various other things. with some trial and error you can figure out what they do and how to tune them, the issue is that image generation still takes 10 or 30 seconds so it's not a quick process. but yes, I'd still welcome more dials. more is a bigger pain to get started but useful after you figure them out

blueblimp 4y ago

The recommendation to support conversational interactions is very good. After all, if you consult with a human expert, one of the main things they'll be doing is conversing with you to figure out your requirements. I find it a bit frustrating that even ChatGPT will give you a giant answer straightaway instead of clarifying your requirements first. (Prompting can help with this somewhat, if you explicitly tell it to ask clarifying questions.)

koromak 4y ago

I feel like bad UX is the least concerning problem we're going to have to deal with.

bighoki2885000 4y ago

[dead]

vrglvrglvrgl 4y ago

[dead]

hyperthesis 4y ago

The computer problems golden-era science fiction predicted have finally come true. Also reminds me of Monty Python's Stay Here and Make Sure He Doesn't Leave https://youtube.com/watch?v=2f5MvVx8RM8 https://youtube.com/watch?v=2f5MvVx8RM8

Izkata 4y ago

The refrigerator "true system model" has the numbers and letters swapped from the instructions in the previous image.

bob1029 4y ago

We are finding that using context hints injected dynamically into ChatGPT prompts by way of separate classifiers is potentially the ultimate solution for many business needs. Having the very first thing be an explicit natural language feature parsing phase can make the downstream generative stuff way more predictable. We could actually go back and look at the traces around a user's original prompt to determine why it might have generated nonsense downstream. The challenging part of this is inverting everything and figuring out what all of your classes are and how to arrange training data to fit them. Set theory is another new dragon here. Smaller models like Ada and Babbage are ideal for this, but we are looking at 90's tech too (SVMs). If we need 10k separate models because we have so many classes, then I'd prefer to be able to train them in a few seconds/minutes on a CPU. Fine tuning gigantic models like Davinci is a dead-end at this point IMO. A fleet of microscopic models working together to feed 1 big powerful (untunable) model feels a lot more manageable.

raldi 4y ago
Penultimate means second-to-last; if that’s what you meant, what’s the ultimate solution to those business needs?
- bob1029 4y ago
  You are right. Apologies. I corrected it. I am still spinning on some results. To answer the hypothetical - I do think handcrafted solutions by professionals are still the best option in a lot of places. Less weird edges and things to worry about. But, a lot of hard things are quickly reduced to routine crap once you have effectively 'done it' a few times.

opisthenar84 4y ago

Prompt engineering is a band-aid fix for the real underlying problem (one we can't solve) - all human languages are inherently vague and prone to a variety of different interpretations.

frizlab 4y ago

I think this guy found the reason why I have disliked AI from the start—and still do.

bighoki2885000 4y ago

[dead]

nilawafer 4y ago

i’m curious to know more about how the model would know what is blade runner outside of stills found online. is it able to watch and analyze YouTube videos or ““ watch movies and transcribe the text and also pull out stylistic elements from the cinematic work?

visarga 4y ago

> Unpredictable Black Boxes Are Terrible Interfaces Like, you know, managing humans?

atoav 4y ago

I am a strange guy — I like to understand things. That translates to me often using microframeworks where I can build most stuff myself. I don't like to use fully featured solutions that already do 75% of what you want but where you have to understand everything and read the whole source code and sell your first born to get the remaining 25%. So I'd rather buy wood and nails to build the hut I like than to buy some hut and hammer it into the hut I wanted. The first is a really fulfilling task, the last annoying. And so I rather drive my car to my destination than let it drive itself and have to control if it doesn't make a mistake.

noobcoder 4y ago

Collaborating with humans may be time-consuming but at least we can establish common ground and fix misunderstandings through conversation. But I feel ChatGPT is a step in the right direction, where you can question or drive the conversation. I dont think we are far from common sense and symbolic reasoning to these models. I bet AI researchers will bridge this gap soon

clement_b 4y ago
I was about to say. The chat paradigm solves the issue. I nearly always get what I need after rephrasing or adding more context, which is exactly how it works with humans.

thrashh 4y ago

Every time I use Siri, it reminds me of this problem. And it’s true that humans are not necessarily better. But that’s not entirely true. My closest friends know me. If I say “put up some turnstile,” they know I’m probably talking about the band because I like the band. Taking to these AIs and voice assistants is like talking to an anonymous person I’ve just met for the first time, every time, so you always need extra qualifiers.

factormeta 4y ago

google ads?

grumbel 4y ago

This seems far less than a fundamental problem with black boxes and far more a problem with image models not yet using the full power of large language models. ChatGPT almost always understands what I want. Even if it doesn't, it's easy add another prompt to put it on the right track. It's shockingly good at that. The answers it provides might not always compile, but neither would most human written code when written without access to a compiler for testing. And getting facts wrong is quite understandable as well without access to outside references. I also find that all the stuff ChatGPT fails at the most, are simply things that don't have any easy answers. It's not magic, it knows what it was feed and little beyond that. Image generation meanwhile uses a far more primitive prompting style. There are no follow questions you can ask to correct the result. And that one prompt you get is not natural language to begin with, it's just keywords used for photobashing, one that is dramatically underspecified no less. If you tell it to make a picture of New York, well, New York is big, you didn't tell where to put the camera. So of course the results will be pretty random. Worse yet, they'll change dramatically with minimum prompt changes, the moment you add a person to a photo, that person will become the primary subject and you'll have a very hard time making it just a background character by prompt. There are just a lot of image features that you can't accurately describe with a few keyword, or even if you could come up with a fitting keyword, as long as that keyword isn't in training set it's not going to help. But that's really nothing more than a current day limitation. The language understanding will get better and so will the integration ways to manipulate the image, we already have inpainting to change specific parts of the image, outpainting to change the framing, people have been playing around with projecting the images onto depthmaps or 3D geometry. In the end what matters is not that the AI is a blackbox, the inner workings are largely irrelevant, but that the output of the AI fits into my personal predictions of what it should be. And as far as ChatGPT is concerned, it's already better than any other user interface I ever touched.

yawnxyz 4y ago

Someone should create a series of prompts based in ACT and/or psych, and are essentially prompts that work on both people and LLMs