7 ms·
Show HN: ffmpeg-english "capture from /dev/video0 every 1 second to jpg files"
- throwaway888abc 2y agocan't decide what is better? 1) curl | sh 2) llm | sh
- mminer237 2y agoUsing curl is surprisingly secure if you have a secure entrusted target. An LLM could be safe the first 99 times and then randomly wipe your hard drive. It's basically the same thing as curl but just randomly picking what you download, like that one thing that picked random code from stack overflow
- lyu07282 2y ago> Using curl is surprisingly secure one thing to remember is that you can make a server respond one thing when a user does "curl <url>" and another thing when the users does "curl <url> | sh": https://lukespademan.com/blog/the-dangers-of-curlbash/ https://lukespademan.com/blog/the-dangers-of-curlbash/ another thing to know is that github.com/<org>/<proj>/[...somethings...] isn't necessarily controlled by <org>: https://vulcan.io/blog/github-comment-malware-what-you-need-to-know/ https://vulcan.io/blog/github-comment-malware-what-you-need-...
- leoh 2y agoYes but you could do something equivalent with a binary you download or some remote repository like a brew keg too. At the end of the day you need to decide whether you trust who you’re downloading from or not and ‘curl … | sh’ isn’t practically worse in any way I can think of.
- rezonant 2y agoAlso, if entropy decides you are unworthy and the download dies after reading "rm -Rf /" instead of the full line "rm -Rf /tmp/setup" then you're going to have a bad time on any Linux that doesn't have preserve-root by default. Of course such deleterious incomplete command execution could take many forms.
- amake 2y agoThis is trivially prevented by wrapping the body in a function that is executed only on the last line of the script. I don't think I've seen a "curl | sh" script in the wild that wasn't written that way.
- kaibee 2y ago> An LLM could be safe the first 99 times and then randomly wipe your hard drive. So like, has anyone ever actually done enough fuzzing to see if this or other actually bad commands ever happen in practice, or are we just going on vibes here? I suppose its possible that you give it a text description to do something bad and it does, but I'm actually curious if this is just 'llms bad' vibes.
- internetter 2y agoNot intentionally, but its given me incorrect SQL that feels one step away from something incredibly dangerous
- amiga386 2y ago3) curl https://github.com/ojaswa1942/russian-roulette.sh/raw/master/russian_roulette.sh https://github.com/ojaswa1942/russian-roulette.sh/raw/master... | sudo sh -
- hawski 2y agoinvoke-undefined-behaviour | sh We live in times where you shouldn't use C or C++, because undefined behavior can eat your face and general memory safety issues, but at the same time let's pipe LLM output to your shell. It is causing a little tingling in my heart.
- deleted 2y ago[deleted]
- baq 2y agoforgot '3) apt-get install -y package; bin-from-package'
- bonyt 2y agoIf you want this to be a little safer, instead of just those guardrails to prevent semicolons and such, you can split the command into an array of arguments, and use subprocess.Popen. It won't execute through a shell, so you don't have to worry about shell injection[1]. Though I'm sure there are unsafe ways to invoke ffmpeg anyway. [1]: https://docs.python.org/3/library/subprocess.html#security-considerations https://docs.python.org/3/library/subprocess.html#security-c...
- ajsnigrutin 2y agoI'm pretty sure you can dump a stream without transcoding directly to a file, and the stream can be sourced from an url, and the destination file can be users ssh authorized_keys
- pseudosavant 2y agoAnd the GPT-4 API would want to respond with an output to do that when that isn't what the user asked for?
- jesprenj 2y agoI am almost certain one can find a seed+temperature pair that will result in such output given a non-malicious prompt.
- pseudosavant 2y agoAnd I am almost certain I could win $10M from a lottery ticket. I'm not worried about that actually happening though because it is statically never going to happen. https://i.imgflip.com/3bhvio.jpg https://i.imgflip.com/3bhvio.jpg
- 6510 2y agoOn the contrary, it happens every time.
- LeoPanthera 2y ago…by sending your request to the ChatGPT API and then executing the result. What could possibly go wrong?
- ajsnigrutin 2y agoBut it has "security" assert(ffmpeg_command.startswith("ffmpeg")) assert(";" not in ffmpeg_command) assert("|" not in ffmpeg_command) :D Surely there's no way to avoid those checks... /s
- oefrha 2y ago> assert(";" not in ffmpeg_command) Well that just made it considerably less useful given that ; is the delimiter in ffmpeg filtergraphs. Also it doesn't defend against && || \n etc. Invoking an untrusted string with sh (through os.system()) is kind of a facepalm when you can easily shlex and posix_spawn it.
- _flux 2y agoSo what kind of scenario do you have in mind?
- deleted 2y ago[deleted]
- TZubiri 2y ago/dev/video0 weirdly out of place there, it seems like going straight from the 80s to 2020s. Maybe "capture from webcam..."
- qiqitori 2y agoWhat if you have multiple webcams?
- superkuh 2y agoThat's how it still works. I asked Claude the same example request and it gave me the below which worked perfectly with my logitech webcam. while true; do ffmpeg -f v4l2 -r 1 -i /dev/video0 -vframes 1 -f image2 output_%04d.jpg sleep 1 done
- dheera 2y agoI have multiple webcams on my system -_- but yeah if you write "from webcam" it should work just fine, though it will be guessing at /dev/video0 The cool thing is that tab completion for /dev/video0 actually works as you're typing the sentence.
- AzzyHN 2y agoTerrible idea, I love it This is good use case for a well-trained LLM, rather than the broad scope of chatGPT
- amiga386 2y agoIt's not a good use case for anything. Never ask a remote endpoint that's not owned by you and not run by you, what commands you should run on your system. Certainly don't execute the answers.
- rezonant 2y agoIf we can't let ChatGPT take the wheel, how will we feel alive?
- dheera 2y agoI do envision training a local LLM which would mostly resolve this concern, but at the moment the vast majority of people don't have a good enough GPU in their system to run an even mildly-competent code generation LLM, but I imagine this will change within a few years.
- lucianbr 2y ago99.9999% of the code running on my machine is written by others and not even readable to me. I'm pretty optimistic that a similar percentage is true on your machines. So yeah, we run remote commands all the time, all of us. There may be a subtle difference between "curl something | bash" and "apt-get install" or "setup.exe", but there is no fundamental one.
- broken-kebab 2y agoThere's a difference between getting code from a repo, and from AI generator though. We can apply an ancient thing known as "reputation" to the former. Not yet to the latter.
- amiga386 2y agoFundamentally: 1. the packages being worked on by Debian et al have a huge pile of infrastructure so that their development happens collaboratively and in the open, with many eyes watching 2. everyone gets the same packages 3. they have their own security teams to _ensure_ everyone is getting the same packages, i.e. that their download servers and checksums haven't been compromised 4. the project has been been working since 1993 to ensure their update system, and the system delivered by those updates, works as expected. If it doesn't, there are IRC channels, mailing lists, bug trackers and a pile of humans to discuss issues with, and if they agree it's a bug, they can fix it for everyone It's not to say it's impossible to sneak an attack past a project dedicated to stopping such attacks, but it's so much more work compared to attacking someone who executes whatever a remote endpoint tells them
- orliesaurus 2y agoIf you used any "new" terminal - like Warp [1] (which requires you to login to use wtf) or Wave Terminal[2] (open source and bring your own AI is supported) - you'll be very familiar with this style of AI-driven completion. I do use it sometimes, but I am very very careful in reviewing what the command does before blindly copy pasting it into the cli [1] https://www.warp.dev https://www.warp.dev [2] https://www.waveterm.dev/ https://www.waveterm.dev/
- Waterluvian 2y agoWhat I want is for Chat AI to be a fallback. I want to say “frogblast the vent core” and the interface either parses it locally like they always have for years, or says “I don’t understand. Should I ask GPT?” I also want it to be more about “help me get the code and show me what it means.” I love Regexr that shows you what each part of the regex does. I’d love for it to annotate what each part of the ffmpeg command does.
- fragmede 2y agolike https://explainshell.com/explain?cmd=+ffmpeg+-f+v4l2+-r+1+-i+%2Fdev%2Fvideo0+-vframes+1+-f+image2+output_%2504d.jpg https://explainshell.com/explain?cmd=+ffmpeg+-f+v4l2+-r+1+-i... ?
- GaggiX 2y agoUsing GPT-4-turbo or GPT-4o would probably be better than using the old GPT-4.
- lemper 2y ago[flagged]
- ecjhdnc2025 2y agoHow have we gone from “the internet routes around damage” to “just connect it to a hallucinating, equivocating galaxy brain controlled by people who swear their employees to silence”?
- smabie 2y agothe real damage is remembering ffmpeg command line options. So I think the internet is working as intended
- eternauta3k 2y agoI agree the ffmpeg command line requires a postdoc, but that's why we have (deterministic, predictable) GUIs.
- seba_dos1 2y agoI really don't understand what makes people complain about ffmpeg options so much. The only problem I ever have with it is that I use it so rarely that I usually need a refresher every time I use it for anything that isn't trivial, but that would still be true with any parameter style.
- fragmede 2y agoWell, they ran out of unicorns and rainbows at the LLM factory, so we're stuck with that one.
- deleted 2y ago[deleted]
- mianos 2y agoAbsolutely so insane it's kinda funny. I think that's the point.
- hallway_monitor 2y agoI needed to speed up a video by dropping 9 out of every 10 frames yesterday. It took all of 30 seconds to type my request into GPT and paste the result, and I could inspect it before executing. I don't see how this is helpful.
- terribleperson 2y agoI'm not quite ready to execute arbitrary output from an LLM. Maybe with more guardrails and if it could guarantee it would only operate inside of a chosen folder, and would back up the folder ahead of time.
- dheera 2y agoOne relatively easy way to be safe is to do this inside a docker container with only whatever files you're working with mounted inside. I created a new script (https://github.com/dheera/scripts/blob/master/helpme https://github.com/dheera/scripts/blob/master/helpme) that is more general, and is safer by presenting the command and requiring you to type "y" to execute, and does NOT auto-execute after a delay. That said, I do believe we are re-living the autonomous car question of "what about the 0.000001%" again and in this case the absolute worst that happens is it wipes your system, and that's a disaster that's extremely easy to prepare for. You could do all your work in a VM and take daily snapshots, among other solutions. As long as the computer isn't wired up to some weapon, I say deploy now, let's not wait a decade. This world is too awesome to pass up just because of some "rm -rf" level risks. If that happens I'll just kick myself for not buying a lottery ticket because the probability of ChatGPT responding to an ffmpeg question with "rm -rf" is far, far lower than winning the lottery.
- terribleperson 2y agoWhile I am concerned about the rm -rf possibility and that's what my initial comment was about, it's not the only concern. I'm also concerned ChatGPT will return a ffmpeg command that is functional but suboptimal, creating a product that's subtly wrong. For example, a slideshow that's subtly misordered, a video file that's 10x the size it needs to be, compromised audio quality, or a video that runs fine on my PC but has poor portability (video players can be surprisingly finicky). When I look up ffmpeg commands on stackexchange, there's always feedback on any suggested command that explains what's wrong with it and what a better solution is. Often the first solution will work, but maybe only with certain ffmpeg distributions or there are major caveats to the result. I do appreciate the container solution, since it's generalizable to other ai-powered tools in this class.
- cynicalsecurity 2y agoThis is a very bad. Like really bad.
- KingMachiavelli 2y agoYou should at least wrap the ffmpeg calls in a systemd-run command with restricted internet access, ro-filesystem except /tmp, etc. It super easy to prevent AI from becoming skynet or even just stopping it from running rm -rf / but you have to understand proper system security; use namespaces and VMs, please.
- rezonant 2y ago> It super easy to prevent AI from becoming skynet If only. It will only take one hapless script to bring about judgement day.
- ecjhdnc2025 2y agoIt will be called “In Conclusion Day”.
- fmbb 2y ago> temperature=0.5 Why not 0? I’m not a prompt engineer so I never worked with the API, but I thought the “temperature” was a little knob they added for variety in responses. Is that what you want in a CLI?
- deleted 2y ago[deleted]
- dheera 2y agoGood question! I used 0.5 out of habit, but I do need to do some more experimenting with this parameter. But yes, intuitively it should probably be low. I'll do some experiments in the morning and see if it works well at 0.
- cwillu 2y agoNon-zero temperature does have a load-bearing function, allowing escape from states where the next token is always the same. You can kinda think of it as a dither.
- dheera 2y agoAuthor here! I just made a more generalized version for ALL commands: https://github.com/dheera/scripts/blob/master/helpme https://github.com/dheera/scripts/blob/master/helpme I've made it safer in that it doesn't auto-execute the command and defaults to "no". You inspect the command and type "y" to execute.
- maCDzP 2y agoIt would be cool to write some tests to see how often it works out. I have noticed that LLM:s often creates command line options that doesn’t exist. Security aside, I bet it would work more often than when I input something in the terminal. I nice feature would be to just loop back the error and get ChatGPT to correct the error. You do this by running the command with bash -n (syntax check) and when it doesn’t return an error it runs the script. Three months from now the next cloud outage at Google will be from “helpme delete that one weird file”. May the lord have mercy on our machines.
- dheera 2y ago> I nice feature would be to just loop back the error and get ChatGPT to correct the error. For code generation this works well, though for command line some additional function calling infrastructure may necessary, e.g. if it gets a file path wrong, its only way to correct it might be to execute a bunch of 'ls' commands. It might need read access to the system, which is okay for some use cases where you can containerize everything and keep private files out, but but opens another can of worms :-/
- Ulti 2y agoCame here to suggest this should be generic, but I'd also do something like pack in `man <command>` into the prompt if you are one shotting. Then it works for "all" commands that have a man page rather than just the commands GPT knows about before its cut off. Even just trying to scrape out `<command> --help` or something would be good too.
- easyThrowaway 2y agoStuff like this makes me wonder if we could apply what we've learned from llm/ml and apply it to a less leak-y abstraction like a node or graph based interface. Fewer chances for hallucinations and a more manageable, finite dataset.
- scopeh 2y ago[flagged]
- liampulles 2y agoVery cool! It does make me think about enterprise usages of LLMs though - the rule I have in my mind is "given a user prompt, only perform a query OR suggest commands, do not execute commands" (in a CQRS sense of the words query and command). Without some kind of principle like that, I really cannot do much with LLMs on the user interface side because I'll be worried it might f something up every so often.
- sambazi 2y agogeneric llm will do that for you, no?
- H8crilA 2y agoI recommend `??` from GitHub Copilot. It's basically this, but for any command, not just ffmpeg. I use it all the time. And it asks for confirmation to execute the command :) https://githubnext.com/projects/copilot-cli/ https://githubnext.com/projects/copilot-cli/
- wccrawford 2y agoSeems like it now defaults to `ghcs` and `ghce` instead of `??`, `git?` and `explain`. It took me a while to figure that out.
- superpope99 2y agoindeed - and because it's a special character you need to do something like this to replicate the ?? shortcut. alias \?\?="gh copilot suggest"
- sublinear 2y agoHow long until someone finds a way to maliciously SEO-ify these tools and cause remote code execution incidents? Is it less malicious if the script only does marketing things instead of more serious harm? What safeguards are in place to sanitize the output of copilot? I ask this because of course a more experienced user might do that sanitization or sandbox testing themselves, but they probably wouldn't get much use out of copilot in the first place.
- throwup238 2y agoThere's also aichat's shell integration [1]. Instead of typing a command I describe what I want in plain English in a terminal and then press Alt-E. It replaces the text with a command. [1] https://github.com/sigoden/aichat/tree/main/scripts/shell-integration https://github.com/sigoden/aichat/tree/main/scripts/shell-in...
- imjonse 2y agoThis is what llm cmd does for any command, and it also offers a chance to edit it before running https://github.com/simonw/llm-cmd https://github.com/simonw/llm-cmd
- epiccoleman 2y agoI've been using shell-gpt[1] for the same, and it is almost irritatingly useful. I fear that my somewhat decent shell-fu is going to atrophy pretty rapidly in this new world. in my experience, this is the kind of thing that LLMs are great for - small, one-off tasks with clearly defined parameters. (and, with careful application - low stakes.) [1]: https://github.com/TheR1D/shell_gpt https://github.com/TheR1D/shell_gpt
- nop_slide 2y agoAre single prompt python wrappers Show HN noteworthy now?
- ecjhdnc2025 2y agoNot only that, they might secure a seed round.
- fzeroracer 2y agoI thought this was meant to be a joke or something, but judging from OPs comments here it doesn't look like it. And the people cheering this on... It's a bleak future for anyone with even a passing interest in software security.
- smarm52 2y agoOr a very bright one in those looking to exploit bad software.
- aunwick 2y agoI suppose LLM injection vulnerability will be the next security product trend
- djyde 2y agoIf you are concerned about security risks, just add a confirm CLI prompt before running, to ask the user to confirm whether to execute the code (Y/N)
- smarm52 2y ago> "Makes ffmpeg easier to use by accepting plain English." Interesting, any testing? Applications? Efficacy demonstrations?