6 ms·

I recently spent over an hour trying to get ChatGPT to give me some pretty simple rsync commands. It kept giving me command line parameters that didn't work on

• BryanLegend 1y ago
  I've recently been misled by ChatGPT a lot as well. I think it's the router. I'm on the free plan so I assume they're just being tight with the GPU cycles.
  • wintermutestwin 1y ago
    I am on a $20 plan and using the "thinking" version of 5.
  • iamnotagenius 1y ago
    [dead]
• Yoric 1y ago
  Pretty much my experience, yes.
• chrisweekly 1y ago
  Why involve an LLM at all, if you're looking up docs for a particular tool like rsync?
  • harvey9 1y ago
    Not the op, but I sometimes find the official documents hard to parse. Not looking at rsync in this case. On the other hand I have the same experience with LLMs as the op. Big thanks to all the doc writers who include vignettes right in the documents.
  • wintermutestwin 1y ago
    Lots of reasons! First off: where else do I go to learn this stuff? Man pages are reference for people who work in CLI all the time and not for virgin learners as they are are necessarily packed with the complete lexicon but with barely a thought to explaining real world examples of common tasks. There are a million Linux websites with the same versioning issues and inadequate explanations. I guess I could buy an oreily book and learn the topic end to end even though I will only need to know the syntax of a couple commands. With an LLM, I can get it to tell me what each parameter it suggests actually does and then I can ask it questions about that to further my understanding. That is a massive leg up over the knowledge spaghetti approach...
    • heckelson 1y ago
      I like the tldr pages to learn the most common features and use cases of new command line tools! I think it's great, albeit, a bit slow sometimes
      • wintermutestwin 1y ago
        tldr pages are a great idea, but the execution is a total fail. Looking at the rsync entry, it fails to provide the most blatantly common requirements: I just needed two commands: one to mirror a folder from one drive to another, updating only the changes (excluding all of the hidden MacOS cruft). And another command to do a deep validation of the copy. These have to be two of the most commonly used commands right??? In the end, I felt that messing around with precious data without being 100% certain of what I am doing just wasn't worth it so I got a GUI app that was intuitive.
    • latexr 1y ago
• latexr 1y ago
  > I am not a coder (much), but I have to wonder if my experience is common in the coding world? It is, yes. Surely someone will come and tell you it doesn’t happen to them, but all that tells you is that it ostensibly isn’t universal, but still common enough you’ll find no end of complaints. > Maybe there are ways to avoid this (and others) pitfall with prompting? Prompting can’t help you with things not in the training set. For many languages, all LLMs absolutely suck. Even for simple CLI tools, telling an LLM you are on macOS or using the BSD version may not be enough to get them to stop giving you the GNU flags. Furthermore, the rsync change in macOS is fairly recent so there’s even fewer data online on it. https://derflounder.wordpress.com/2025/04/06/rsync-replaced-with-openrsync-on-macos-sequoia/ https://derflounder.wordpress.com/2025/04/06/rsync-replaced-... > As it is, I do not see at all how LLMs could really save time on programming tasks without also costing more time dealing with its quirks. And that’s the best case scenario. It also happens that people blindly commit LLM code and introduce bugs and security flaw they cannot understand or fix. https://secondthoughts.ai/p/ai-coding-slowdown https://secondthoughts.ai/p/ai-coding-slowdown https://arxiv.org/abs/2211.03622 https://arxiv.org/abs/2211.03622
  • Helmut10001 1y ago
    Usually, in these edge cases, I go to the documentation page and dump all pages as Markdown into the AI tool (most often Gemini, due to token count). This context engeneering has helped a lot to get better answers. However, it also means I am consuming sometimes 1 Million tokens on relatively simple problems. Like recently, when I needed to solve a relativey simple but specific MermaidJS issue.
• goalieca 1y ago
  > I tell it to validate each parameter against my version moving forward and it clearly doesn't do that. I would like an AI expert to weigh in on this point. I run into this a lot. It seems that LLMs, being language models and all, don't actually understand what i'm asking. Whenever i dive into the math, superficially, it kind of makes sense why they don't. But it also seems like transformers or some secret sauce is code up specially for tasks like counting letters in a word so that AI doesn't seem embarrassing. Am I missing something?
  • bashy 1y ago
    Words are tokens so it can't really 'see' the word(s), it just knows how to link them.
  • nerdponx 1y ago
    LLMs are next-token prediction models. They "understand" in that, if the previous 1000 tokens are such-and-such, then they emit a best guess at the 1001th token. It "knows" what rsync is because it has a lot of material about rsync in the training data. However it has no idea about that particular version because it doesn't have much training data where the actual version is stated, and differences are elaborated. What would probably produce a much better result if you included the man page for the specific version you have on your system. Then you're not relying on the model having "memorized" the relationship relationships among the specific tokens you are trying to get the model to focus on, instead just passing it all in as part of the input sequence to be completed. It is absolutely astounding that LLMs work at all, but they're not magic, and some understanding of how they actually work can be helpful when it comes to using them effectively.
    • pglevy 1y ago
      Our low-code expression language is not well-represented in the pre-training data. So as a baseline we get lots of syntax errors and really bad-looking UIs. But we're getting much better results by setting up our design system documentation as an MCP server. Our docs include curated guidance and code samples, so when the LLM uses the server, it's able to more competently search for things and call the relevant tools. With this small but high-quality dataset, it also looks better than some of our experiments with fine tuning. I imagine this could work for other docs use cases that are more dynamic (ie, we're actively updating the docs so having the LLM call APIs for what it needs seems more appropriate than a static RAG setup).
• athrowaway3z 1y ago
  > Maybe there are ways to avoid this (and others) pitfall with prompting? Not sure about Codex, but in Claude Code you can run commands. So instead of letting it freestyle / guess, do a: `! man rsync` or `! rsync --help` This puts the output into context.
• erichocean 1y ago
  Ask Gemini Pro 2.5 to build the rsync command and then give it the man page for your version of rsync. It should succeed the first time. Here's a command to copy the man page to the clipboard than you can immediately paste into aistudio (on a Mac): man rsync | col -b | pbcopy As a general rule, if you would need to look something up to complete a task, the AI needs the same information you do—but it's your job to provide it.
  • wintermutestwin 1y ago
    So I paste the man page into llm and tell it to only give me parameters that are in that page? Even if it obeyed, it would still choke on how to exclude hidden MacOS kruft files from the copy...
    • erichocean 1y ago
      No, you don't need to "tell it to only give me parameters that are in that page." Here's the entire prompt: I need the rsync command to copy local files from `/foo/bar` to `~/baz/qux` on my `user@example.com` server. Exclude macOS cruft like `.DS_Store`, etc. Here's the man page: <paste man page for your rsync that you copied earlier, see above>
    • nehal3m 1y ago
      If you have trouble talking to an AI, how do you ever expect to merge with Neuromancer’s twin?
• meowface 1y ago
  As a programmer I have noticed this problem much more with command help than with code. Maybe partly because the training data has way more, and more diverse, code examples than all the relevant permutations and use cases for command argument examples.
• righthand 1y ago
  What is the cost (in tokens/$$$) of spending an hour restating questions to a chat bot vs typing `man rsync`?
  • pletnes 1y ago
    Lower than the cost of a meatbag-office with heating, cooling and coffee for the time spent reading the manual.
    • righthand 1y ago
      How is the cost of an office relative? Llms didn’t destroy offices. You can type `man rsync` without any of that.
  • danielbln 1y ago
    Telling the agent to execute "man rsync" and synthesize the answer from there is probably the cheapest and most efficient option. Letting some detached LLM fumble around for an hour is never the right way to go, and inversely sifting through the man page of rsync or fmmpeg or (God forbid) jq to figure out some arcane syntax isn't exactly a great use of anyone's time either, all things considered.
    • righthand 1y ago
      Sifting through just means you don’t know how to use the man interface to search/grep (which from a discoverability perspective is fair). However I think reeling through an Llm (using an agent or not) for a task that probably could take <10mins at $0, demonstrates enthusiasts disregard for a good set of research and reading habits. All of this is an attempt at circumventing RTFM because you’re privileged enough to afford it. Just lay yourself down on the WALL-E floating bed and give up already.
      • wintermutestwin 1y ago
        The “fucking” manual is obtuse, overly verbose, and almost always lacking in super clear real world examples. That you claim it is a <10 min problem demonstrates experts disregard for the degree of arcane crap, a beginner needs to synthesize. I am backing up and verifying critical data here. This is not a task that should be taken lightly. And as I learned, it is not a task that one can rely on an LLM for.
• alain94040 1y ago
  Instead of just saying: rsync on my system is version 3.2, have you tried copy/pasting rsync --help? In my experience, that would be enough for the AI to figure out what it needs to do and which arguments to use. I don't treat AI like an oracle, I treat it like an eager CS grad. I must give it the right information for the task.
• BinaryIgor 1y ago
  All the time I find that if I know the stack and tools I am working in, it's faster to just write code on my own, manually; If I want to learn on the other hand, LLMs are quite useful - as long as you understand (or learn to) and validate the output
• dimgl 1y ago
  LLMs are a very specific kind of beast. Using an `rsync --help` or getting any kind of specific documentation into context would have unblocked you.
• dangerface 1y ago
  Yup even when you tell it your version it forgets pretty quickly. Or agrees it messed up and assures you this time it will give you the correct info for your version number then gives you the same command. Javascript is a nightmare as they change everything constantly. PHP has backwards compatibility for everything so its not really an issue. It also gives out dated info on salesforce, and im not just talking about the latest and greatest, it recommends stuff that was deprecated years ago.
• doug_durham 1y ago
  I'm a coder and I've never had your experience. It usually does an amazing job. I think that coders have an advantage because there are many questions I would never ask an LLM because of my intuition on what would work well and what wouldn't. In your case I would have dumped the output of `rsync --help` into the context window once I saw it wasn't familiar with my particular version of rsync. That's they way these tools work.
• deleted 1y ago
  [deleted]
• KronisLV 1y ago
  > I recently spent over an hour trying to get ChatGPT to give me some pretty simple rsync commands. Try N times, adding more context about the environment and error messages along the way. If it doesn't work after those, try other models (Claude, Gemini, ...). If none of those work on whatever number of attempts you've chosen, then LLMs won't be able to help you well enough and you should save yourself some time and look elsewhere. A good starting point is trying for 10-20 minutes, after which point an LLM might actually become slower than you going the old fashioned way of digging into docs and reading forum posts and such. There are also problems that are a bit too complex for LLMs as well and they'd just take you in circles no matter for how long you try.