3 ms·

I find LLM's too fall into the trap of writing a bash script for a task that clearly needs to be implemented in an Actual Language with Real Data Structures. Fo

by genidoi 2mo ago

I find LLM's too fall into the trap of writing a bash script for a task that clearly needs to be implemented in an Actual Language with Real Data Structures. For example, ask an LLM to bring up a SQL server with some schema + data preload step, and it will write a profoundly long bash script to do that task, every time.

IshKebab 2mo ago
I think they've learnt from decades of humans using Bash for tasks that clearly need to be implemented in an Actual Language. The knowledge that Bash is awful and should be avoided as much as possible is surprisingly and disappointingly rare.
csydas 2mo ago
my experience as well. claude produces functional but over-engineered scripts fairly frequently with often very questionably useful safety checks, especially for powershell. a common tell of ai generated powershell is a script that has dedicated functions to check types, often via several methods, and happily prints the output of the checks to shell. i do not get why it does this but it often adds dozens of lines that really serve no purpose but to make the shell output look fancy
- genidoi 2mo ago
  It's frustrating when I catch it doing this because it's wasting tokens and time on what it reports as "one-off" scripts. Might add a requirement to never use shell scripting unless the task is truly a one liner.
- garethrowlands 2mo ago
  $ Hey, claude, I've figured out the powershell one-liner we need and I've pasted it below. Please insert it into the application. ... I have analysed your idiomatic one-liner and propose adding the following 30 lines into the application Yes/Yes and always accept such idiocy/No ESCAPE ESCAPE /clear I want to update my powershell skill so that it writes idiomatic pipelines and does not introduce idioms from other languages. [PASTES overeagerness wording from Claude prompting best practices]
Grimburger 2mo ago
Try being on linux and using zsh, it constantly fucks up scripts. Having it run commands via SSH into a windows machine is even more a comedy of errors. I have to put in every claude.md that the shell is zsh. It's reached the point of annoyance I might just go back to bash.
- delta_p_delta_x 2mo ago
  I find that Claude does this much more than other models. GPT, for instance, knows I'm on Windows running PowerShell, and writes very idiomatic PowerShell. But maybe that's because I'm the sucker, and since PowerShell is more verbose it costs me more tokens than the terseness of Unix shells. Oh well.
  - garethrowlands 2mo ago
    Some models are better than others. But it's all fixable with a relatively small skill. Try PowerShell Windows Skill perhaps.
- kergonath 2mo ago
  > It's reached the point of annoyance I might just go back to bash. You can use zsh as the default shell and still write your scripts for bash. Actually, it’s an advice I saw more than a couple of times. Otherwise you need to translate the bash-isms when you get bits of code from random places on the Internet. Just put the right shebang (or ask Claude to do it). What’s the problem?
- srcoder 2mo ago
  Just run your llm in a container with the tools it needs. It makes your pc more secure by removing the risk of doing something destructive to your computer and the llm can do llm. Also, llms ssh'ing to the outside world? Asking for trouble later on

by genidoi 2mo ago

IshKebab 2mo ago

I think they've learnt from decades of humans using Bash for tasks that clearly need to be implemented in an Actual Language. The knowledge that Bash is awful and should be avoided as much as possible is surprisingly and disappointingly rare.

csydas 2mo ago

my experience as well. claude produces functional but over-engineered scripts fairly frequently with often very questionably useful safety checks, especially for powershell. a common tell of ai generated powershell is a script that has dedicated functions to check types, often via several methods, and happily prints the output of the checks to shell. i do not get why it does this but it often adds dozens of lines that really serve no purpose but to make the shell output look fancy

genidoi 2mo ago
It's frustrating when I catch it doing this because it's wasting tokens and time on what it reports as "one-off" scripts. Might add a requirement to never use shell scripting unless the task is truly a one liner.
garethrowlands 2mo ago
$ Hey, claude, I've figured out the powershell one-liner we need and I've pasted it below. Please insert it into the application. ... I have analysed your idiomatic one-liner and propose adding the following 30 lines into the application Yes/Yes and always accept such idiocy/No ESCAPE ESCAPE /clear I want to update my powershell skill so that it writes idiomatic pipelines and does not introduce idioms from other languages. [PASTES overeagerness wording from Claude prompting best practices]

Grimburger 2mo ago

Try being on linux and using zsh, it constantly fucks up scripts. Having it run commands via SSH into a windows machine is even more a comedy of errors. I have to put in every claude.md that the shell is zsh. It's reached the point of annoyance I might just go back to bash.

delta_p_delta_x 2mo ago
I find that Claude does this much more than other models. GPT, for instance, knows I'm on Windows running PowerShell, and writes very idiomatic PowerShell. But maybe that's because I'm the sucker, and since PowerShell is more verbose it costs me more tokens than the terseness of Unix shells. Oh well.
- garethrowlands 2mo ago
  Some models are better than others. But it's all fixable with a relatively small skill. Try PowerShell Windows Skill perhaps.
kergonath 2mo ago
> It's reached the point of annoyance I might just go back to bash. You can use zsh as the default shell and still write your scripts for bash. Actually, it’s an advice I saw more than a couple of times. Otherwise you need to translate the bash-isms when you get bits of code from random places on the Internet. Just put the right shebang (or ask Claude to do it). What’s the problem?
srcoder 2mo ago
Just run your llm in a container with the tools it needs. It makes your pc more secure by removing the risk of doing something destructive to your computer and the llm can do llm. Also, llms ssh'ing to the outside world? Asking for trouble later on