4 ms·

IMO it's completely the other way around. Shell scripts can be audited. The average user may not do it due to laziness and/or ignorance, but it is perfectly do

by cuu508 9mo ago

IMO it's completely the other way around. Shell scripts can be audited. The average user may not do it due to laziness and/or ignorance, but it is perfectly doable. On the other hand, how do you make sure your LLM, a non-deterministic black box, will not misinterpret the instructions in some freak accident?

nobodywillobsrv 9mo ago
How about both worlds? Instead of asking the agent to execute it for you, you ask the agent to write an install.sh based on the install.md? Then you can both audit whatever you want before running or not.
- catlifeonmars 9mo ago
  This is much better. Plus you get reproducibility and can leverage the AI for more repeat performances without expending more tokens.
- chme 9mo ago
  So... What you are saying is that we don't need 'install.md'. Because a developer can just use a LLM to generate a 'install.sh', validate that, and put it into the repo? Good idea. That seems sensible. Bonus: LLM is only used once, not every time anyone wants to install some software. With some risks of having to regenerate, because the output was nonsensical.
  - michaelmior 9mo ago
    > What you are saying is that we don't need 'install.md' I think the point was that install.md is a good way to generate an install.sh. > validate that, and put it into the repo The problem being discussed is that the user of the script needs to validate it. It's great if it's validated by the author, but that's already the situation we're in.
    - chme 9mo ago
      > The problem being discussed is that the user of the script needs to validate it. It's great if it's validated by the author, but that's already the situation we're in. The user is free to use a LLM to 'validate' the `install.sh` file. Just asking it if the script does anything 'bad'. That should be similarly successful as the LLM generating the script based on a description. Maybe even more successful.
    - falloutx 9mo ago
      I still dont understand why we need any of them. If I am installing something, It would take me more time to write this install.md or install.sh than if I just went to the correct website and copied the command, see the contents, run it and opening help.
- franga2000 9mo ago
  And since LLM tokens are expensive and generation is slow, how about we cache that generated code on the server side, so people can just download the pre-generated install.sh? And since not everyone can be bothered to audit LLM code, the publisher can audit and correct it before publishing, so we're effectively caching and deduplicating the auditing work too.
- vrighter 9mo ago
  then how about you cut out the llm middleman and just audit the bash scripts already provided?

by cuu508 9mo ago

nobodywillobsrv 9mo ago

How about both worlds? Instead of asking the agent to execute it for you, you ask the agent to write an install.sh based on the install.md? Then you can both audit whatever you want before running or not.

catlifeonmars 9mo ago
This is much better. Plus you get reproducibility and can leverage the AI for more repeat performances without expending more tokens.
chme 9mo ago
So... What you are saying is that we don't need 'install.md'. Because a developer can just use a LLM to generate a 'install.sh', validate that, and put it into the repo? Good idea. That seems sensible. Bonus: LLM is only used once, not every time anyone wants to install some software. With some risks of having to regenerate, because the output was nonsensical.
- michaelmior 9mo ago
  > What you are saying is that we don't need 'install.md' I think the point was that install.md is a good way to generate an install.sh. > validate that, and put it into the repo The problem being discussed is that the user of the script needs to validate it. It's great if it's validated by the author, but that's already the situation we're in.
  - chme 9mo ago
    > The problem being discussed is that the user of the script needs to validate it. It's great if it's validated by the author, but that's already the situation we're in. The user is free to use a LLM to 'validate' the `install.sh` file. Just asking it if the script does anything 'bad'. That should be similarly successful as the LLM generating the script based on a description. Maybe even more successful.
  - falloutx 9mo ago
    I still dont understand why we need any of them. If I am installing something, It would take me more time to write this install.md or install.sh than if I just went to the correct website and copied the command, see the contents, run it and opening help.
franga2000 9mo ago
And since LLM tokens are expensive and generation is slow, how about we cache that generated code on the server side, so people can just download the pre-generated install.sh? And since not everyone can be bothered to audit LLM code, the publisher can audit and correct it before publishing, so we're effectively caching and deduplicating the auditing work too.
vrighter 9mo ago
then how about you cut out the llm middleman and just audit the bash scripts already provided?