6 ms·

This is just wrong. Also, even if a LLM could do that, so could a shell script, without the risks involved in using "AI" for it, or for now the ridiculous exte

This is just wrong. Also, even if a LLM could do that, so could a shell script, without the risks involved in using "AI" for it, or for now the ridiculous external dependence that would involve. I wonder if in 10 years people will be stuck debugging Rube-Goldberg machines composed of LLM api calls doing stuff that if-statements can do, probably cobbled together with actual if-statements

JohnFen 4y ago
> doing stuff that if-statements can do, probably cobbled together with actual if-statements In other words, old-school expert systems.
- baq 4y ago
  With the limit of 25k words it might actually be reasonable to test out a prompt for an expert system… but I’d still leave reasoning to something else, for now. Z3, prolog or some forward chaining tool like clips, but have the LLM hallucinate some of the rules?
nico 4y ago
Sure, maybe you can use a shell script, but now the AI assistant can write it based on your verbal/text description, and then the assistant can also run it for you after you’ve checked it. What your are saying is: “why use the washing machine, if I my clothes are even cleaner when I wash them myself - I also spend less detergent and less water”. You are free to keep doing your laundry by hand. But I bet most people prefer the washing machine.
- xwdv 4y ago
  Spare me the shitty analogies. We write shell scripts because it’s cheap, fast, and the behavior is very predictable. Like it or not, an AI’s behavior is a black box and can’t be “proven” to execute exactly the same every time for the scenarios you are targeting. A shell script will do exactly what it has been written to do every time, unless tampered with. And if changes need to be made, it can be done quickly without need for retraining, god knows how long that would take for an AI to learn something new. God help you if you need to maintain “versions” of your AI, trained for different things. Face it, AI are pointless and slow for certain classes of problems.
  - com2kid 4y ago
    > A shell script will do exactly what it has been written to do every time, unless tampered with. Or unless some magic environment variable changes, or one of the runtime dependencies changes, or it is run on a different operating system, or permissions aren't setup right, or one of its tasks errors out. Shell scripts are digital duct tape, the vast majority of shell scripts do not come close to being reliable software. > god knows how long that would take for an AI to learn something new Did you watch OpenAI's demo yesterday? They pasted in new versions of API docs and GPT4 updated its output code. When GPT forgot a parameter, the presenter fed back the error message and GPT added the parameter to the request.
    - xwdv 4y ago
      AI proponents are missing the point. Anything you write to make an AI produce something is basically code. Docs are code. You don’t have to feed a developer code or docs, you can give them a high level idea and they’ll figure it out on their own if you want.
      - echelon 4y ago
        That code will eventually fall away. The big thing everyone in this single thread is missing is that AI is a metaheuristic. I wouldn't expect to use AI to run_script.py. That's easy. I'd expect it to look at the business signals and do the work of an intern. To look at metrics and adjust some parameters or notify some people. To quickly come up with and prototype novel ways to glue new things together. To solve brand new problems.
        xwdv 4y ago
        To do the work of an intern an AI must go on Jira, read a ticket, then find the appropriate repositories where code needs to be modified, write tests for its modification, submit for code review, respond to feedback in code review, deploy its changes. It’s not there yet.
        4y ago
- whatshisface 4y ago
  I think you're fighting an uphill battle because of what you picked to defend here - shell scripts are very easy to write, and I have a hard time imagining a future where someone tells an LLM, "Write me a shell script that runs run_control.py with the speed argument set to one hundred." to get, "./run_control.py --speed 100"
  - pstorm 4y ago
    That's a trivial example, sure. Think of saying "make this image more red, flip it vertically, then crop the bottom 25%, and finally output it with 80% compression as a jpg". That would take 15 minutes to figure out how to write that with imagick, or just have an LLM do it for you.
    - tough 4y ago
      I had to try, seems reasonable, haven't tested the code As an AI language model, I do not have direct access to image editing software, but I can provide you with a shell script that could achieve the described image manipulation using ImageMagick, assuming it is installed on your system. Here's an example shell script that you could use as a starting point: bash Copy code #!/bin/bash # Set the input image filename input_file="input.jpg" # Set the output image filename output_file="output.jpg" # Increase the redness of the image convert "$input_file" -modulate 100,150,100 -channel R -evaluate multiply 1.2 +channel temp1.jpg # Flip the image vertically convert temp1.jpg -flip temp2.jpg # Crop the bottom 25% of the image convert temp2.jpg -gravity South -chop 0x25% temp3.jpg # Compress the image to 80% convert temp3.jpg -quality 80 "$output_file" # Clean up temporary files rm temp*.jpg To use this script, save it to a file (e.g., "image_manipulation.sh"), make it executable (e.g., "chmod +x image_manipulation.sh"), and run it in the same directory as your input image. The output image will be saved in the same directory with the name "output.jpg". Please note that this is just an example script and may need to be modified depending on your specific image manipulation needs.
      - fulafel 4y ago
        This has lots of lossy encode/decode cycles that will accumulate artifacts. Good anecdote.
querez 4y ago
It's not wrong. It's how modern systems operate. E.g. look at Google's SayCan (https://say-can.github.io/ https://say-can.github.io/) which operates exactly like this (an LLM ordering a Robot around).
jrussino 4y ago
> I wonder if in 10 years people will be stuck debugging Rube-Goldberg machines composed of LLM api calls doing stuff that if-statements can do, probably cobbled together with actual if-statements Sounds like an extension of https://en.wikipedia.org/wiki/Wirth%27s_law https://en.wikipedia.org/wiki/Wirth%27s_law. How many times have I done some simple arithmetic by typing it into my browser's bar and checking out the google calculator results? When a generation ago I would have plugged it into a calculator on my desk (or done it in my head, for that matter...). I would be entirely unsurprised to hear that in another generation we're using monstrously complicated "AI" systems to perform tasks that could be done way more simply/efficiently just because it's convenient.
- vidarh 4y ago
  My son regularly uses Alexa as a calculator, and also asks Alexa all kinds of things without a thought as to whether the output triggers a simple pattern match and gets fed to a specialised process or triggers a web search or is processed some other way. It's all conversational anyway. So the day Amazon plugs an LLM into it, it's not a given he'll even notice the difference for some time.
Hermitian909 4y ago
LLMs are already taking over these sorts of systems in industry. There are lots of systems where you're taking some information about a user and making a best guess at what action the system should take. Even without a need for super high accuracy these rule systems can get surprisingly complex and adding in new possible decisions can be tricky to maintain. In LLM world you just maintain a collection of possible actions and let the LLM map user inputs to those.
DonHopkins 4y ago
You're totally undermining your own argument by using "shell script" instead of "python script".

JohnFen 4y ago

> doing stuff that if-statements can do, probably cobbled together with actual if-statements In other words, old-school expert systems.

baq 4y ago
With the limit of 25k words it might actually be reasonable to test out a prompt for an expert system… but I’d still leave reasoning to something else, for now. Z3, prolog or some forward chaining tool like clips, but have the LLM hallucinate some of the rules?

nico 4y ago

Sure, maybe you can use a shell script, but now the AI assistant can write it based on your verbal/text description, and then the assistant can also run it for you after you’ve checked it. What your are saying is: “why use the washing machine, if I my clothes are even cleaner when I wash them myself - I also spend less detergent and less water”. You are free to keep doing your laundry by hand. But I bet most people prefer the washing machine.

xwdv 4y ago
Spare me the shitty analogies. We write shell scripts because it’s cheap, fast, and the behavior is very predictable. Like it or not, an AI’s behavior is a black box and can’t be “proven” to execute exactly the same every time for the scenarios you are targeting. A shell script will do exactly what it has been written to do every time, unless tampered with. And if changes need to be made, it can be done quickly without need for retraining, god knows how long that would take for an AI to learn something new. God help you if you need to maintain “versions” of your AI, trained for different things. Face it, AI are pointless and slow for certain classes of problems.
- com2kid 4y ago
  > A shell script will do exactly what it has been written to do every time, unless tampered with. Or unless some magic environment variable changes, or one of the runtime dependencies changes, or it is run on a different operating system, or permissions aren't setup right, or one of its tasks errors out. Shell scripts are digital duct tape, the vast majority of shell scripts do not come close to being reliable software. > god knows how long that would take for an AI to learn something new Did you watch OpenAI's demo yesterday? They pasted in new versions of API docs and GPT4 updated its output code. When GPT forgot a parameter, the presenter fed back the error message and GPT added the parameter to the request.
  - xwdv 4y ago
    AI proponents are missing the point. Anything you write to make an AI produce something is basically code. Docs are code. You don’t have to feed a developer code or docs, you can give them a high level idea and they’ll figure it out on their own if you want.
    - echelon 4y ago
      That code will eventually fall away. The big thing everyone in this single thread is missing is that AI is a metaheuristic. I wouldn't expect to use AI to run_script.py. That's easy. I'd expect it to look at the business signals and do the work of an intern. To look at metrics and adjust some parameters or notify some people. To quickly come up with and prototype novel ways to glue new things together. To solve brand new problems.
      - xwdv 4y ago
        To do the work of an intern an AI must go on Jira, read a ticket, then find the appropriate repositories where code needs to be modified, write tests for its modification, submit for code review, respond to feedback in code review, deploy its changes. It’s not there yet.
        4y ago
whatshisface 4y ago
I think you're fighting an uphill battle because of what you picked to defend here - shell scripts are very easy to write, and I have a hard time imagining a future where someone tells an LLM, "Write me a shell script that runs run_control.py with the speed argument set to one hundred." to get, "./run_control.py --speed 100"
- pstorm 4y ago
  That's a trivial example, sure. Think of saying "make this image more red, flip it vertically, then crop the bottom 25%, and finally output it with 80% compression as a jpg". That would take 15 minutes to figure out how to write that with imagick, or just have an LLM do it for you.
  - tough 4y ago
    I had to try, seems reasonable, haven't tested the code As an AI language model, I do not have direct access to image editing software, but I can provide you with a shell script that could achieve the described image manipulation using ImageMagick, assuming it is installed on your system. Here's an example shell script that you could use as a starting point: bash Copy code #!/bin/bash # Set the input image filename input_file="input.jpg" # Set the output image filename output_file="output.jpg" # Increase the redness of the image convert "$input_file" -modulate 100,150,100 -channel R -evaluate multiply 1.2 +channel temp1.jpg # Flip the image vertically convert temp1.jpg -flip temp2.jpg # Crop the bottom 25% of the image convert temp2.jpg -gravity South -chop 0x25% temp3.jpg # Compress the image to 80% convert temp3.jpg -quality 80 "$output_file" # Clean up temporary files rm temp*.jpg To use this script, save it to a file (e.g., "image_manipulation.sh"), make it executable (e.g., "chmod +x image_manipulation.sh"), and run it in the same directory as your input image. The output image will be saved in the same directory with the name "output.jpg". Please note that this is just an example script and may need to be modified depending on your specific image manipulation needs.
    - fulafel 4y ago
      This has lots of lossy encode/decode cycles that will accumulate artifacts. Good anecdote.

querez 4y ago

It's not wrong. It's how modern systems operate. E.g. look at Google's SayCan (https://say-can.github.io/ https://say-can.github.io/) which operates exactly like this (an LLM ordering a Robot around).

jrussino 4y ago

> I wonder if in 10 years people will be stuck debugging Rube-Goldberg machines composed of LLM api calls doing stuff that if-statements can do, probably cobbled together with actual if-statements Sounds like an extension of https://en.wikipedia.org/wiki/Wirth%27s_law https://en.wikipedia.org/wiki/Wirth%27s_law. How many times have I done some simple arithmetic by typing it into my browser's bar and checking out the google calculator results? When a generation ago I would have plugged it into a calculator on my desk (or done it in my head, for that matter...). I would be entirely unsurprised to hear that in another generation we're using monstrously complicated "AI" systems to perform tasks that could be done way more simply/efficiently just because it's convenient.

vidarh 4y ago
My son regularly uses Alexa as a calculator, and also asks Alexa all kinds of things without a thought as to whether the output triggers a simple pattern match and gets fed to a specialised process or triggers a web search or is processed some other way. It's all conversational anyway. So the day Amazon plugs an LLM into it, it's not a given he'll even notice the difference for some time.

Hermitian909 4y ago

LLMs are already taking over these sorts of systems in industry. There are lots of systems where you're taking some information about a user and making a best guess at what action the system should take. Even without a need for super high accuracy these rule systems can get surprisingly complex and adding in new possible decisions can be tricky to maintain. In LLM world you just maintain a collection of possible actions and let the LLM map user inputs to those.

DonHopkins 4y ago

You're totally undermining your own argument by using "shell script" instead of "python script".