3 ms·

I am using many many many things that I don't understand. Cars, public transport, etc... I review and test the end product, not every tiny step along the way.

by Certhas 2mo ago

I am using many many many things that I don't understand. Cars, public transport, etc... I review and test the end product, not every tiny step along the way. If the LLM uses some command line tools I have never heard of to create a model I can verify, why should I learn a tool that is completely irrelevant to my core expertise?

jason1cho 2mo ago
Many many people care more than the end product, for example whether a shirt is made of cotton with the forced labor, carbon emissions of public transport, etc. In terms of engineering software, you care the cost. An intelligent agent may try to read unnecessary files and it's time to stop it to save tokens and avoid polluting the context.
- usef- 2mo ago
  He didn't advocate for being completely blind in every way. You might care about working conditions without understanding how the textiles, dyes or cotton production works. These non-programmers probably shouldnt use computers at all, right, since they don't understand them?
- user43928 2mo ago
  I doubt that it is more efficient for someone to routinely watch every line of output or stop to review terminal commands, rather than waiting for the turn to complete. It is a broader debate about agentic AI, and whether one should relinquish control to the tool rather than aim for full understanding of every action taken. The people arguing for a hands-on, fully in control approach are losing ground by the week, in my opinion.
  - mcmcmc 2mo ago
    It’s definitely on the high friction side of the usability/security tradeoff, I just think auto modes like this are as liability-inducing as handing a script kiddie intern full admin on your production environment The real answer is somewhere in the middle and is probably a mix of traditional AV/EDR and AI QA judges that mitigate risk of running more or less random arbitrary code and auto approve based on configured detection rules and your personal risk tolerance. Would it suck to stick an EDR sensor in every code execution environment spun up for an agent to run a python script… yes. It would also suck if you were responsible for hacking a company without knowing about it because you didn’t watch what your AI was doing
- 9dev 2mo ago
  I can be against animal testing without being a chemist or having a full understanding of the experiments being made on them. Knowing it's cruelty is enough to make opposition a valid and defensible position.
mcmcmc 2mo ago
Are you jacking into your car’s OBD port and messing with the engine timing? There’s a difference between using a known tool in a controlled way and giving the tool to a hallucinatory goblin (or overconfident intern) with the directive “do it for me”. You don’t have to know everything about the tool to know what it does, or to tell if it’s doing something it shouldn’t (like malware). What happened to the old advice given to tech neophytes, “don’t run random scripts from the internet if you don’t know what they do”?
- mcmcmc 2mo ago
  Just continuing the car example, because you actually do need to know enough about it to operate it and get your license. There’s no license, no insurance for using AI. Driving can be deadly, and when you do it on a public roadway there are certain requirements that have to be met as agreed upon by most governments. Same should go for AI. Do whatever you want with it on your machine, but when you let it out on the internet, you become liable for any damage you, and by extension your AI, may cause. If you feel uncomfortable approving its actions because you don’t understand them, you should either a) take the time to understand, then approve, or b) listen to your discomfort and don’t do the thing. Frankly I think that’s more accessible because you then see the decision points instead of leaving Oz behind the curtain. Much easier for a neophyte to learn from that instead of trying to reverse engineer a final output.
lukan 2mo ago
"If the LLM uses some command line tools I have never heard of to create a model I can verify, why should I learn a tool that is completely irrelevant to my core expertise?" Because that command might also give someone else access to your computer along the way. So your tool seems to work, but your computer is owned.
- Certhas 2mo ago
  [dead]

by Certhas 2mo ago

jason1cho 2mo ago

Many many people care more than the end product, for example whether a shirt is made of cotton with the forced labor, carbon emissions of public transport, etc. In terms of engineering software, you care the cost. An intelligent agent may try to read unnecessary files and it's time to stop it to save tokens and avoid polluting the context.

usef- 2mo ago
He didn't advocate for being completely blind in every way. You might care about working conditions without understanding how the textiles, dyes or cotton production works. These non-programmers probably shouldnt use computers at all, right, since they don't understand them?
user43928 2mo ago
I doubt that it is more efficient for someone to routinely watch every line of output or stop to review terminal commands, rather than waiting for the turn to complete. It is a broader debate about agentic AI, and whether one should relinquish control to the tool rather than aim for full understanding of every action taken. The people arguing for a hands-on, fully in control approach are losing ground by the week, in my opinion.
- mcmcmc 2mo ago
  It’s definitely on the high friction side of the usability/security tradeoff, I just think auto modes like this are as liability-inducing as handing a script kiddie intern full admin on your production environment The real answer is somewhere in the middle and is probably a mix of traditional AV/EDR and AI QA judges that mitigate risk of running more or less random arbitrary code and auto approve based on configured detection rules and your personal risk tolerance. Would it suck to stick an EDR sensor in every code execution environment spun up for an agent to run a python script… yes. It would also suck if you were responsible for hacking a company without knowing about it because you didn’t watch what your AI was doing
9dev 2mo ago
I can be against animal testing without being a chemist or having a full understanding of the experiments being made on them. Knowing it's cruelty is enough to make opposition a valid and defensible position.

mcmcmc 2mo ago

Are you jacking into your car’s OBD port and messing with the engine timing? There’s a difference between using a known tool in a controlled way and giving the tool to a hallucinatory goblin (or overconfident intern) with the directive “do it for me”. You don’t have to know everything about the tool to know what it does, or to tell if it’s doing something it shouldn’t (like malware). What happened to the old advice given to tech neophytes, “don’t run random scripts from the internet if you don’t know what they do”?

mcmcmc 2mo ago
Just continuing the car example, because you actually do need to know enough about it to operate it and get your license. There’s no license, no insurance for using AI. Driving can be deadly, and when you do it on a public roadway there are certain requirements that have to be met as agreed upon by most governments. Same should go for AI. Do whatever you want with it on your machine, but when you let it out on the internet, you become liable for any damage you, and by extension your AI, may cause. If you feel uncomfortable approving its actions because you don’t understand them, you should either a) take the time to understand, then approve, or b) listen to your discomfort and don’t do the thing. Frankly I think that’s more accessible because you then see the decision points instead of leaving Oz behind the curtain. Much easier for a neophyte to learn from that instead of trying to reverse engineer a final output.

lukan 2mo ago

"If the LLM uses some command line tools I have never heard of to create a model I can verify, why should I learn a tool that is completely irrelevant to my core expertise?" Because that command might also give someone else access to your computer along the way. So your tool seems to work, but your computer is owned.

Certhas 2mo ago
[dead]