3 ms·
Completely agree! People tend to forget we are non deterministic too! Yet we are able to write code fine, and fairly reliably by using tools that can help keep
by khasan222 5mo ago
Completely agree! People tend to forget we are non deterministic too! Yet we are able to write code fine, and fairly reliably by using tools that can help keep us fairly honest.
I think most problems with ai tend to be around can you deterministically test the thing you are asking it to do?
How many of us would never ever show work, without going to check the thing we just built first?
- cluckindan 5mo ago> can you deterministically test the thing you are asking it to do? Of course: have it write tests first; and run them to check its work. Works well for refactoring, but greenfield implementations still rely on a spec that is guaranteed to be incomplete, overcomplete and wrong in many ways.
- pishpash 5mo agoYou can't ask something to check its own work without external reward/penalty. It'll cheat.
- khasan222 5mo agoWeirdly, and i fully think this is just some cognitive bias I don't have the knowledge to name, the ai seems very happy to please me. Like when it gets something done in one shot, it seems very happy to do so.
- daveguy 5mo agoIt's because expressing emotion tests well in RLHF (reinforcement learning, human feedback), which is the layer on top of the next-token-predictor LLM. As a bonus, it helps manipulate operator reactions to incorrect output, and improve engagement (aka token use). The "thought process" of an LLM only exists as inference response to next token prediction prompts. It's the illusion of emotion.
- khasan222 5mo agoWell if the spec is incomplete it sounds like you should lower scope for the AI, and then go from there. I wouldn't be too keen to give a junior engineer free reign and expect awesomeness