This is the bit that maybe programmers haven’t quite realized yet: if I can get an LLM to just do that task, I don’t need to go and get someone to bui

This is the bit that maybe programmers haven’t quite realized yet: if I can get an LLM to just do that task, I don’t need to go and get someone to build some software to do it. I think everybody understands that part, and that the cynicism/skepticism is about LLMs' ability to deliver on that aspect of it. Part of training John, or writing a script, or telling an LLM, is handling all of the edge cases that exist in even a "simple" import job. - What to do if there is a duplicate record? Create a dupe? Do an upsert? If data conflicts how we decide which version "wins?" - What about fuzzy dupes? Two records that are probably identical but not really the same, probably due to typos or slightly different accent characters, etc - "find the CRM record for each company listed" <-- okay, and what if the record doesn't exist? fail? - Is the import job an all-or-nothing thing? In other words should we wrap everything in a transaction and back it out if there's a single failure? Or do we keep going? Do we log failures? Email somebody? - If we're wrapping the whole job in a transaction, should we lock out the users while this is happening so they don't see stale data? Do we notify them? Do we just do it in the middle of the night? - Should we log changes to an audit table for debugging purposes? - What if the import file is invalid? Do we log, email, etc? etc etc etc. Like Visual Basic, Excel macros, etc I think LLM will enable some non-programmers to do scripty, programmery stuff and I think that is excellent. But anything really robust (and honestly, "simple" import jobs are often critical path and therefore need to be fairly robust) is going to require some rigor. A person telling LLM what to do is still going to need to address all of those edge cases and then some. It's also very unclear to me how we'll debug LLM-managed processes. When things go wrong, how do we determine what/when/why? And in fact, how do we determine LLM-managed processes are correct in the first place? These particualr concerns seem completely solvable, but they will need to be solved.