Currently working on an AI product (essentially a OS-context-sensitive agent caller, basically a "smart" Spotlight for Windows/macOS), and from my experience (~

Currently working on an AI product (essentially a OS-context-sensitive agent caller, basically a "smart" Spotlight for Windows/macOS), and from my experience (~2 months of working on this basically full-time), complex prompts (basically all of the ones provided by Anthropic here) are a mistake. Not only does the quality of output decrease exponentially with complexity (do x which gives you y, and for every y, do z), but also (and this is key), it messes with end-user expectations. When looking at an email, asking the AI to "summarize this email" will basically work 100% of the time; when looking at a Github webpage and asking it to "clone this repo" will also work 100% of the time. But doing some weird combination of the two "clone the repo this email references" works reliably around 1/2^n (25%) of the time. I think that there are many very low hanging fruit that are totally achievable using local LLMs, but too many people enamored by a mythical pie-in-the-sky "it does everything" app that we're missing basic things like: clone a git repo without alt-tabbing in my shell, semantic search of PDFs in a folder, add an event to my calendar without alt-tabbing in a new Google Calendar tab, and so on. Users need to be trained to give simple one-shot requests to AI. The SQL example is particularly egregious: "Get the list of customers who have placed orders but have not provided any reviews, along with the total amount they have spent on orders." This is a disaster waiting to happen (I'd be curious to mess with the schema and watch it crash and burn); I'm almost certain that the example output is cherry picked.