This was my chief critique when my company forced us to use their AI tooling. I was trying to stitch together our CMDB, two different VMware products, and the

This was my chief critique when my company forced us to use their AI tooling. I was trying to stitch together our CMDB, two different VMware products, and the corporate technology directory into a form of product tenancy for our customers. At one point I was trying to move and transform data from our CRM into mongoDB, and figured "eh, let's knock these mandatory agent queries out of the way by asking the chatbot to help." I wrote a few prompts to try and explain what I was trying to accomplish (context), and how I'd like it done (instruction). The bot hallucinated a non-existent mongoDB Powershell cmdlet, complete with documentation on how it works, and then spat out a "solution" to the problem I asked. Every time I reworked the prompt, cut it up into smaller chunks, narrowed the scope of the problem, whatever I tried, the chatbot kept flatly hallucinating non-existent cmdlets, Python packages, or CLI commands, sometimes even providing (non-working) "solutions" in languages I didn't explicitly ask for (such as bash scripting instead of Powershell). This was at a large technology company, no less, one that's "all-in" on AI. If you're staying in a very narrow line with a singular language throughout and not calling custom packages, cmdlets, or libraries, then I suspect these things look and feel quite magical. Once you start doing actual work, they're complete jokes in my experience.