I think this point is fair given that "AI" is a highly provisional term. This point implicitly re-frames the capability as search engine, and steers away from

I think this point is fair given that "AI" is a highly provisional term. This point implicitly re-frames the capability as search engine, and steers away from any idea of thought as an innate capability the AI device. As long as as the oracle's answer is expected to require human verification, what's going on is a human thought amplification (bulldozer for for the mind, to quote a famous computational linguist). Even taking the point as an upside, I see tremendous hazards for human-mediated oracles (as already observed by many others): that the "well" of the commons can be polluted by AI output feeding back into the training sets, and that human hallucinations are amplified. From a POV of ethics, hallucination is absolutely intolerable in human decision-making in industrial settings. Yet industrial users seem not just tolerant of AI acting crazy but completely accepting and forgiving of AI craziness. It's routinely considered to be interesting and exciting to see it go wrong and tremendous effort attends trying to steer the craziness in new directions, whereas in a human setting the hallucinating agent would be disqualified from participation. The IT industry is taking very dangerous tacks without a second thought. I recently saw a job posting for a firm called UpCodes that invites junior web stack programmers to add LLM tech to building code compliance systems as follows: https://up.codes/careers?ashby_jid=1ed9b2d6-86fb-4ff0-9dff-4d215e5e08e4 https://up.codes/careers?ashby_jid=1ed9b2d6-86fb-4ff0-9dff-4... // EXAMPLE PROJECTS • Use an LLM to identify references to other sections in the text of the law • Create Rap Genius-style annotations on sections of building code, to create a more interactive user experience • Write algorithms to detect and display how versions of a code have changed through time, so users understand how regulations have evolved • Migrate our NLP algorithms over to use LLMs // WHAT COULD GO WRONG? And what qualifications, standards and practices are demanded of this developer position? // • 3+ years of experience building web products, ideally across the stack • Clean simple code, even when solving complex problems • Strong sense of UX • Good systems design • Ability to learn new tech quickly // DONE AND DONE