While LLMs have incredible potential, and are even downright useful in their current format, they have the rather nasty tendency to confidently present bullshit

While LLMs have incredible potential, and are even downright useful in their current format, they have the rather nasty tendency to confidently present bullshit that passes the smell test for most people. When it's wrong, it's not just wrong, by design, it's wrong but sounds plausible. Considering most people with subject matter expertise aren't going to consult such a bot for their own areas of expertise, that means most people will not be able to detect it intuitively. Good example: I sunk probably an hour into trying to get Gemini Advanced to help me integrate it with a personal Google Calendar account. I kept asking it things and going crazy because nothing lined up with the way things worked. Finally, it referred to itself as Bard and I realized it was giving me information for a different product. As soon as I asked "are you giving me instructions for Gemini Advanced or Bard?" it was like "OH LOL WOOPS!! YOU GOT ME BRO! XD I CAN'T DO ANY OF THAT! LOL." Which, honestly, is great. Being able to evaluate its answers to realize it's wrong is really neat. Unfortunately, it was neat too late and too manually to stop me from wasting a ton of time. I have decades of experience working in software-- imagine some rando that didn't know what the hell Bard was or even imagine this thing with "Advanced" in the name couldn't even distinguish between its own and other products' documentation.