How many leaf URLs have you memorised? I don't mean "google.com", I mean the URL of a specific paper in a journal, or an article in some random news site? If y

How many leaf URLs have you memorised? I don't mean "google.com", I mean the URL of a specific paper in a journal, or an article in some random news site? If you were forced at gunpoint to provide such URLs -- valid or not -- with literally no option other than to keep talking, you would... make up some vaguely correct-sounding URLs! The AIs are doing precisely what an intelligent human in their place would do. Blurting out something, anything, even if they don't have the answers. That's because the current crop of AIs are essentially being forced to provide the next token, directly equivalent to a human being forced to continue speaking against their will. There are fixes being put in place already for this, such as using human feedback to tune the AI so that it answers with a generic "I don't know" blurb if it's not sufficiently confident in the next token. Annoyingly, the new "Turbo" versions of ChatGPT have overdone this a bit, and now it refuses to answer even when it does actually know the answer! There are also variations of neural networks that encode not just "values", but probabilities and hence confidence intervals. These can directly output both tokens and the true confidence values, which would allow complex tree searches, etc... and avoid outputting this kind of gibberish entirely. The maths for this is well established, but it would be much more expensive to run models using it, so nobody has bothered yet.