4 ms·

Increasingly, I'm able to empathise with people who use this psychobable language (hallucination, bullshit, etc.) to describe associative statistical models. T

Increasingly, I'm able to empathise with people who use this psychobable language (hallucination, bullshit, etc.) to describe associative statistical models. They just don't know how they work, really want to believe they work like a person, or have otherwise been duped by the boards of the ad-ai industrial complex whose spiel is whatever increases market valuation (google, fb, ms, openai, etc.). I guess if you have absolutely no idea how these systems work there really isn't anything better than analogizing to people -- it is this 'failure of analogical reasoning' which ai/ad corps exploit in their desire to dupe ever more misinformed investors. This realisation helps me, only, since I can now fathom better when i'm wasting my time talking to someone whose basis for understanding stats is human psychology -- the amount of discussion needed to undo that is possibly a lot. Incidentally, all statistical systems of this kind (almost all ML, including NNs) work in the same way. They compress data into a smaller representation, and take a weighted average of pieces of that data relevant (by freq association) to the prediction. There is no reasoning, no truth, no falsehood, no lies, no bullshit, no hallucination, no veridical perception, no interiority, no subjectivity, no capacity to represent the world. There is something much more like a few pages of a book stuck together, with the second shinning through the first.

macNchz 3y ago
Regardless of how they work, LLMs are already quite enmeshed in the world of humans–“participating” to a degree in our interactions and discourse, used every day by millions of people who do not and will not have a mathematical mental model for how they work–so exploring how they operate from this lens is a useful exercise, in my opinion.
- mjburgess 3y ago
  Well, I don't begrudge people doing it -- I'm doubtful its useful. More-so dangerous, maladaptive, pathological, etc. People will eventually acquire a more accurate mental model, like using search engines in the early days. What riles me is the grubbiness of the commercial pushers, and the foolishness of the press which repeat their marketing material as-if its science. People should not rely on computers in the way they rely on people -- and any inducement to doing so is unethical.
alex_c 3y ago
“There is no reasoning, no truth, no falsehood, no lies” etc is as close as it gets to the actual definition of “bullshit” used here. I agree with you about anthropomorphizing terms like “hallucinating” and “lies”, which imply belief, intent, or knowledge in a human sense. “Bullshit”, as used here, is a perfect term to describe LLM output specifically because it indicates that none of those terms and concepts apply.
flyval 3y ago
We use language like “hallucinations, bullshit, etc.”, because it’s more accurate descriptors for what’s happening. When I tell you the LLM hallucinated something, you know exactly what I mean. Same for “It’s rationalizing”, “It’s refusing to answer”, “It’s being illogical”. Your desire to reduce it all down to stats is as a bad as someone arguing that we shouldn’t talk about balls and goals in basketball, because it’s all just atoms.

macNchz 3y ago

Regardless of how they work, LLMs are already quite enmeshed in the world of humans–“participating” to a degree in our interactions and discourse, used every day by millions of people who do not and will not have a mathematical mental model for how they work–so exploring how they operate from this lens is a useful exercise, in my opinion.

mjburgess 3y ago
Well, I don't begrudge people doing it -- I'm doubtful its useful. More-so dangerous, maladaptive, pathological, etc. People will eventually acquire a more accurate mental model, like using search engines in the early days. What riles me is the grubbiness of the commercial pushers, and the foolishness of the press which repeat their marketing material as-if its science. People should not rely on computers in the way they rely on people -- and any inducement to doing so is unethical.

alex_c 3y ago

“There is no reasoning, no truth, no falsehood, no lies” etc is as close as it gets to the actual definition of “bullshit” used here. I agree with you about anthropomorphizing terms like “hallucinating” and “lies”, which imply belief, intent, or knowledge in a human sense. “Bullshit”, as used here, is a perfect term to describe LLM output specifically because it indicates that none of those terms and concepts apply.

flyval 3y ago

We use language like “hallucinations, bullshit, etc.”, because it’s more accurate descriptors for what’s happening. When I tell you the LLM hallucinated something, you know exactly what I mean. Same for “It’s rationalizing”, “It’s refusing to answer”, “It’s being illogical”. Your desire to reduce it all down to stats is as a bad as someone arguing that we shouldn’t talk about balls and goals in basketball, because it’s all just atoms.