4 ms·

A while ago I needed some info about a document for tax reasons. When I called in to the UK tax office line a robot voice required me to name the department tha

A while ago I needed some info about a document for tax reasons. When I called in to the UK tax office line a robot voice required me to name the department that I wanted to talk to. I didn't know which this was and it wasn't on their website (and indeed "who do I request this document from?" was the essence of my question). Speak to a human operator was not offered as an option. I think I just said random words until it put me through to some departmen and from there they had a normal call tree via which I got an unrelated human who could tell me who I actually needed to ask for. But I'm not looking forward to the day that no humans are in the loop and unanticipated circumstances are completely unresolvable. I fear our AI future not because of evil but because of bureaucrats.

telesilla 2y ago
I'm also resorting to punching 1,1,1,1 or whatever combo works and asking the first person who answers on that tree to put me through where I need to go. For voice activation it means making unintelligible sounds in the hopes of the system switching to a human. Strangely enough, whatsapp business is becoming a much better experience in place of calling, but sometimes even chat isn't enough.
ben_w 2y ago
> I fear our AI future not because of evil but because of bureaucrats. Fair, though the advantage of an actual LLM here is that it's not limited to a dumb hard-coded menu, so if done right (I know, I know) an LLM would help a lot. (One of the disadvantages is that current models sometimes extemporise answers even if none exist).
- more_corn 2y ago
  That mildly stated problem is the crux of the matter. LLMS hallucinate and always will.
  - ben_w 2y ago
    "Always" is a risky claim in AI. Though given humans also do so, perhaps warranted in this case. https://en.wikipedia.org/wiki/Dunning–Kruger_effect https://en.wikipedia.org/wiki/Dunning–Kruger_effect
    - more_corn 2y ago
      LLMS are not AI. Large language models have no formal reasoning, they have no long term recall, they contain no structured logic. I’m not saying it can’t be solved. I’m saying it can’t be solved INSIDE the LLM. Anyone with a phd in machine learning would probably agree.
      - ben_w 2y ago
        > LLMS are not AI. Ironic demonstration that humans also do what is deemed "hallucinations" when AI do it: https://en.wikipedia.org/wiki/AI_effect https://en.wikipedia.org/wiki/AI_effect LLMs, transformer models in particular, are artificial neural networks. They have always been AI. AI is the field which led to this, the research is published as AI research. It's amazing how often us humans (me included!) don't use RAG (retrieval augmented generation) in the form of a search engine and just trust our gut instinct for off-the-cuff responses :D > Large language models have no formal reasoning, they have no long term recall, they contain no structured logic. > I’m not saying it can’t be solved. I’m saying it can’t be solved INSIDE the LLM. Do you mean transformers then? Because that is the current vogue architecture for large language models which is clearly a broader category. The full details for the current best models are secret, but they're still large language models, and they're demonstrating surprisingly high performance on logic and reasoning.
        handoflixue 2y ago
        For all that LLMs produce good performance, it's still just predictive texting. If you can get the hallucination rate on that down to 0% without using anything else, I'll be extremely shocked. Now, layer a few sanity checks on top of an LLM, especially some clever thing we haven't invented yet, and I'll totally believe it - the task is absolutely doable, I'd just find it really weird if a predictive engine could do it 100% accurately, using only modern resources.

telesilla 2y ago

I'm also resorting to punching 1,1,1,1 or whatever combo works and asking the first person who answers on that tree to put me through where I need to go. For voice activation it means making unintelligible sounds in the hopes of the system switching to a human. Strangely enough, whatsapp business is becoming a much better experience in place of calling, but sometimes even chat isn't enough.

ben_w 2y ago

> I fear our AI future not because of evil but because of bureaucrats. Fair, though the advantage of an actual LLM here is that it's not limited to a dumb hard-coded menu, so if done right (I know, I know) an LLM would help a lot. (One of the disadvantages is that current models sometimes extemporise answers even if none exist).

more_corn 2y ago
That mildly stated problem is the crux of the matter. LLMS hallucinate and always will.
- ben_w 2y ago
  "Always" is a risky claim in AI. Though given humans also do so, perhaps warranted in this case. https://en.wikipedia.org/wiki/Dunning–Kruger_effect https://en.wikipedia.org/wiki/Dunning–Kruger_effect
  - more_corn 2y ago
    LLMS are not AI. Large language models have no formal reasoning, they have no long term recall, they contain no structured logic. I’m not saying it can’t be solved. I’m saying it can’t be solved INSIDE the LLM. Anyone with a phd in machine learning would probably agree.
    - ben_w 2y ago
      > LLMS are not AI. Ironic demonstration that humans also do what is deemed "hallucinations" when AI do it: https://en.wikipedia.org/wiki/AI_effect https://en.wikipedia.org/wiki/AI_effect LLMs, transformer models in particular, are artificial neural networks. They have always been AI. AI is the field which led to this, the research is published as AI research. It's amazing how often us humans (me included!) don't use RAG (retrieval augmented generation) in the form of a search engine and just trust our gut instinct for off-the-cuff responses :D > Large language models have no formal reasoning, they have no long term recall, they contain no structured logic. > I’m not saying it can’t be solved. I’m saying it can’t be solved INSIDE the LLM. Do you mean transformers then? Because that is the current vogue architecture for large language models which is clearly a broader category. The full details for the current best models are secret, but they're still large language models, and they're demonstrating surprisingly high performance on logic and reasoning.
      - handoflixue 2y ago
        For all that LLMs produce good performance, it's still just predictive texting. If you can get the hallucination rate on that down to 0% without using anything else, I'll be extremely shocked. Now, layer a few sanity checks on top of an LLM, especially some clever thing we haven't invented yet, and I'll totally believe it - the task is absolutely doable, I'd just find it really weird if a predictive engine could do it 100% accurately, using only modern resources.