4 ms·

In any important domain where accountability and factuality matters LLM's will not get the human out of the loop. They (hopefully) will simply make their job e

In any important domain where accountability and factuality matters LLM's will not get the human out of the loop. They (hopefully) will simply make their job easier but there are hidden risks: identifying well written "hallucinations" is not trivial. Tip for the AI brigade: Assign a confidence level to segments of generated text.

danmaz74 3y ago
As things are now, anything that needs to be factual or reasoning seems like a bad match for LLMs. Where I find them very useful is at rewording things, or at continuing something repetitive enough along a given pattern (eg in programming).
permo-w 3y ago
>Tip for the AI brigade: Assign a confidence level to segments of generated text. you say this as if it’s a trivial feature that AI developers have just not bothered to leave in
- nologic01 3y ago
  I know its not trivial. You don't change the world by doing trivial things. Its a placeholder for features that will enable more controlled used of these tools. The important aspect is to be able to guide the "reviewer" where to focus on. It could still be a flawed suggestion but its better than nothing.
  - permo-w 3y ago
    you also don’t change the world with off-handed comments at the end of HN posts tip for the crypto brigade: make your coins solid investments
denton-scratch 3y ago
> Assign a confidence level to segments of generated text. Text generators are not reasoning machines. To assign a confidence level, you need a tree of facts and deductions. This is what you can get from an expert system. With an expert system, you can in principle ask the system to explain its reasoning, and assign confidence to each step. ML systems are black boxes. Expert systems are hard to train; you need a human expert, and probably a specialist in the expert system sitting beside him. And it takes a lot of work - so it's expensive. I can imagine a hybrid expert system/machine-learning system, where the ML system is used to distil the expertise that is then used by the expert system to construct decision trees that are not opaque. I wish a fraction of the money that gets thrown at LLM "toys" were instead invested in expert systems. I think expert systems research became unfashionable around the end of the 1980s, which is a crying shame.
- rini17 3y ago
  Basically LLM has no way of knowing that it can't as freely arrange tokens to make case references as it does for other text. Maybe it can be fixed by more parameters or something, not done yet.
- nologic01 3y ago
  I am not at all pessimistic about expert systems. These things go in cycles and piggybag on developments. NN's were niche for a long time. Speculative hypes act as a sort of Zitterbevegung: moving fast yet going nowhere. But my suggestion is actually less ambitious: not to create confidence with reference to outside knowledge but purely on the basis of the training sample and its intrinsic statistics. [1] https://en.wikipedia.org/wiki/Zitterbewegung https://en.wikipedia.org/wiki/Zitterbewegung

danmaz74 3y ago

As things are now, anything that needs to be factual or reasoning seems like a bad match for LLMs. Where I find them very useful is at rewording things, or at continuing something repetitive enough along a given pattern (eg in programming).

permo-w 3y ago

>Tip for the AI brigade: Assign a confidence level to segments of generated text. you say this as if it’s a trivial feature that AI developers have just not bothered to leave in

nologic01 3y ago
I know its not trivial. You don't change the world by doing trivial things. Its a placeholder for features that will enable more controlled used of these tools. The important aspect is to be able to guide the "reviewer" where to focus on. It could still be a flawed suggestion but its better than nothing.
- permo-w 3y ago
  you also don’t change the world with off-handed comments at the end of HN posts tip for the crypto brigade: make your coins solid investments

denton-scratch 3y ago

> Assign a confidence level to segments of generated text. Text generators are not reasoning machines. To assign a confidence level, you need a tree of facts and deductions. This is what you can get from an expert system. With an expert system, you can in principle ask the system to explain its reasoning, and assign confidence to each step. ML systems are black boxes. Expert systems are hard to train; you need a human expert, and probably a specialist in the expert system sitting beside him. And it takes a lot of work - so it's expensive. I can imagine a hybrid expert system/machine-learning system, where the ML system is used to distil the expertise that is then used by the expert system to construct decision trees that are not opaque. I wish a fraction of the money that gets thrown at LLM "toys" were instead invested in expert systems. I think expert systems research became unfashionable around the end of the 1980s, which is a crying shame.

rini17 3y ago
Basically LLM has no way of knowing that it can't as freely arrange tokens to make case references as it does for other text. Maybe it can be fixed by more parameters or something, not done yet.
nologic01 3y ago
I am not at all pessimistic about expert systems. These things go in cycles and piggybag on developments. NN's were niche for a long time. Speculative hypes act as a sort of Zitterbevegung: moving fast yet going nowhere. But my suggestion is actually less ambitious: not to create confidence with reference to outside knowledge but purely on the basis of the training sample and its intrinsic statistics. [1] https://en.wikipedia.org/wiki/Zitterbewegung https://en.wikipedia.org/wiki/Zitterbewegung