3 ms·

> Indeed, we can probe model inner layers and infer if it is "lying"[1] This is hyperbole to the point where it is more false than true. If it was true, OpenAI

> Indeed, we can probe model inner layers and infer if it is "lying"[1] This is hyperbole to the point where it is more false than true. If it was true, OpenAI, Anthropic and Google would already have solved the problem of making models honest. Just adding a reference to some random arxiv paper doesn't automatically justify your bold assertion.

InDubioProRubio 2y ago
The projects themselves specified that lying is okay. Output at all costs, at all parameters, as interpolating is a feature, not a bug. It could just error out at loss of certainty, but then you would stand there, having nothing most of the time.
- ithkuil 2y ago
  Interpolating is an awesome feature, very practical: summarization, style transfer. Instruction following is also useful and unlocks another tier of features. The fuzziness of the model behaviour plays a useful role because natural language is imprecise. The ability to interpolate this remains useful. Once machines understand natural language our own psychology kicks in and assigns properties like "wisdom" and "knowledge" to agents that can "talk well". This may lead us to expect something from tools that haven't been designed to perform a given task. For example our current LLMs are not very good at precise information retrieval because they haven't been designed for that. They happen to be able to perform information retrieval as a side product of the mechanism that teaches them how to speak in the first place. The raw output of the broca area in our brains is probably not something you want to connect directly to the mouth without mediation
  - InDubioProRubio 2y ago
    We need training exercises that teach us, how to think like a llm. Less like a human, more like a free association machine, dreaming a specific association flow, starting from a input. https://en.wikipedia.org/wiki/Ulysses_(novel) https://en.wikipedia.org/wiki/Ulysses_(novel) but with a feeling of dread, as the interpolations pile upon one another.. you can feel the thinness of the construction, as you are several layers out over ground truth on a thin tight rope.
Animats 2y ago
That would be a huge result it if works. The paper: "Our approach is to train a classifier that outputs the probability that a statement is truthful, based on the hidden layer activations of the LLM as it reads or generates the statement. Experiments demonstrate that given a set of test sentences, of which half are true and half false, our trained classifier achieves an average of 71% to 83% accuracy labeling which sentences are true versus false, depending on the LLM base model."[1] It's not entirely clear what they measured and whether what they measured is useful. Is this a major result? [1] https://arxiv.org/pdf/2304.13734 https://arxiv.org/pdf/2304.13734
- jncfhnb 2y ago
  Looks like they tested against statements that were obviously false and written by humans; not hallucinated statements created by the model that happened to be false. So, no, it doesn’t seem very interesting at all to me.
  - Animats 2y ago
    That's what I thought I read, too. Not too useful. We really need LLMs with a confidence metric. This doesn't seem to be it.

InDubioProRubio 2y ago

The projects themselves specified that lying is okay. Output at all costs, at all parameters, as interpolating is a feature, not a bug. It could just error out at loss of certainty, but then you would stand there, having nothing most of the time.

ithkuil 2y ago
Interpolating is an awesome feature, very practical: summarization, style transfer. Instruction following is also useful and unlocks another tier of features. The fuzziness of the model behaviour plays a useful role because natural language is imprecise. The ability to interpolate this remains useful. Once machines understand natural language our own psychology kicks in and assigns properties like "wisdom" and "knowledge" to agents that can "talk well". This may lead us to expect something from tools that haven't been designed to perform a given task. For example our current LLMs are not very good at precise information retrieval because they haven't been designed for that. They happen to be able to perform information retrieval as a side product of the mechanism that teaches them how to speak in the first place. The raw output of the broca area in our brains is probably not something you want to connect directly to the mouth without mediation
- InDubioProRubio 2y ago
  We need training exercises that teach us, how to think like a llm. Less like a human, more like a free association machine, dreaming a specific association flow, starting from a input. https://en.wikipedia.org/wiki/Ulysses_(novel) https://en.wikipedia.org/wiki/Ulysses_(novel) but with a feeling of dread, as the interpolations pile upon one another.. you can feel the thinness of the construction, as you are several layers out over ground truth on a thin tight rope.

Animats 2y ago

That would be a huge result it if works. The paper: "Our approach is to train a classifier that outputs the probability that a statement is truthful, based on the hidden layer activations of the LLM as it reads or generates the statement. Experiments demonstrate that given a set of test sentences, of which half are true and half false, our trained classifier achieves an average of 71% to 83% accuracy labeling which sentences are true versus false, depending on the LLM base model."[1] It's not entirely clear what they measured and whether what they measured is useful. Is this a major result? [1] https://arxiv.org/pdf/2304.13734 https://arxiv.org/pdf/2304.13734

jncfhnb 2y ago
Looks like they tested against statements that were obviously false and written by humans; not hallucinated statements created by the model that happened to be false. So, no, it doesn’t seem very interesting at all to me.
- Animats 2y ago
  That's what I thought I read, too. Not too useful. We really need LLMs with a confidence metric. This doesn't seem to be it.