3 ms·

Is there a score for internal consistency? I dunno, maybe have another LLM extract structure into some kind of a logic language and score how many assertions ar

by baq 2y ago

Is there a score for internal consistency? I dunno, maybe have another LLM extract structure into some kind of a logic language and score how many assertions are incompatible? Or to a probabilistic language and do some Bayesian analysis?

mentalgear 2y ago
I was thinking the same about consistency, which is useful for many contexts, but if you let another LLM do the extraction, you are again at the mercy of that LLMs quality/hallucinations. Of course mitigation strategies were using many different LLMs and comparing results (voting) or using a highly trained/specialised model for only entity / context extraction. An interesting benchmark would be when those extraction techniques/models would exceed what a human professional is able to do.
- baq 2y ago
  > you are again at the mercy of that LLMs quality/hallucinations. I guess I'm fine with a tall error bar - better than nothing. > An interesting benchmark would be when those extraction techniques/models would exceed what a human professional is able to do. Slashdot-style +1 Funny, -1 Inconsistent for RLHF maybe...?
- Tostino 2y ago
  Another that I don't see enough is just resampling and showing the human more 'final answers' if they are easy and quick to check for correctness.
112233 2y ago
And where that another LLM will get the capability to judge consistency? Had fun recently prompting different models with "a story about everyday life of <some stereotypical character>". Absolutely no awareness about anything spatial - what is in which place, who is where, body positions. Like, my favorite, person coming home and removing socks to reveal fluffy slippers. Will the consistency checker catch that. (edit) just asked GPT-4o to make a photo of hands tying two ropes (my favorite way to see how bad actually gen-ai is) - yup, it has an odd number of ropes coming out of the knot. That is state of art supposedly.
- baq 2y ago
  > And where that another LLM will get the capability to judge consistency? Had fun recently prompting different models with "a story about everyday life of <some stereotypical character>". > Absolutely no awareness about anything spatial - what is in which place, who is where, body positions. These are some things I'd like to at least attempt to quantify - LLMs are better at translation than thinking[0], so the idea was to translate the slop into something that can be then analyzed by a non-LLM tool. E.g. translate the story into a prolog fact database and generate a couple queries for each paragraph/chapter/combination of these. Rough idea, just something I haven't seen done. [0] don't @ me
- thaumasiotes 2y ago
  > Like, my favorite, person coming home and removing socks to reveal fluffy slippers. Will the consistency checker catch that. I could see this working as comedy, though it'd be better on film than in print. Either way you'd have to establish a very specific tone in the surrounding material.
  - 112233 2y ago
    This comedy is a good comedy while I am goofing around with "make a story". However, this wordpuker tech is positioned at solving strategic problems, writing software and (something equally serious). It has no mechanism to build a world model "in head" while it generinates. chess pieces teleport on board. people hug through doors. any exercise is a body horror. do you think this tech deals with invariants in software algorithms any better? I mean, haha but not funny.
sam-paech 2y ago
Not internal consistency exactly, but there are criteria checking how well the chapter plan was followed (which is all the way up at the top of the context window). This is done per chapter, and the score trendline is what you see in the "degradation" column. I would say this could be a reasonable proxy for internal consistency since it is measuring more or less the same ability, i.e. how well it's keeping track of details as context window increases.
kristianp 2y ago
There was a recent post on here about LLMs making mermaid diagrams for character relationships. That could be analysed by another model for evals.

by baq 2y ago

mentalgear 2y ago

I was thinking the same about consistency, which is useful for many contexts, but if you let another LLM do the extraction, you are again at the mercy of that LLMs quality/hallucinations. Of course mitigation strategies were using many different LLMs and comparing results (voting) or using a highly trained/specialised model for only entity / context extraction. An interesting benchmark would be when those extraction techniques/models would exceed what a human professional is able to do.

baq 2y ago
> you are again at the mercy of that LLMs quality/hallucinations. I guess I'm fine with a tall error bar - better than nothing. > An interesting benchmark would be when those extraction techniques/models would exceed what a human professional is able to do. Slashdot-style +1 Funny, -1 Inconsistent for RLHF maybe...?
Tostino 2y ago
Another that I don't see enough is just resampling and showing the human more 'final answers' if they are easy and quick to check for correctness.

112233 2y ago

And where that another LLM will get the capability to judge consistency? Had fun recently prompting different models with "a story about everyday life of <some stereotypical character>". Absolutely no awareness about anything spatial - what is in which place, who is where, body positions. Like, my favorite, person coming home and removing socks to reveal fluffy slippers. Will the consistency checker catch that. (edit) just asked GPT-4o to make a photo of hands tying two ropes (my favorite way to see how bad actually gen-ai is) - yup, it has an odd number of ropes coming out of the knot. That is state of art supposedly.

baq 2y ago
> And where that another LLM will get the capability to judge consistency? Had fun recently prompting different models with "a story about everyday life of <some stereotypical character>". > Absolutely no awareness about anything spatial - what is in which place, who is where, body positions. These are some things I'd like to at least attempt to quantify - LLMs are better at translation than thinking[0], so the idea was to translate the slop into something that can be then analyzed by a non-LLM tool. E.g. translate the story into a prolog fact database and generate a couple queries for each paragraph/chapter/combination of these. Rough idea, just something I haven't seen done. [0] don't @ me
thaumasiotes 2y ago
> Like, my favorite, person coming home and removing socks to reveal fluffy slippers. Will the consistency checker catch that. I could see this working as comedy, though it'd be better on film than in print. Either way you'd have to establish a very specific tone in the surrounding material.
- 112233 2y ago
  This comedy is a good comedy while I am goofing around with "make a story". However, this wordpuker tech is positioned at solving strategic problems, writing software and (something equally serious). It has no mechanism to build a world model "in head" while it generinates. chess pieces teleport on board. people hug through doors. any exercise is a body horror. do you think this tech deals with invariants in software algorithms any better? I mean, haha but not funny.

sam-paech 2y ago

Not internal consistency exactly, but there are criteria checking how well the chapter plan was followed (which is all the way up at the top of the context window). This is done per chapter, and the score trendline is what you see in the "degradation" column. I would say this could be a reasonable proxy for internal consistency since it is measuring more or less the same ability, i.e. how well it's keeping track of details as context window increases.

kristianp 2y ago

There was a recent post on here about LLMs making mermaid diagrams for character relationships. That could be analysed by another model for evals.