3 ms·

> Give the same task to GPT4 and it proves it’s modeled the concept appropriately because it can pull the right words out of the ether. I take your point, but

by xpe 3y ago

> Give the same task to GPT4 and it proves it’s modeled the concept appropriately because it can pull the right words out of the ether. I take your point, but we should be careful when using phrased like “modeled the concept appropriately” here. That implies correctness. There are many ‘concept models’ that could work well enough. Unless we can inspect the concept model directly (inside the LLM, somehow!) we are left to reason about how the concept is represented via statistics over a set of interactions. So, if, over the course of many interactions, it seems that a model is saying reasonable things we can have more confidence that it has achieved a concept model that is “good enough”. But such an assessment is always in relationship to the test protocol. Claims to generality should be done carefully. In some ways, my standard here is probably not that different than a rigorous human educational assessment. I’m not talking about standardized tests, I’m talking about adversarial challenges like one would hope to see during a Ph.D. defense.

teaearlgraycold 3y ago
Yeah I’m setting the bar at the level of a child’s understanding. Because smaller models are well below that, just picking tokens at random. That’s how misunderstanding manifests with LLMs. They don’t say “I don’t know”, they just go off the rails. The quality bar for different use cases might be such that an understanding of a system on par with a 6 year old is isn’t useful. But right now we clearly have two clusters and I’m drawing the boundary where it’s most useful.
nl 3y ago
> I take your point, but we should be careful when using phrased like “modeled the concept appropriately” here. That implies correctness. All models are wrong, but some are useful. Perhaps we should say "modeled the concept usefully". > In some ways, my standard here is probably not that different than a rigorous human educational assessment. I’m not talking about standardized tests, I’m talking about adversarial challenges like one would hope to see during a Ph.D. defense. Usefulness is very contextual obviously. Newton's laws aren't entirely correct but are still useful.

by xpe 3y ago

teaearlgraycold 3y ago

Yeah I’m setting the bar at the level of a child’s understanding. Because smaller models are well below that, just picking tokens at random. That’s how misunderstanding manifests with LLMs. They don’t say “I don’t know”, they just go off the rails. The quality bar for different use cases might be such that an understanding of a system on par with a 6 year old is isn’t useful. But right now we clearly have two clusters and I’m drawing the boundary where it’s most useful.

nl 3y ago

> I take your point, but we should be careful when using phrased like “modeled the concept appropriately” here. That implies correctness. All models are wrong, but some are useful. Perhaps we should say "modeled the concept usefully". > In some ways, my standard here is probably not that different than a rigorous human educational assessment. I’m not talking about standardized tests, I’m talking about adversarial challenges like one would hope to see during a Ph.D. defense. Usefulness is very contextual obviously. Newton's laws aren't entirely correct but are still useful.