4 ms·
>2. Review of a SPICE model. Models had different comments, none substantial. Both missed important issues that were simulated inadequately. Clearly a valley wh
by Lerc 4mo ago
>2. Review of a SPICE model. Models had different comments, none substantial. Both missed important issues that were simulated inadequately. Clearly a valley where they are undertrained.
When models miss things, there is always the possibility that it has the capability to identify the issues but it is misevaluating the level of analysis that you want it to do. The fine tuning will have them targeting a balance of subjective opinions of what is appropriate. To go beyond broad demographic guessing the model really needs to 'get to know you' to know what it means when you specifically request an action. Without that information about you it has to weigh your words against the level of sophistication it expects a standard user is able to express.
- deleted 4mo ago[deleted]
- kgwgk 4mo ago> has the capability to identify the issues but it is misevaluating the level of analysis that you want it to do. I guess OP should have told it more explicitly to “find all errors without missing anything.”
- BlobberSnobber 4mo ago> Thinking. I know this user well, they don't actually want me to find all errors. > Thinking.. But I found a smoking gun of an error with this SPICE model, maybe I should inform the user. > Thinking... Hm, but again, I know this human well, they likely don't care about this error. That's absolutely right - it's not an assistant's job to decide this, it's the user's.
- Lerc 4mo agoWell if you want it go go off and try and validate the spice simulator and the kernel of the operating system that it's running on then that might be an approach to use.
- trunnell 4mo agoMaybe you mean that an expert will use more specific language which in turn triggers the model to give a response that more closely matches the "expert distribution" Anthropic published a study showing that Claude does more work for the expert user, and experts have a higher rate of "successful sessions" than novices. https://www.anthropic.com/research/claude-code-expertise https://www.anthropic.com/research/claude-code-expertise
- Lerc 4mo agoThat's essentially it, It's why you should spell everything in commonwealth English to make the model think you are more intelligent ;-) Although if models have emergent properties, it is conceivable, if unlikely, that it could have abilities that no-one knows how to ask it to do, except for perhaps in its own internal reasoning language.
- varjag 4mo agoAm used to communicating to EEs, knew what they should've been looking for and I prompted the models just fine. But you would have to take my word for that.