3 ms·

I've got some personal litmus tests: 1) Syntax/semantic split. Can the person accept that a function called "multiplyBy5(a,b) { return a+b }" doesn't actually

I've got some personal litmus tests: 1) Syntax/semantic split. Can the person accept that a function called "multiplyBy5(a,b) { return a+b }" doesn't actually multiply by five, but adds the numbers? 2) PR speak: Does the person recognize that public relation speak is usually intentionally misleading, as in "the Russian Ministry of Defense said that a fire [onboard the Moskva] had caused ammunition to explode" (obviously caused by an Ukrainian missile and not an accidental fire, even though that's what's implied.) [0] 3) They're, their, there: There easy to tell apart, since they're meaning is so different. /s 4) Viewpoints: Can this person understand and articulate viewpoints that they consider "wrong" or simply don't hold themselves? 5) (new) LLM introspection: Does the person understand that LLMs have no secret understanding of themselves? An LLM like "Grok" doesn't actually understand "Grok" better than Gemini understands "Grok" - apart from minor differences in model strength maybe. [0] https://en.wikipedia.org/wiki/Sinking_of_the_Moskva https://en.wikipedia.org/wiki/Sinking_of_the_Moskva

doxeddaily 6mo ago
Not bad litmus tests. And yes a lot of idiots seem to fail at steel manning. I mean if you can't steel man your opponent what are you even doing?
D-Coder 6mo ago
I've heard that a non-Mensan asked a Mensan what it's like at a Mensa event. They replied, "If I have to explain something during a conversation, I only have to explain it _once_."
fuzzzerd 6mo ago
Are there people that would legitimately argue point 1? If you are only looking at the call site, sure that could be confusing, but if you are looking at the definition as provided in your post, surely anyone that is able understand the concept of a function can see the problem? I'm not arguing they don't exist, sure they do, but I'm confused as to how you came up with it as a litmus test? Is it that common? Surely we can agree in a real scenario renaming it (or fixing implementation to match name) is likely appropriate, but to completely miss the error? Hope this comes across as curiousity, because I am curious about this one from your list in particular.
- krackers 6mo ago
  I think it's similar to the case of counterfactuals, hypotheticals, or steelmaning and how well you can handle them. ("Can you accept that there can be a function named multiplyBy5 that does something else instead"). But I think if someone already is comfortable with working with abstractions such as "function" the thing is trivial, so it's a bit of a weird litmus test.
  - fuzzzerd 6mo ago
    > "Can you accept that there can be a function named multiplyBy5 that does something else instead"). I think anyone that can understand a function can understand this, but one might not be happy accepting it's the case, and endeavor to change it. I think it can be easy to lose sight of that distinction, and eagerness to fix it can be confused with not accepting it could be, but is also probably wrong.
- 650 6mo ago
  I've heard the parable, "If you didn't eat breakfast or lunch yesterday, how would you feel around 3pm?", a common response is apparently "but I did eat lunch and breakfast yesterday".
krackers 6mo ago
>LLMs have no secret understanding of themselves What do you mean by "themselves" here? Grok is RL'd to behave like a Grok, so it trivially knows the qualities that define Grok better than Gemini does, which can only go by second hand sources.
- yetihehe 6mo ago
  > so it trivially knows the qualities that define Grok How does it know? Where did he get that knowledge from? Did they train Grok, check it's qualities and included them in next training set? Was his source code and summarization of weights included, or maybe he has access to them for "introspection"?

doxeddaily 6mo ago

Not bad litmus tests. And yes a lot of idiots seem to fail at steel manning. I mean if you can't steel man your opponent what are you even doing?

D-Coder 6mo ago

I've heard that a non-Mensan asked a Mensan what it's like at a Mensa event. They replied, "If I have to explain something during a conversation, I only have to explain it _once_."

fuzzzerd 6mo ago

Are there people that would legitimately argue point 1? If you are only looking at the call site, sure that could be confusing, but if you are looking at the definition as provided in your post, surely anyone that is able understand the concept of a function can see the problem? I'm not arguing they don't exist, sure they do, but I'm confused as to how you came up with it as a litmus test? Is it that common? Surely we can agree in a real scenario renaming it (or fixing implementation to match name) is likely appropriate, but to completely miss the error? Hope this comes across as curiousity, because I am curious about this one from your list in particular.

krackers 6mo ago
I think it's similar to the case of counterfactuals, hypotheticals, or steelmaning and how well you can handle them. ("Can you accept that there can be a function named multiplyBy5 that does something else instead"). But I think if someone already is comfortable with working with abstractions such as "function" the thing is trivial, so it's a bit of a weird litmus test.
- fuzzzerd 6mo ago
  > "Can you accept that there can be a function named multiplyBy5 that does something else instead"). I think anyone that can understand a function can understand this, but one might not be happy accepting it's the case, and endeavor to change it. I think it can be easy to lose sight of that distinction, and eagerness to fix it can be confused with not accepting it could be, but is also probably wrong.
650 6mo ago
I've heard the parable, "If you didn't eat breakfast or lunch yesterday, how would you feel around 3pm?", a common response is apparently "but I did eat lunch and breakfast yesterday".

krackers 6mo ago

>LLMs have no secret understanding of themselves What do you mean by "themselves" here? Grok is RL'd to behave like a Grok, so it trivially knows the qualities that define Grok better than Gemini does, which can only go by second hand sources.

yetihehe 6mo ago
> so it trivially knows the qualities that define Grok How does it know? Where did he get that knowledge from? Did they train Grok, check it's qualities and included them in next training set? Was his source code and summarization of weights included, or maybe he has access to them for "introspection"?