6 ms·

It's wild that people post papers that they haven't read or don't understand because the headline supports some view they have. To wit, in your first link it s

It's wild that people post papers that they haven't read or don't understand because the headline supports some view they have. To wit, in your first link it seems the figure is just showing the trivial fact that the model is trained on the MMLU dataset (and after RLHF it is no longer optimized for that). The second link main claim seems to be contradicted by their Figure 12 left panel which shows ~0 correlation between model-predicted and actual truth. I'm not going to bother going through the rest. I don't yet understand exactly what they are doing in the OP's article but I suspect it also suffers from serious problems.

famouswaffles 2y ago
>It's wild that people post papers that they haven't read or don't understand because the headline supports some view they have. It's related research either way. And I did read them. I think there's probably issues with the methodology of 4 but it's there anyway because it's interesting research that is related and is not without merit. >The second link main claim seems to be contradicted by their Figure 12 left panel which shows ~0 correlation between model-predicted and actual truth. The panel is pretty weak on correlation but it's quite clearly also not the only thing that supports that particular claim neither does it contradict it. >I'm not going to bother going through the rest. Ok? That's fine >I don't yet understand exactly what they are doing in the OP's article but I suspect it also suffers from serious problems. You are free to assume anything you want.
- foobarqux 2y ago
  > The panel is pretty weak on correlation but it's quite clearly also not the only thing that supports that particular claim neither does it contradict it. It very clearly contradicts it: There is no correlation between the predicted truth value and the actual truth value. That is the essence of the claim. If you had read and understood the paper you would be able to specifically detail why that isn't so rather than say vaguely that "it is not the only thing that supports that particular claim".
  - famouswaffles 2y ago
    >If you had read and understood the paper you would be able to specifically detail why that isn't so rather than say vaguely that "it is not the only thing that supports that particular claim". Not every internet conversation need end in a big debate. You've been pretty rude and i'd just rather not bother. You also seem to have a lot to say on how much people actually read papers but your first response also took like 5 minutes. I'm sorry but you can't say you've read even one of those in that time. Why would i engage with someone being intellectually dishonest?
    - foobarqux 2y ago
      > I guess i understand seeing as you couldn't have read the paper in the 5 minutes it took for your response. You've posted the papers multiple times over the last few months, so no I did not read them in the last five minutes though you could in fact find both of the very basic problems I cited in that amount of time.
      - famouswaffles 2y ago
        If you've come upon the previous posts organically, why not address it then? and Why act like it's the first time here ? I'm even less willing to engage.
        foobarqux 2y ago
        Because it's pointless to reply to a comment days after it was made or after engagement with the post has died down. All of this is a convenient misdirection for not having read and understood the papers you keep posting because you like the headlines.
        famouswaffles 2y ago
        Ok. I've addressed it now.
    - godelski 2y ago
mattnewton 2y ago
Please keep the conversation in good faith.
bee_rider 2y ago
They just posted a list of articles, and said that they were related. What view do you think they have, that these papers support? They haven’t expressed a view as far as I can see… Maybe you’ve inferred some view based on the names of the titles, but in that case you seem to be falling afoul of your own complaint?
- foobarqux 2y ago
  Search the poster's history for those links where their view is explicitly expressed.
  - erikerikson 2y ago
    Responding to a poster's history of posts rather than the post you are responding to seems problematic.
    - tsimionescu 2y ago
      If you have discussed those things previously with the poster, I don't agree. If you were to go digging through their history only to respond to the current comment, that's more debatable. But, we're supposed to assume good faith here on HN, so I would take the first explanation.
      - erikerikson 2y ago
        In this case the poster seems to have projected opinions on to a post where none were expressed. That seems problematic regardless of how they came to associate the opinions with their respondent. Maybe the poster they responded to still hold the projected opinions, perhaps that poster abandoned the projected opinions, or perhaps they thought the projected opinions distracting and resultantly chose not to share. If I am wrong or not useful in my posts, I would hope to be allowed to remove what was wrong and/or not useful without losing my standing to share the accurate, useful things. Anything else seems like residual punishment outside the appropriate context.
      - 2y ago
famouswaffles 2y ago
>The second link main claim seems to be contradicted by their Figure 12 left panel which shows ~0 correlation between model-predicted and actual truth. The claim in the abstract is: """We first show that larger models are well-calibrated on diverse multiple choice and true/false questions when they are provided in the right format. Next, we investigate whether models can be trained to predict "P(IK)", the probability that "I know" the answer to a question, without reference to any particular proposed answer. Models perform well at predicting P(IK) and partially generalize across tasks, though they struggle with calibration of P(IK) on new tasks.""" The plot is much denser in the origin and top right. How is that 0 correlation ? Depending on the number of their held-out test set, that could be pretty strong correlation even. And how does that contradict the claims they've made, especially on calibration (Fig 13 down) ?
- foobarqux 2y ago
  Figure 13 right panel also shows there isn't a y=x relationship on out-of-sample tests. First we agree by observation that outside of the top-right and bottom-left corners there isn't any meaningful relationship in the data, regardless of what the numerical value of the correlation is. Second, in those corners it is not clear to me what the relationship is but it looks flattish (i.e. if the ground truth is ~0 then the model-guess-for-truth could be anywhere from 0 to 0.5). This is also consistent with the general behavior displayed in figure 13. If you have some other interpretation of the data you should lay it out. The authors certainly did not do that. edit: By the way there are people working on a re-sampling algorithm based on the entropy and variance of the output logits called entropix: if the output probabilities for the next token are spread evenly for example (and not have overwhelming probability for a single token) they prompt for additional clarification. They don't really claim anything like the model "knows" whether it's wrong but they say it improves performance.
  - famouswaffles 2y ago
    >Figure 13 right panel also shows there isn't a y=x relationship on out-of-sample tests. A y=x relationship is not necessary for meaningful correlation and the abstract is quite clear on out of sample performance either way. >Second, in those corners it is not clear to me what the relationship is but it looks flattish (i.e. if the ground truth is ~0 then the model-guess-for-truth could be anywhere from 0 to 0.5). The upper bound for guess-for-truth is not as important as the frequency. Yes it could guess 0.5 for 0 but how often compared to reasonable numbers? A test set on TriviaQA could well be thousands of questions. >edit: By the way there are people working on a re-sampling algorithm based on the entropy and variance of the output logits called entropix I know about entropix. It hinges strongly on the model's representations. If it works, then choosing to call it "knowing" or not is just semantics.

famouswaffles 2y ago

>It's wild that people post papers that they haven't read or don't understand because the headline supports some view they have. It's related research either way. And I did read them. I think there's probably issues with the methodology of 4 but it's there anyway because it's interesting research that is related and is not without merit. >The second link main claim seems to be contradicted by their Figure 12 left panel which shows ~0 correlation between model-predicted and actual truth. The panel is pretty weak on correlation but it's quite clearly also not the only thing that supports that particular claim neither does it contradict it. >I'm not going to bother going through the rest. Ok? That's fine >I don't yet understand exactly what they are doing in the OP's article but I suspect it also suffers from serious problems. You are free to assume anything you want.

foobarqux 2y ago
> The panel is pretty weak on correlation but it's quite clearly also not the only thing that supports that particular claim neither does it contradict it. It very clearly contradicts it: There is no correlation between the predicted truth value and the actual truth value. That is the essence of the claim. If you had read and understood the paper you would be able to specifically detail why that isn't so rather than say vaguely that "it is not the only thing that supports that particular claim".
- famouswaffles 2y ago
  >If you had read and understood the paper you would be able to specifically detail why that isn't so rather than say vaguely that "it is not the only thing that supports that particular claim". Not every internet conversation need end in a big debate. You've been pretty rude and i'd just rather not bother. You also seem to have a lot to say on how much people actually read papers but your first response also took like 5 minutes. I'm sorry but you can't say you've read even one of those in that time. Why would i engage with someone being intellectually dishonest?
  - foobarqux 2y ago
    > I guess i understand seeing as you couldn't have read the paper in the 5 minutes it took for your response. You've posted the papers multiple times over the last few months, so no I did not read them in the last five minutes though you could in fact find both of the very basic problems I cited in that amount of time.
    - famouswaffles 2y ago
      If you've come upon the previous posts organically, why not address it then? and Why act like it's the first time here ? I'm even less willing to engage.
      - foobarqux 2y ago
        Because it's pointless to reply to a comment days after it was made or after engagement with the post has died down. All of this is a convenient misdirection for not having read and understood the papers you keep posting because you like the headlines.
        famouswaffles 2y ago
        Ok. I've addressed it now.
  - godelski 2y ago

mattnewton 2y ago

Please keep the conversation in good faith.

bee_rider 2y ago

They just posted a list of articles, and said that they were related. What view do you think they have, that these papers support? They haven’t expressed a view as far as I can see… Maybe you’ve inferred some view based on the names of the titles, but in that case you seem to be falling afoul of your own complaint?

foobarqux 2y ago
Search the poster's history for those links where their view is explicitly expressed.
- erikerikson 2y ago
  Responding to a poster's history of posts rather than the post you are responding to seems problematic.
  - tsimionescu 2y ago
    If you have discussed those things previously with the poster, I don't agree. If you were to go digging through their history only to respond to the current comment, that's more debatable. But, we're supposed to assume good faith here on HN, so I would take the first explanation.
    - erikerikson 2y ago
      In this case the poster seems to have projected opinions on to a post where none were expressed. That seems problematic regardless of how they came to associate the opinions with their respondent. Maybe the poster they responded to still hold the projected opinions, perhaps that poster abandoned the projected opinions, or perhaps they thought the projected opinions distracting and resultantly chose not to share. If I am wrong or not useful in my posts, I would hope to be allowed to remove what was wrong and/or not useful without losing my standing to share the accurate, useful things. Anything else seems like residual punishment outside the appropriate context.
    - 2y ago

famouswaffles 2y ago

>The second link main claim seems to be contradicted by their Figure 12 left panel which shows ~0 correlation between model-predicted and actual truth. The claim in the abstract is: """We first show that larger models are well-calibrated on diverse multiple choice and true/false questions when they are provided in the right format. Next, we investigate whether models can be trained to predict "P(IK)", the probability that "I know" the answer to a question, without reference to any particular proposed answer. Models perform well at predicting P(IK) and partially generalize across tasks, though they struggle with calibration of P(IK) on new tasks.""" The plot is much denser in the origin and top right. How is that 0 correlation ? Depending on the number of their held-out test set, that could be pretty strong correlation even. And how does that contradict the claims they've made, especially on calibration (Fig 13 down) ?

foobarqux 2y ago
Figure 13 right panel also shows there isn't a y=x relationship on out-of-sample tests. First we agree by observation that outside of the top-right and bottom-left corners there isn't any meaningful relationship in the data, regardless of what the numerical value of the correlation is. Second, in those corners it is not clear to me what the relationship is but it looks flattish (i.e. if the ground truth is ~0 then the model-guess-for-truth could be anywhere from 0 to 0.5). This is also consistent with the general behavior displayed in figure 13. If you have some other interpretation of the data you should lay it out. The authors certainly did not do that. edit: By the way there are people working on a re-sampling algorithm based on the entropy and variance of the output logits called entropix: if the output probabilities for the next token are spread evenly for example (and not have overwhelming probability for a single token) they prompt for additional clarification. They don't really claim anything like the model "knows" whether it's wrong but they say it improves performance.
- famouswaffles 2y ago
  >Figure 13 right panel also shows there isn't a y=x relationship on out-of-sample tests. A y=x relationship is not necessary for meaningful correlation and the abstract is quite clear on out of sample performance either way. >Second, in those corners it is not clear to me what the relationship is but it looks flattish (i.e. if the ground truth is ~0 then the model-guess-for-truth could be anywhere from 0 to 0.5). The upper bound for guess-for-truth is not as important as the frequency. Yes it could guess 0.5 for 0 but how often compared to reasonable numbers? A test set on TriviaQA could well be thousands of questions. >edit: By the way there are people working on a re-sampling algorithm based on the entropy and variance of the output logits called entropix I know about entropix. It hinges strongly on the model's representations. If it works, then choosing to call it "knowing" or not is just semantics.