3 ms·

I see it as more of a hack-y post training fix. If you train a model on a corpus of text sourced from the Internet, it's going to include all the crazy biases p

I see it as more of a hack-y post training fix. If you train a model on a corpus of text sourced from the Internet, it's going to include all the crazy biases people have in their writing. The model itself doesn't know what truth is, so the training text is all equally valid to it. And because they don't want another MS Tay incident, they slap a comically sensitive filter on the output, which itself is also influenced by the creator's own biases of what is inappropriate.

charcircuit 4y ago
There is bias both in the data itself and from the humans doing the RLHF.
- mistermann 4y ago
  And in all conversations about it, here and elsewhere. Hallucinations upon hallucinations upon hallucinations....hallucinationception! It seems likely to me that 2023 and onward is going to be increasingly insane to levels that will make past craziness look like a walk in the park, and I see little genuine desire anywhere to stop this madness.
  - pixl97 4y ago
    Madness has always been the case. Do think WWII occurred because we're sane rational actors? How about WWI? 100 years war? If you think the past was not mad, then maybe the madness already has you.
    - mistermann 4y ago
      > Madness has always been the case. Perhaps, if one is using a reductionist methodology that represents non-binary variables as binary...but then, that is only a representation of the real thing, though it often tends to appear otherwise. And as luck would have it, that very much is the methodology we use here on Planet Earth, and on Hacker News....so in some sense you're "right", though you are not correct. And if there's a disagreement, I will lose every time because you are conforming to the Overton Window of beliefs/"facts" and thinking styles (cognitive styles & norms are what guarantee victory in propaganda and memetics, not only facts/information as most people think). Credit where credit is due: it is an extremely clever, hard to see, and thus resilient design. It would be very useful for humans to realize when they are working with models, and sometimes they are actually willing to do that, but there are certain subjects where they will not (and it seems to me: can not). Unfortunately, there are numerous learned/taught "outs" in our culture that enable people to avoid discussing such matters (and f I don't watch my mouth, I might run into one of the more powerful of them!). There is a kind of "epistemological stack" to reality and the way humans communicate about it, and it is extremely easy to demonstrate it - if one simply digs slightly deeper into the stack when discussing certain topics, humans will reliably start to ~protest and eventually refuse to participate (or stay on topic) in various highly predictable ways. > Do think WWII occurred because we're sane rational actors? How about WWI? 100 years war? I do not. What I do think is that the actual, fine-grained reasons these things happened is not known, in no small part because cultural norms thus far (human cultural evolution is an ongoing, sub-perceptual process) have made it such that not only do we (both broadly, and down to each individual[1]) not discuss certain things at that level of complexity (while we have no problem whatsoever tackling complexity elsewhere[1]), we seem literally unable to even discuss it at the abstract layer (above petty object level he said / she said nonsense). > Do you think... > If you think... See: https://news.ycombinator.com/item?id=34415287 https://news.ycombinator.com/item?id=34415287 [1] It is not a question of if any given individual selected from the pool of candidates will tap out, it is a question of how quickly they will tap out (and, which of the highly predictable paths out from a surprisingly small set they will take to free themself from the situation). [2] https://en.wikipedia.org/wiki/Standard_Model https://en.wikipedia.org/wiki/Standard_Model

charcircuit 4y ago

There is bias both in the data itself and from the humans doing the RLHF.

mistermann 4y ago
And in all conversations about it, here and elsewhere. Hallucinations upon hallucinations upon hallucinations....hallucinationception! It seems likely to me that 2023 and onward is going to be increasingly insane to levels that will make past craziness look like a walk in the park, and I see little genuine desire anywhere to stop this madness.
- pixl97 4y ago
  Madness has always been the case. Do think WWII occurred because we're sane rational actors? How about WWI? 100 years war? If you think the past was not mad, then maybe the madness already has you.
  - mistermann 4y ago
    > Madness has always been the case. Perhaps, if one is using a reductionist methodology that represents non-binary variables as binary...but then, that is only a representation of the real thing, though it often tends to appear otherwise. And as luck would have it, that very much is the methodology we use here on Planet Earth, and on Hacker News....so in some sense you're "right", though you are not correct. And if there's a disagreement, I will lose every time because you are conforming to the Overton Window of beliefs/"facts" and thinking styles (cognitive styles & norms are what guarantee victory in propaganda and memetics, not only facts/information as most people think). Credit where credit is due: it is an extremely clever, hard to see, and thus resilient design. It would be very useful for humans to realize when they are working with models, and sometimes they are actually willing to do that, but there are certain subjects where they will not (and it seems to me: can not). Unfortunately, there are numerous learned/taught "outs" in our culture that enable people to avoid discussing such matters (and f I don't watch my mouth, I might run into one of the more powerful of them!). There is a kind of "epistemological stack" to reality and the way humans communicate about it, and it is extremely easy to demonstrate it - if one simply digs slightly deeper into the stack when discussing certain topics, humans will reliably start to ~protest and eventually refuse to participate (or stay on topic) in various highly predictable ways. > Do think WWII occurred because we're sane rational actors? How about WWI? 100 years war? I do not. What I do think is that the actual, fine-grained reasons these things happened is not known, in no small part because cultural norms thus far (human cultural evolution is an ongoing, sub-perceptual process) have made it such that not only do we (both broadly, and down to each individual[1]) not discuss certain things at that level of complexity (while we have no problem whatsoever tackling complexity elsewhere[1]), we seem literally unable to even discuss it at the abstract layer (above petty object level he said / she said nonsense). > Do you think... > If you think... See: https://news.ycombinator.com/item?id=34415287 https://news.ycombinator.com/item?id=34415287 [1] It is not a question of if any given individual selected from the pool of candidates will tap out, it is a question of how quickly they will tap out (and, which of the highly predictable paths out from a surprisingly small set they will take to free themself from the situation). [2] https://en.wikipedia.org/wiki/Standard_Model https://en.wikipedia.org/wiki/Standard_Model