3 ms·

I like the definition of bug as “unexpected behavior”. So this isn’t a bug when it comes the underlying service. But for ChatGPT, a consumer-facing web app that

• lcnPylGDnU4H9OF 4y ago
  It sounds like this is unexpected behavior, even from the perspective of those developing at the lowest level in these models. From the essay: > What I find fascinating about this is that these extremely problematic behaviours are not the system working as intended: they are bugs! And we haven’t yet found a reliable way to fix them. Right below that is this link: https://arxiv.org/abs/2212.09251 https://arxiv.org/abs/2212.09251. From the introduction on that page: > As language models (LMs) scale, they develop many novel behaviors, good and bad, exacerbating the need to evaluate how they behave. Especially with a definition as broad as "unexpected behavior", these "novel behaviors" seem to fit. But even without that: > We also find some of the first examples of inverse scaling in RL from Human Feedback (RLHF), where more RLHF makes LMs worse. For example, RLHF makes LMs express stronger political views ... and a greater desire to avoid shut down.
  • int_19h 4y ago
    Whether it's a bug or not depends on the application, basically. You want the model to hallucinate things like that if it's writing a fiction book, for example.