5 ms·

One thing I forgot to mention in the post which I think is kinda cool: at the NeurIPS Efficiency Challenge this year, where Tim Dettmers and I both did keynotes

by jph00 3y ago

One thing I forgot to mention in the post which I think is kinda cool: at the NeurIPS Efficiency Challenge this year, where Tim Dettmers and I both did keynotes, every single top-ranked entry used QLoRA! The challenge was to create the most accurate model on a single GPU in 24 hours. I think that is a great example of how important and useful QLoRA is. Maybe we should run a dual-GPU challenge next time not that multi-GPU is working...

TaylorAlexander 3y ago
Congrats on the release. On Twitter someone asked you if you would provide any risk assessment reflection, and you replied that the risk would be similar to the release of a new model of pen or pencil. That reply, while cute, isn’t accurate. A new pencil does not mean new capabilities for humanity, while the whole point of your release is that it does afford new capabilities and therefore new risks. The woman, a professor working on the societal impacts of AI, asked a straightforward question in apparent good faith [1]. Your reply did not seem to me to be in good faith. Can you explain the apparent disconnect? I’m less concerned as to whether or not you would release an assessment and more concerned at the dismissive attitude, especially towards a female professor studying societal impacts of AI, a fellow researcher. [1] https://x.com/dr_atoosa/status/1765874818283385071 https://x.com/dr_atoosa/status/1765874818283385071
- jph00 3y ago
  I believe my answer is accurate. I don't know on what basis you claim otherwise. This is my work, and I'm well placed to make the assessment. My response is based on reading a great many papers and doing many experiments over the last few years, and I am confident of it. By way of background: I studied the philosophy of ethics at university, I co-authored a book chapter on AI/ML ethics, my wife and I quit our jobs and worked for free for years entirely focused on trying to help society adapt to and benefit from AI, I wrote the actual article we're commenting on, and the article is about LLM fine-tuning -- a field I to some extent created by developing the ULMFiT algorithm. The person in question is, IIRC, at Governance AI, a group whose work I spent months studying in depth -- work which I believe is more likely to cause harm to society than to benefit it, as I explained here: https://www.fast.ai/posts/2023-11-07-dislightenment.html https://www.fast.ai/posts/2023-11-07-dislightenment.html
  - TaylorAlexander 3y ago
    [flagged]
    - d-z-m 3y ago
      Implicit in every reply you've given is the assumption that OP is treating the criticism from this researcher differently because she's a woman. Do you have any basis on which you're making this assumption? OP explained that they have substantive issues with the organization of which this researcher is a member.
      - TaylorAlexander 3y ago
        [flagged]
        rmbyrro 3y ago
        Then why's it relevant to keep mentioning it's a woman?
        d-z-m 3y ago
        > I can see why it would appear that I’m saying that, but that was not my intention. You are talking out of both sides of your mouth. In another comment on this same thread, you say this: > [...]women in the field are more readily dismissed, and I think they shouldn’t be. It’s a moment to check our internalized biases and make sure we’re operating in good faith. In your original comment you explicitly accuse the OP of operating in bad faith, presumably as a result of "internalized biases" as described above. How does this not add up to an assumption that OP treated the researcher differently because she's a woman? It is exactly what you are implying.
        Tomte 3y ago
        And you would have called a simple "no" dismissive, too…
    - razodactyl 3y ago
      I think it's easier to dismiss risk with this project as it allows democratised access to AI models and furthers research in the field. The ability to generate low-quality content has been available since long before LLM technology, additionally these 70B param models are just barely fitting into $10,000 worth of hardware (not accounting for M-series chips). The scaling issue with potential runaway AI can be excluded. The potential for virus writing / security exploitation perhaps but such risks are already present with existing models so this point too can be excluded. I'm not sure there's any inherent risk here compared to what's easily available with a considerably reduced amount of resource requirements. The write up here seems concerned with allowing independent and democratised research which is a greater benefit than concentrated efforts.
- imdsm 3y ago
  > especially towards a female professor studying societal impacts of AI > especially towards a professor studying societal impacts of AI I can't help but feel like you're trying to load this question with some expectation that sex/gender should change how people react and respond. It shouldn't, at all, positively or negatively. Everyone is human (for now).
  - TaylorAlexander 3y ago
    [flagged]
    - Borealid 3y ago
      I think you're saying that correcting a bias isn't itself applying bias. I think the poster to which you're responding is saying there wasn't any visible evidence of bias in the original behaviour. From one point of view, you're correcting a bias (in this case, one you suspect might exist), and you believe that isn't a bias. From another point of view, you're introducing a new bias (that definitely exists) to something that was previously (presumably) agnostic and neutral, by saying that a correction should only be applied when the question-asker meets certain criteria. Both points of view are legitimate. PERSONALLY I'd rather we not bring the "have you thought about how this might be discriminatory" bits into the converation unless there's at least some vague reason to think that it was, rather than presuming it's always discrimination when the genders and/or sexes and/or races line up a certain way. But that's because I think it's more important to debate the ideas in their pure form than to correct every instance of discrimination, and that's an arbitrary value judgement I made.
      - TaylorAlexander 3y ago
        [flagged]
- drittich 3y ago
  I completely get your take on this and don't disagree. IMO it reads as a flippant response to a potentially serious question, and I am going to assume this is unintentional, which does not mean it is unfair to call it out. Stepping back, this is the kind of discourse that Twitter can sometimes reinforce: short responses that, lacking context, can be interpreted in polarizing ways. After a bit of reading on the participants (because both of them are working in interesting areas), my belief is that the "pencil" response is actually shorthand for a whole set of beliefs that provide a lot of context and nuance to the discussion, but if you don't know that, it sounds like saying "AI is as dangerous as a new brand of chewing gum". In addition, without defining what risks we're talking about, it's really hard to know the scope the answer is addressing. E.g., societal risk due to AI in general? vs. say, the risks of affecting the quality an existing model with fine-tuning? So, I am chalking this up to a misunderstanding due to the limitations of the platform as well as the short-hand used in the response. And I could be completely wrong :)
- Frantanata 3y ago
  You are reading a lot into a tweet....
3abiton 3y ago
Are any of the NIPS resources available online?
- notpublic 3y ago
  Tim Dettmers QLoRA https://nips.cc/virtual/2023/83963 https://nips.cc/virtual/2023/83963 More here: https://nips.cc/virtual/2023/competition/66594 https://nips.cc/virtual/2023/competition/66594
paul_mk1 3y ago
It's clear that QLoRA has opened up finetuning to a wider audience with limited compute, which is a good thing. One thing I've wondered about: what are the drawbacks to using QLoRA? For example if compute is not a limit, I'm guessing one should not use QLoRA and finetune in full precision instead? Afaik when a model is first quantized to nf4 (before finetuning begins), model performance is degraded from baseline (see https://x.com/Tim_Dettmers/status/1661482614811918338?s=20 https://x.com/Tim_Dettmers/status/1661482614811918338?s=20). Dettmers shows that after finetuning wrt the dataset, the result is as good as full precision. But afaik never explored the effects outside the finetuning data. Assuming the finetuning dataset is small, the model will just be the degraded nf4 version, right? Or perhaps finetuning will even skew the model in weird ways (trying to fix quantization errors). Anecdotally models finetuned wth QLoRA perform well. Does anyone have any papers or a careful analysis of this?

by jph00 3y ago

TaylorAlexander 3y ago

Congrats on the release. On Twitter someone asked you if you would provide any risk assessment reflection, and you replied that the risk would be similar to the release of a new model of pen or pencil. That reply, while cute, isn’t accurate. A new pencil does not mean new capabilities for humanity, while the whole point of your release is that it does afford new capabilities and therefore new risks. The woman, a professor working on the societal impacts of AI, asked a straightforward question in apparent good faith [1]. Your reply did not seem to me to be in good faith. Can you explain the apparent disconnect? I’m less concerned as to whether or not you would release an assessment and more concerned at the dismissive attitude, especially towards a female professor studying societal impacts of AI, a fellow researcher. [1] https://x.com/dr_atoosa/status/1765874818283385071 https://x.com/dr_atoosa/status/1765874818283385071

jph00 3y ago
I believe my answer is accurate. I don't know on what basis you claim otherwise. This is my work, and I'm well placed to make the assessment. My response is based on reading a great many papers and doing many experiments over the last few years, and I am confident of it. By way of background: I studied the philosophy of ethics at university, I co-authored a book chapter on AI/ML ethics, my wife and I quit our jobs and worked for free for years entirely focused on trying to help society adapt to and benefit from AI, I wrote the actual article we're commenting on, and the article is about LLM fine-tuning -- a field I to some extent created by developing the ULMFiT algorithm. The person in question is, IIRC, at Governance AI, a group whose work I spent months studying in depth -- work which I believe is more likely to cause harm to society than to benefit it, as I explained here: https://www.fast.ai/posts/2023-11-07-dislightenment.html https://www.fast.ai/posts/2023-11-07-dislightenment.html
- TaylorAlexander 3y ago
  [flagged]
  - d-z-m 3y ago
    Implicit in every reply you've given is the assumption that OP is treating the criticism from this researcher differently because she's a woman. Do you have any basis on which you're making this assumption? OP explained that they have substantive issues with the organization of which this researcher is a member.
    - TaylorAlexander 3y ago
      [flagged]
      - rmbyrro 3y ago
        Then why's it relevant to keep mentioning it's a woman?
      - d-z-m 3y ago
        > I can see why it would appear that I’m saying that, but that was not my intention. You are talking out of both sides of your mouth. In another comment on this same thread, you say this: > [...]women in the field are more readily dismissed, and I think they shouldn’t be. It’s a moment to check our internalized biases and make sure we’re operating in good faith. In your original comment you explicitly accuse the OP of operating in bad faith, presumably as a result of "internalized biases" as described above. How does this not add up to an assumption that OP treated the researcher differently because she's a woman? It is exactly what you are implying.
      - Tomte 3y ago
        And you would have called a simple "no" dismissive, too…
  - razodactyl 3y ago
    I think it's easier to dismiss risk with this project as it allows democratised access to AI models and furthers research in the field. The ability to generate low-quality content has been available since long before LLM technology, additionally these 70B param models are just barely fitting into $10,000 worth of hardware (not accounting for M-series chips). The scaling issue with potential runaway AI can be excluded. The potential for virus writing / security exploitation perhaps but such risks are already present with existing models so this point too can be excluded. I'm not sure there's any inherent risk here compared to what's easily available with a considerably reduced amount of resource requirements. The write up here seems concerned with allowing independent and democratised research which is a greater benefit than concentrated efforts.
imdsm 3y ago
> especially towards a female professor studying societal impacts of AI > especially towards a professor studying societal impacts of AI I can't help but feel like you're trying to load this question with some expectation that sex/gender should change how people react and respond. It shouldn't, at all, positively or negatively. Everyone is human (for now).
- TaylorAlexander 3y ago
  [flagged]
  - Borealid 3y ago
    I think you're saying that correcting a bias isn't itself applying bias. I think the poster to which you're responding is saying there wasn't any visible evidence of bias in the original behaviour. From one point of view, you're correcting a bias (in this case, one you suspect might exist), and you believe that isn't a bias. From another point of view, you're introducing a new bias (that definitely exists) to something that was previously (presumably) agnostic and neutral, by saying that a correction should only be applied when the question-asker meets certain criteria. Both points of view are legitimate. PERSONALLY I'd rather we not bring the "have you thought about how this might be discriminatory" bits into the converation unless there's at least some vague reason to think that it was, rather than presuming it's always discrimination when the genders and/or sexes and/or races line up a certain way. But that's because I think it's more important to debate the ideas in their pure form than to correct every instance of discrimination, and that's an arbitrary value judgement I made.
    - TaylorAlexander 3y ago
      [flagged]
drittich 3y ago
I completely get your take on this and don't disagree. IMO it reads as a flippant response to a potentially serious question, and I am going to assume this is unintentional, which does not mean it is unfair to call it out. Stepping back, this is the kind of discourse that Twitter can sometimes reinforce: short responses that, lacking context, can be interpreted in polarizing ways. After a bit of reading on the participants (because both of them are working in interesting areas), my belief is that the "pencil" response is actually shorthand for a whole set of beliefs that provide a lot of context and nuance to the discussion, but if you don't know that, it sounds like saying "AI is as dangerous as a new brand of chewing gum". In addition, without defining what risks we're talking about, it's really hard to know the scope the answer is addressing. E.g., societal risk due to AI in general? vs. say, the risks of affecting the quality an existing model with fine-tuning? So, I am chalking this up to a misunderstanding due to the limitations of the platform as well as the short-hand used in the response. And I could be completely wrong :)
Frantanata 3y ago
You are reading a lot into a tweet....

3abiton 3y ago

Are any of the NIPS resources available online?

notpublic 3y ago
Tim Dettmers QLoRA https://nips.cc/virtual/2023/83963 https://nips.cc/virtual/2023/83963 More here: https://nips.cc/virtual/2023/competition/66594 https://nips.cc/virtual/2023/competition/66594

paul_mk1 3y ago

It's clear that QLoRA has opened up finetuning to a wider audience with limited compute, which is a good thing. One thing I've wondered about: what are the drawbacks to using QLoRA? For example if compute is not a limit, I'm guessing one should not use QLoRA and finetune in full precision instead? Afaik when a model is first quantized to nf4 (before finetuning begins), model performance is degraded from baseline (see https://x.com/Tim_Dettmers/status/1661482614811918338?s=20 https://x.com/Tim_Dettmers/status/1661482614811918338?s=20). Dettmers shows that after finetuning wrt the dataset, the result is as good as full precision. But afaik never explored the effects outside the finetuning data. Assuming the finetuning dataset is small, the model will just be the degraded nf4 version, right? Or perhaps finetuning will even skew the model in weird ways (trying to fix quantization errors). Anecdotally models finetuned wth QLoRA perform well. Does anyone have any papers or a careful analysis of this?