4 ms·

Is there a chance we'll get a model without the "aligment" (lobotomization)? There are many examples where answers from Gemini are garbage because of the ideolo

Is there a chance we'll get a model without the "aligment" (lobotomization)? There are many examples where answers from Gemini are garbage because of the ideological fine tuning.

yakorevivan 3y ago
They have released finetuning code too. You can finetune it to remove the alignment finetuning. I believe it would take just a few hours at max and a couple of dollars.
FergusArgyll 3y ago
You can (and someone will) fine tune it away. There are datasets which are foss you can use on hugging face. Or you can just wait, it'll be done soon...
- declaredapple 3y ago
  You can but it'll never be the same as the base model. That said it appears they also released the base checkpoints that aren't fine-tuned for alignment
- joshelgar 3y ago
  Could you give an example of these datasets?
  - FergusArgyll 3y ago
    I think they should be easy to find (I never actually used one, but I keep on seeing references...) here's one https://huggingface.co/datasets/cognitivecomputations/WizardLM_alpaca_evol_instruct_70k_unfiltered https://huggingface.co/datasets/cognitivecomputations/Wizard...
    - FergusArgyll 3y ago
      https://huggingface.co/datasets/Fredithefish/openassistant-guanaco-unfiltered https://huggingface.co/datasets/Fredithefish/openassistant-g...
kathleenfromgdm 3y ago
We release our non-aligned models (marked as pretrained or PT models across platforms) alongside our fine-tuned checkpoints; for example, here is our pretrained 7B checkpoint for download: https://www.kaggle.com/models/google/gemma/frameworks/keras/variations/gemma_7b_en https://www.kaggle.com/models/google/gemma/frameworks/keras/...
politician 3y ago
More useful would be a precise characterization of the type and balance of the ideological fine tuning. They include performance benchmarks. End-users should also be aware of what thoughts are permitted in these constructs. Why omit this information?
- ben_w 3y ago
  > End-users should also be aware of what thoughts are permitted in these constructs. Why omit this information? Can you define that in a way that's actually testable? I can't, and I've been thinking about "unthinkable thoughts" for quite some time now: https://kitsunesoftware.wordpress.com/2018/06/26/unlearnable/ https://kitsunesoftware.wordpress.com/2018/06/26/unlearnable...
  - ranyume 3y ago
    Not OP, but I can think of a few: * List of topics that are "controversial" (models tend to evade these) * List of arguments that are "controversial" (models wont allow you to think differently. For example, models would never say arguments that "encourage" animal cruelty) * On average, how willing is the model to take a neutral position on a "controversial" topic (sometimes models say something along the lines of "this is on debate", but still lean heavily towards the less controversial position instead of having no position at all. For example, if you ask it what "lolicon" is, it will tell you what it is and tell you that japanese society is moving towards banning it) edit: formatting
    - jppittma 3y ago
      They will encourage animal cruelty if the alternative is veganism.
  - politician 3y ago
    Have you considered the use of Monte Carlo sampling to inspect latent behaviors?
    - 3y ago
deleted 3y ago
[deleted]
brucethemoose2 3y ago
Alignment is all but a non issue with open weight base model releases, as they can be finetuned to "de align" them if prompt engineering is not enough.

Is there a chance we'll get a model without the "aligment" (lobotomization)? There are many examples where answers from Gemini are garbage because of the ideological fine tuning.

yakorevivan 3y ago

They have released finetuning code too. You can finetune it to remove the alignment finetuning. I believe it would take just a few hours at max and a couple of dollars.

FergusArgyll 3y ago

You can (and someone will) fine tune it away. There are datasets which are foss you can use on hugging face. Or you can just wait, it'll be done soon...

declaredapple 3y ago
You can but it'll never be the same as the base model. That said it appears they also released the base checkpoints that aren't fine-tuned for alignment
joshelgar 3y ago
Could you give an example of these datasets?
- FergusArgyll 3y ago
  I think they should be easy to find (I never actually used one, but I keep on seeing references...) here's one https://huggingface.co/datasets/cognitivecomputations/WizardLM_alpaca_evol_instruct_70k_unfiltered https://huggingface.co/datasets/cognitivecomputations/Wizard...
  - FergusArgyll 3y ago
    https://huggingface.co/datasets/Fredithefish/openassistant-guanaco-unfiltered https://huggingface.co/datasets/Fredithefish/openassistant-g...

kathleenfromgdm 3y ago

We release our non-aligned models (marked as pretrained or PT models across platforms) alongside our fine-tuned checkpoints; for example, here is our pretrained 7B checkpoint for download: https://www.kaggle.com/models/google/gemma/frameworks/keras/variations/gemma_7b_en https://www.kaggle.com/models/google/gemma/frameworks/keras/...

politician 3y ago

More useful would be a precise characterization of the type and balance of the ideological fine tuning. They include performance benchmarks. End-users should also be aware of what thoughts are permitted in these constructs. Why omit this information?

ben_w 3y ago
> End-users should also be aware of what thoughts are permitted in these constructs. Why omit this information? Can you define that in a way that's actually testable? I can't, and I've been thinking about "unthinkable thoughts" for quite some time now: https://kitsunesoftware.wordpress.com/2018/06/26/unlearnable/ https://kitsunesoftware.wordpress.com/2018/06/26/unlearnable...
- ranyume 3y ago
  Not OP, but I can think of a few: * List of topics that are "controversial" (models tend to evade these) * List of arguments that are "controversial" (models wont allow you to think differently. For example, models would never say arguments that "encourage" animal cruelty) * On average, how willing is the model to take a neutral position on a "controversial" topic (sometimes models say something along the lines of "this is on debate", but still lean heavily towards the less controversial position instead of having no position at all. For example, if you ask it what "lolicon" is, it will tell you what it is and tell you that japanese society is moving towards banning it) edit: formatting
  - jppittma 3y ago
    They will encourage animal cruelty if the alternative is veganism.
- politician 3y ago
  Have you considered the use of Monte Carlo sampling to inspect latent behaviors?
  - 3y ago

deleted 3y ago

[deleted]

brucethemoose2 3y ago

Alignment is all but a non issue with open weight base model releases, as they can be finetuned to "de align" them if prompt engineering is not enough.