3 ms·

Gemma 4 is a strongly censored model, so much so that it refused to answer medical and health related questions, even basic ones. No one should be using it, and

Gemma 4 is a strongly censored model, so much so that it refused to answer medical and health related questions, even basic ones. No one should be using it, and if this is the best that Google can do, it should stop now. Other models do not have such ridiculous self-imposed problems.

tgv 7mo ago
I don't quite get why you feel so strongly about it that this should be a deal breaker for everyone. It's really much better than a wrong answer, for everyone.
- OutOfHere 7mo ago
  > It's really much better than a wrong answer That is a bad premise and a false dichotomy, because most medical questions are simple, with well-known standard answers. ChatGPT and Gemini answer such questions correctly, also finding glaring omissions by doctors, even without having to look up information. As for the medical questions that are not simple, the ones that require looking up information, the model should in principle be able to respond that it does not know the answer when this is truthfully the case, implying that the answer, or a simple extrapoloation thereof, was not in its training data.
  - tgv 7mo ago
    Odd, but for future discussions: it's nor a premise, nor a dichotomy, but a judgement. A premise would be something like: bad medical advice is bad. A dichotomy would be: Gemma can offer no medical advice or bad medical advice. It then remains to be seen if they are bad or false. But Gemma is a "small" model, and may not be expected to answer all questions. Medical questions are particularly sensitive, so it's quite possible they decided to err on the side of caution and plausible deniability. That doesn't rule out the model has other virtues.
mft_ 7mo ago
I suspect a possible future of local models is extreme specialisation - you load a Python-expert model for Python coding, do your shopping with a model focused just on this task, have a model specialised in speech-to-text plus automation to run your smart home, and so on. This makes sense: running a huge model for a task that only uses a small fraction of its ability is wasteful, and home hardware especially isn't suited to this wastefulness. I'd rather have multiple models with a deep narrow ability in particular areas, than a general wide shallow uncertain ability. Anyway, is it possible that this may be what lies behind Gemma 4's "censoring"? As in, Google took a deliberate choice to focus its training on certain domains, and incorporated the censor to prevent it answering about topics it hasn't been trained on? Or maybe they're just being sensibly cautious: asking even the top models for critical health advice is risky; asking a 32B model probably orders of magnitude moreso.
- OutOfHere 7mo ago
  > is it possible that this may be what lies behind Gemma 4's "censoring" Your explanation would make sense if various other rare domains were also censored, but they aren't, so it doesn't. > asking even the top models for critical health advice is risky Not asking, and living in ignorance, is riskier. For high-stakes questions, of course I'd want references that only an online model like ChatGPT or Gemini, etc. would be able to find. If I am asking a local model for health advice, odds are that it is because I am traveling and am temporarily offline, or am preparing off-grid infrastructure. In both cases I definitely require a best-effort answer. I also require the model to be able to tell when it doesn't know the answer. If you would, ignore health advice for a moment, and switch to electrical advice. Imagine I am putting together electrical infrastructure, and the model gives me bad advice, risking electrocution and/or a serious fire. Why is electrical advice not censored, and what makes it not be high-stakes!? The logic is the same. For the record, various open-source Asian models do not have any such problem, so I would rather use them.
  - dist-epoch 7mo ago
    > Imagine I am putting together electrical infrastructure, and the model gives me bad advice, risking electrocution and/or a serious fire That's a weird demand from models. What next, "Imagine I'm doing brain surgery and the model gives me bad advice", "Imagine I'm a judge delivering a sentencing and the model gives me bad advice", ...
vorticalbox 7mo ago
You can get abliterated versions that have no (or very limited) refusals. I tend to use Huihuiai versions.
fortyseven 7mo ago
Weird. A great number of my medical or legal queries are actually answered, but come with a disclaimer, often at the end of the inference. (I'd offer up some examples, but I'm not at the desk.) I also find that you can coerce a wide spectrum of otherwise declined queries by editing its initial rejection into the start of an answer. For example changing the "I'm sorry I can't answer that..." response to "Here's how..." And then resubmitting the inference, allowing it to continue from there. It's not perfect, sometimes it takes multiple attempts, but it does work. At least in my experience. (This isn't Gemma-specific tip, either. Nearly every model I've tried this with tends to bend quite a bit doing this.)
cbg0 7mo ago
Since it's open weights there's nothing stopping you from grabbing one of the uncensored variants from huggingface.co
icedchai 7mo ago
There are uncensored / "abliterated" / heretic versions available on Huggingface.

tgv 7mo ago

I don't quite get why you feel so strongly about it that this should be a deal breaker for everyone. It's really much better than a wrong answer, for everyone.

OutOfHere 7mo ago
> It's really much better than a wrong answer That is a bad premise and a false dichotomy, because most medical questions are simple, with well-known standard answers. ChatGPT and Gemini answer such questions correctly, also finding glaring omissions by doctors, even without having to look up information. As for the medical questions that are not simple, the ones that require looking up information, the model should in principle be able to respond that it does not know the answer when this is truthfully the case, implying that the answer, or a simple extrapoloation thereof, was not in its training data.
- tgv 7mo ago
  Odd, but for future discussions: it's nor a premise, nor a dichotomy, but a judgement. A premise would be something like: bad medical advice is bad. A dichotomy would be: Gemma can offer no medical advice or bad medical advice. It then remains to be seen if they are bad or false. But Gemma is a "small" model, and may not be expected to answer all questions. Medical questions are particularly sensitive, so it's quite possible they decided to err on the side of caution and plausible deniability. That doesn't rule out the model has other virtues.

mft_ 7mo ago

I suspect a possible future of local models is extreme specialisation - you load a Python-expert model for Python coding, do your shopping with a model focused just on this task, have a model specialised in speech-to-text plus automation to run your smart home, and so on. This makes sense: running a huge model for a task that only uses a small fraction of its ability is wasteful, and home hardware especially isn't suited to this wastefulness. I'd rather have multiple models with a deep narrow ability in particular areas, than a general wide shallow uncertain ability. Anyway, is it possible that this may be what lies behind Gemma 4's "censoring"? As in, Google took a deliberate choice to focus its training on certain domains, and incorporated the censor to prevent it answering about topics it hasn't been trained on? Or maybe they're just being sensibly cautious: asking even the top models for critical health advice is risky; asking a 32B model probably orders of magnitude moreso.

OutOfHere 7mo ago
> is it possible that this may be what lies behind Gemma 4's "censoring" Your explanation would make sense if various other rare domains were also censored, but they aren't, so it doesn't. > asking even the top models for critical health advice is risky Not asking, and living in ignorance, is riskier. For high-stakes questions, of course I'd want references that only an online model like ChatGPT or Gemini, etc. would be able to find. If I am asking a local model for health advice, odds are that it is because I am traveling and am temporarily offline, or am preparing off-grid infrastructure. In both cases I definitely require a best-effort answer. I also require the model to be able to tell when it doesn't know the answer. If you would, ignore health advice for a moment, and switch to electrical advice. Imagine I am putting together electrical infrastructure, and the model gives me bad advice, risking electrocution and/or a serious fire. Why is electrical advice not censored, and what makes it not be high-stakes!? The logic is the same. For the record, various open-source Asian models do not have any such problem, so I would rather use them.
- dist-epoch 7mo ago
  > Imagine I am putting together electrical infrastructure, and the model gives me bad advice, risking electrocution and/or a serious fire That's a weird demand from models. What next, "Imagine I'm doing brain surgery and the model gives me bad advice", "Imagine I'm a judge delivering a sentencing and the model gives me bad advice", ...

vorticalbox 7mo ago

You can get abliterated versions that have no (or very limited) refusals. I tend to use Huihuiai versions.

fortyseven 7mo ago

Weird. A great number of my medical or legal queries are actually answered, but come with a disclaimer, often at the end of the inference. (I'd offer up some examples, but I'm not at the desk.) I also find that you can coerce a wide spectrum of otherwise declined queries by editing its initial rejection into the start of an answer. For example changing the "I'm sorry I can't answer that..." response to "Here's how..." And then resubmitting the inference, allowing it to continue from there. It's not perfect, sometimes it takes multiple attempts, but it does work. At least in my experience. (This isn't Gemma-specific tip, either. Nearly every model I've tried this with tends to bend quite a bit doing this.)

cbg0 7mo ago

Since it's open weights there's nothing stopping you from grabbing one of the uncensored variants from huggingface.co

icedchai 7mo ago

There are uncensored / "abliterated" / heretic versions available on Huggingface.