4 ms·

This sounds interesting: "We will continue to offer a suite of safety filters that developers may apply to Google’s models. For the models released today, the

by naiv 2y ago

This sounds interesting: "We will continue to offer a suite of safety filters that developers may apply to Google’s models. For the models released today, the filters will not be applied by default so that developers can determine the configuration best suited for their use case."

ancorevard 2y ago
This is the most important update. Pricing and speed doesn't matter when your call fails because of "safety".
- CSMastermind 2y ago
  Also Google's safety filters are absolutely awful. Beyond parody levels of bad. This is a query I did recently that got rejected for "safety" reasons: Who are the current NFL starting QBs? Controversial I know, I'm surprised I'd be willing to take the risk with submitting such a dangerous query to the model.
  - elashri 2y ago
    Not stranger than my experience with openai. I got banned from DELL-3 access when it first came because I asked in the prompt about generating a particle moving in magnetic field of a forward direction and decays to two other particles with a kink angle between the particle and the charged daughter. I don't recall exact prompt but it should be something close to that. I really wonder what filters they had about kink tracks and why? Do they have a problem with Beyond standard model searches /s.
    - CSMastermind 2y ago
      For what it's worth I run every query I make through all the major models and Google's censorship is the only one I consistently hit. I think I bumped into Anthropics once? And I know I hit ChatGPTs a few months back but I don't even remember what the issue was. I hit Google's safety blocks at least a few times a week during the course of my regular work. It's actually crazy to me that they allowed someone to ship these restrictions. They must either think they will win the market no matter the product quality or just not care about winning it.
      - ancorevard 2y ago
        Microsoft's OpenAI models seems even worse. They run their own "safety" filter. Using their models in a medical setting is impossible. It refuses to describe scientific photos and will not summarize HCP discussions (texts) that it misinterprets.
    - bobwell 2y ago
      I'm reminded of this short story about a government bureaucracy banning research in certain areas to prevent dangerous technology being discovered: https://en.wikipedia.org/wiki/The_Dead_Past https://en.wikipedia.org/wiki/The_Dead_Past
  - nrvn 2y ago
    You know what. I just ran this query in gemini and it spit out “I'm not able to help with that, as I'm only a language model.” but just before that I got a glimpse of a real answer: NFL Starting Quarterbacks for the 2024 Season Note: Quarterback situations can change throughout the season due to injuries, trades, or poor performance. AFC • Baltimore Ravens: Lamar Jackson Buffalo Bills: Josh Allen But then it gets wiped and you cannot see it even in the drafts. The text above is from the screenshot I managed to make before the response vanished. This non-deterministic unpredictable behavior blended with poor “safety” policies is one of those major “dealbreakers” that pushes me back from trusting any existing LLMs.
KaoruAoiShiho 2y ago
There's still basic filters even if you take all the ones that you can turn off from the UI all off. It's still not capable of summarizing some YA novels I tried to feed it because of those filters.
panarky 2y ago
The "safety" filters used to make Gemini models nearly unusable. For example, this prompt was apparently unsafe: "Summarize the conclusions of reputable econometric models that estimate the portion of import tariffs that are absorbed by the exporting nation or company, and what portion of import tariffs are passed through to the importing company or consumers in the importing nation. Distinguish between industrial commodities like steel and concrete from consumer products like apparel and electronics. Based on the evidence, estimate the portion of tariffs passed through to the importing company or nation for each type of product." I can confirm that this prompt is no longer being filtered which is a huge win given these new lower token prices!

by naiv 2y ago

ancorevard 2y ago

This is the most important update. Pricing and speed doesn't matter when your call fails because of "safety".

CSMastermind 2y ago
Also Google's safety filters are absolutely awful. Beyond parody levels of bad. This is a query I did recently that got rejected for "safety" reasons: Who are the current NFL starting QBs? Controversial I know, I'm surprised I'd be willing to take the risk with submitting such a dangerous query to the model.
- elashri 2y ago
  Not stranger than my experience with openai. I got banned from DELL-3 access when it first came because I asked in the prompt about generating a particle moving in magnetic field of a forward direction and decays to two other particles with a kink angle between the particle and the charged daughter. I don't recall exact prompt but it should be something close to that. I really wonder what filters they had about kink tracks and why? Do they have a problem with Beyond standard model searches /s.
  - CSMastermind 2y ago
    For what it's worth I run every query I make through all the major models and Google's censorship is the only one I consistently hit. I think I bumped into Anthropics once? And I know I hit ChatGPTs a few months back but I don't even remember what the issue was. I hit Google's safety blocks at least a few times a week during the course of my regular work. It's actually crazy to me that they allowed someone to ship these restrictions. They must either think they will win the market no matter the product quality or just not care about winning it.
    - ancorevard 2y ago
      Microsoft's OpenAI models seems even worse. They run their own "safety" filter. Using their models in a medical setting is impossible. It refuses to describe scientific photos and will not summarize HCP discussions (texts) that it misinterprets.
  - bobwell 2y ago
    I'm reminded of this short story about a government bureaucracy banning research in certain areas to prevent dangerous technology being discovered: https://en.wikipedia.org/wiki/The_Dead_Past https://en.wikipedia.org/wiki/The_Dead_Past
- nrvn 2y ago
  You know what. I just ran this query in gemini and it spit out “I'm not able to help with that, as I'm only a language model.” but just before that I got a glimpse of a real answer: NFL Starting Quarterbacks for the 2024 Season Note: Quarterback situations can change throughout the season due to injuries, trades, or poor performance. AFC • Baltimore Ravens: Lamar Jackson Buffalo Bills: Josh Allen But then it gets wiped and you cannot see it even in the drafts. The text above is from the screenshot I managed to make before the response vanished. This non-deterministic unpredictable behavior blended with poor “safety” policies is one of those major “dealbreakers” that pushes me back from trusting any existing LLMs.

KaoruAoiShiho 2y ago

There's still basic filters even if you take all the ones that you can turn off from the UI all off. It's still not capable of summarizing some YA novels I tried to feed it because of those filters.

panarky 2y ago

The "safety" filters used to make Gemini models nearly unusable. For example, this prompt was apparently unsafe: "Summarize the conclusions of reputable econometric models that estimate the portion of import tariffs that are absorbed by the exporting nation or company, and what portion of import tariffs are passed through to the importing company or consumers in the importing nation. Distinguish between industrial commodities like steel and concrete from consumer products like apparel and electronics. Based on the evidence, estimate the portion of tariffs passed through to the importing company or nation for each type of product." I can confirm that this prompt is no longer being filtered which is a huge win given these new lower token prices!