6 ms·

I would be curious if this can do moderation with an arbitrary ruleset, or if it's just "that one moderation style" we already know from current big tech platfo

• charcircuit 2mo ago
  It sounds like it is. You have a set of moderation policies and then you evaluate the model 1 time per policy if it is violating it. Then you combine the results into a score you use for taking actions off of.
• nikcub 2mo ago
  > which seems to be mistrals whole thing They got a lot of hate for not keeping up with frontier model releases, but have managed to carve out a nice business that isn't even really niche. Before the datacenter deals their revenue was higher than xAI's There is a whole world out there of purpose built and hosted task specific vertical llms - especially with an emphasis on cost. Mistral, Microsoft model releases and Thinking Machines are all over this, and it's smart. Scoop up all the tasks that don't require large and expensive frontier general-purpose llms.
  • bunyak 2mo ago
    [flagged]
    • nikcub 2mo ago
      public ones are content moderation as above and previously llama guard, et al OCR is also another field - Mistral have a model, so do deepseek The ones I have experience with where you fine-tune smaller / faster models for business tasks like content writing, support, etc. by their nature stay private
      • Fnoord 2mo ago
        Voxtral [1], maybe Robostral? I just saw on their blog they released OCR 4. Their is usually informative [2] [1] https://mistral.ai/news/voxtral-transcribe-2/ https://mistral.ai/news/voxtral-transcribe-2/ [2] https://mistral.ai/news/ https://mistral.ai/news/
• nozzlegear 2mo ago
  > Question is just if it is also useful for society to hand the SV prefab morals down like that. Kinda like cultural imperialism but with an ethical spin. Isn't Mistral a French company? Not that the French can't do cultural imperialism either, but they (the French) don't strike me as very SV.
• rancar2 2mo ago
  Having grown a large healthcare review platform, I can attest to the success we had mapping specific policy violations to natural language is incredibly useful. At scale, patients having terrible situations and/days can write about in ways that can be deeply unhealthy for the community or the doctors reading/receiving the feedback and sometimes very threatening beyond that purposes for the community. We built a custom ML engine to handle our levels of traffic for reviews, which was among the largest in the US typical ranking top 3 on Google for the domain keywords. Back when BERT was the edge, a policy-adaptive model like this one from Mistral would have been an incredible cold-start solution. Most sites never have the massive volume nor budget needed nor skillset needed before you can train domain-specific models that outperform OOTB solutions. Generally, most people and site mean well and try to do well, so empowering those people with models like this can help the collective in my opinion, so I’m happy to see this released in this manner myself.
  • rancar2 2mo ago
    A bit of editorial and cultural note from a US native, the subsection of the original article “Teach discrimination, not memorization” is better worded as something like 'Differentiation' or 'Distinction' instead of ‘Discrimination’. In English, the word 'discrimination' can (and in this social context may) imply social prejudice or unfair treatment. I think this may have been a bit of carry over from the rather benign French translation of “Enseigner la discrimination" which I also see awkwardly translated in the paper as well.
    • NopIdoN 2mo ago
      "Discrimination" is exactly correct. What you suggest changes meaning.
      • rancar2 2mo ago
        Thanks for the clarification. I’m very happily wrong here, and I appreciate the correction. This does emphasize from an editorial review that the double meaning can be distracting for someone who isn’t very deep in the terminology of this particular area, so it may still be worth rewording the subheading for a broader audience which will be interested in this model.
• HawtAds 2mo ago
  > Kinda like cultural imperialism but with an ethical spin. As the old adage goes, US innovates, China imitates, EU regulates.
• asveikau 2mo ago
  > "that one moderation style" we already know from current big tech platforms. > The kind where malicious intent is okay if the words are nice. Where are you experiencing that? I hit "report" on social media for overt, violent threats and hate speech all the time and I almost never see moderation kick in.
  • dannyw 2mo ago
    I thought OP was referring to AI platforms. For example, by positioning what you’re doing as an accessibility tool and using the right words, you can get the latest models to write incredibly powerful malware without safeguards kicking in.
  • inigyou 2mo ago
    It's the one where you get censored for using words like "kill" or "shit", but if you say "I'm going to come to your house and unalive you" nothing happens because you didn't trip the word filter. When the company gets in trouble for this, to fix it, the company starts removing all comments that include the word "house".
  • numpad0 2mo ago
    I think you're referring to a different part of the same phenomenon as GP - the current big tech moderation feels a bit like using the shape of a crescent moon shape to cover a square.
• Fnoord 2mo ago
  if you pull stuff like that off, and it gets flagged, it is obvious you are trying to game the system. Same with a system like this. It could be used in addition to human moderators, where the human moderator have to meta-moderate the AI's work. This could also be used as training exercise for new moderators. Then, the good and experienced moderators have more time to spend on edge cases, complex cases, fine-tune the AI, etc. In other words, it is able to do the most boring things to you. And EU regulate, I mean as a counter example: we are not in panic about a nipple. When I was in a large museum in Paris, multiple women were breast feeding their infant. And why not? Kid's gotta eat. I'll refrain from insulting any world leaders, too easy, but you know many examples are available there regarding censorship. Finally, it can take that BS argument away of 'oh we don't have manpower to moderate'. That is a low blow, too, by large commercial entities who could, you know hire and train? However, even a small company with not much money to burn could -in theory- win here. I'd give this model a chance, if not only cause I've been impressed by Mistral past years. Yes, Le Chat / Vibe probably lags behind, but something like Voxtral (real-time and transcribe) is neat, and efficient.
• xp84 2mo ago
  I’ve felt for quite a long while that the moderation regime we fell into sometime around 2018-2020 has been shockingly bad. The rules are known and evaded by everyone, to the point I’m pretty sure Webster’s is adding “unalive” to the dictionary. What have we gained by making everyone use Newspeak to discuss everything? The 10-year-olds, who shouldn’t even be on these sites anyway, sure aren’t being tricked by all the thinly-coded language, so why are we censoring everything in the first place?
  • Gigachad 2mo ago
    The unalive thing I think is mostly down to silent deranking rather than regular moderation. People know certain things cause your posts to be deranked by the algorithm but you can’t know exactly what they are or when it’s happened. Which has lead to people preemptively avoiding things they think get deranked regardless of if it actually would have or not.
    • wongarsu 2mo ago
      Which is the most insidious kind of automated moderation. In an algorithmic feed like TikTok, strongly downranking content and removing it are close to the same. And by doing it silently with no way to easily get feedback on what you did wrong people self-censor both the things you are moderating and the things people imagine you would like to moderate
      • gwerbin 2mo ago
        That's the point
    • gwerbin 2mo ago
      Moderation by well-intentioned humans is totally different from moderation by a large corporation with a profit motive and political pressures.
  • torginus 2mo ago
    Yeah, one extreme case of this was on Reddit, where people DM'd 'kill yourself' messages to others. When mods started banning people for this, attackers switched to abusing Reddit's mental health features, reporting people as suicidal, which led to victims being flooded with links to suicide hotlines. That feature got taken offline as well.
• nolok 2mo ago
  Mistral specializes in tuning their models by customers. That and self hosting are like 90% of their business, they aim straight at what Europe would like to get (that fit needs, economic and regulatory). So the goal is probably to be able to tune this basis to your ruleset. As an addendum: US-style moderation is a big issue in europe and notably in France, with a very different touch on what's ok and what's not (obvious differences: hate speech and sex). Mistral is an european company with a french basis, so I doubt they didn't plan for that (otherwise they're complete morons, which I don't think they are).
  • egorfine 2mo ago
    It sure feels to me an LLM company stands no chance in the industry unless they succumb to specific North American moral values.
    • AdamN 2mo ago
      There are two usecases: 1/ general purpose, 2/ customer controlled. Mistral is focused on the second one and every customer, whether it's Boeing or Airbus or Nokia or Glencore, will want lots of control over 'their' models. That's not a North American moral values thing. For the first one yes there will be at most 2 or 3 model cultures but even there I think some customers will want really open and some will want more walled gardens.