5 ms·

I don't understand countries (especially governments) wanting to have their own models when there are already pretty solid open source (weights) models out ther

• Achterlangs 4mo ago
  It is not about the country but the language. Most llms have poor or no support for Dutch.
  • tgv 4mo ago
    Idk which models you refer to, but I tested a bunch recently, and they performed well on Dutch. Only the smallest, such as qwen 3.6 27B, made up words and switched languages.
    • dvdkon 4mo ago
      There would be a bunch of value in having, say, a good 30B-class model that used my local language as well as it does English. There's lots of cases, especially in the government sphere, where local processing is a requirement and frontier-level capabilities aren't required. Making those cheap to run seems like a fine goal.
      • throw310822 4mo ago
        Can you provide some examples of these use cases?
        • bigfudge 4mo ago
          Support bots and question answering with access to sensitive pii?
          • throw310822 4mo ago
            Yes, but what's the point of a support bot that writes good Dutch when it can't follow instructions, doesn't understand the questions or can't solve problems? I might be wrong, but I don't think atm these models have the cognitive ability to perform any task in a satisfactory manner. As for accessing pii, I imagine the value here is in the fact they're local, which has nothing to do with the "sovereignty" of these models. If anything, a model is more likely to be tricked by a malicious prompt the farther it is from the sota.
            • bigfudge 4mo ago
              A good harness and engineering is important no matter which model you use. But Sovereignty of hosting is also important because without it all pii is being leaked.
    • numeri 4mo ago
      There's a large gap between making up words and an actually native text distribution. LLMs have a clear pattern, clear tells, a "feel" in English, and it's normally even more pronounced in non-English languages. Lots of bias towards English sentence structure, idioms, etiquette, etc.
      • tgv 4mo ago
        I didn't notice any of that. Such a bias would be strange, because certainly smaller models don't have the luxury of learning grammar independently: it's still word sequences, and languages are quite well separated.
  • throw310822 4mo ago
    I don't understand this. Even if that were true (and it isn't in my experience), a model that is trained on a Dutch corpus and arguably "knows Dutch well" but has the reasoning and comprehension abilities of a three year old is useless in any case. I'd rather use a model that can only speak English and put an automatic translator around it.
• joe_mamba 4mo ago
  [flagged]
  • dwa3592 4mo ago
    >>Yeah but Europe doesn't build any computer hardware, Well, then this is will be a good start.
    • joe_mamba 4mo ago
      EU bureaucrats are too busy trying to keep the welfare/pension system from collapsing, defeating Russia, supporting Ukraine, managing the fossil fuels energy shortages, figuring out how to nerf Chinese EVs while supporting domestic car companies, and restricting social media free speech to make sure the "far right" don't win elections. So of course, semiconductor manufacturing sovereignty is very low on their priority list.
      • ks2048 4mo ago
        How many in that list of things is the US also doing?
        • nazgul17 4mo ago
          The US is a single country. Russia is not on the US' doorstep. The US has its own oil. The US prints the world reserve currency.
        • joe_mamba 4mo ago
          Different scale for those problems. Way different scale. US is monetary rich, oil rich, energy rich, manufacturing rich, and doesn't suffer from russian aggression at its borders. US is so bored from how rich and problem free it is compared to Europe, that it can afford to keep starting foreign wars as if nothing ever happens. Also back on the topic, the US managed to bring TSMC to open a cutting edge fab in the US and has already been operational for a while. Which already puts it way ahead of the EU on this front as well. The thing is, US is much better on actually making things happen when push comes to shove. It saw it's deficient and vulnerable on domestic semiconductor manufacturing, it then made it happen with TSMC. It's doing the same with domestic ship building with Korean partners. US might be slow moving, but somehow EU is even way slower at realizing and addressing its vulnerabilities, only waking up when it's far too late, causing it to pay a much more painful price for sleeping at the wheel (Russian invaded Ukraine in 2014 BTW, not in 2022, and they were building another gas pipeline with them), and when this type of own-goaling keeps repeating enough times you see the correlation with EU's decline as their economic rivals keep biting more and more market share from their industries as they sleep on critical changes and developments.
• applfanboysbgon 4mo ago
  Why should Dutch people be expected to make do with models 99% trained on American/Chinese cultural context and language?
  • dwa3592 4mo ago
    Understood, but they could fine tune base models on their own cultural context and language. Why reinventing the wheel?
    • applfanboysbgon 4mo ago
      This gets better short-term results for a fraction of the cost, for sure, but what do you when China places an export control banning the release of open weight models? If you don't have your own talent, you're then relegated to using a base model from 2026 or whatever the cutoff date is, forever. That defeats the purpose of a 'sovereign' model made for and by your people.
    • DonHopkins 4mo ago
      They could apply the Polder Model of consensus decision making with a mixture of experts. https://en.wikipedia.org/wiki/Polder_model https://en.wikipedia.org/wiki/Polder_model
      • nehal3m 4mo ago
        Funny, that's what I thought when PewDiePie set up his monster AI rig and what he called a 'council'. Quote: "PewDiePie has built a custom web UI for self-hosting AI models called "ChatOS" that runs on his custom PC with 2x RTX 4000 Ada cards, along with 8x modded RTX 4090s with 48 GB of VRAM. Running open-source models from Baidu and OpenAI, PewDiePie made a "council" of bots that voted on the best responses, and then built "The Swarm" for data collection that will become the foundation of his own model coming next month." https://www.tomshardware.com/tech-industry/artificial-intelligence/pewdiepie-goes-all-in-on-self-hosting-ai-using-modded-gpus-with-plans-to-build-own-model-soon-youtuber-pits-multiple-sentient-chatbots-against-each-other-to-find-the-best-answers https://www.tomshardware.com/tech-industry/artificial-intell...
• SiempreViernes 4mo ago
  Really? Because I'm pretty sure that at least every two days there's a active post with a top voted comment along the lines of "The EU isn't doing AI themselves, they are so hosed".
• vrganj 4mo ago
  An LLM is an encoding of a culture, a way of viewing the world. They are not neutral technology, they are a direct representation of the training set that has been chosen and how they are aligned. In many ways, they are ideology made code. If we leave building them to the US and China, only their way of seeing things will be digitized. I don't like the idea of that.
  • wolvoleo 4mo ago
    Yes and also, US and Chinese models are censored in different ways. US models are way too prudish for personal use in Europe because they're afraid to piss off religious investors. Chinese models are too censored on history and current affairs, eg the tiananmen massacre never happened stuff like that.
    • slopinthebag 4mo ago
      Chinese models aren't censored as much as you think, you can download the model and run it somewhere else and they will happily tell you about Tiananmen Square. Or heck, ask DeepSeek via Openrouter, it will do the same. The censorship works kind of like with Fabel, it kicks in before the model responds.
• keynha 4mo ago
  [flagged]
• jeroenhd 4mo ago
  There's an absolutely massive cultural and behavioural bias in those models. Models will suggest things like "go to the hospital" for things that require GP appointments, "just drive three hours" while it's faster to go places by train, and so on. They will do it in anglicised Dutch (compound words split, English-like grammar structures) that's perfectly understandable, but the cultural bias is there if you know to look for it. Furthermore, the expertise in designing and training these models is valuable as well. The existing models are good as a starting point in terms of learning from previous mistakes, but we should not just let a handful of American and Chinese people keep the knowledge and expertise. One problem with this particular project, though, is that copyright has been enforced for Dutch LLM training before, and the AI industry cannot exist without massive scale piracy, the likes of which has never been seen before. A lot of Dutch training material exists in pirated books that AI companies in countries that do not care about copyright have access to, but are exempted from the training set here. The impact of enforcing copyright on an AI model will be quite interesting to see.
• andy12_ 4mo ago
  To be fair. There is a security concern angle: even open-source models could be trained as sleeper agents that act adversarially (for example, adding backdoors) when used in specific national companies in specific settings. This is very difficult to detect or void, so if you want to be sure 100% that this isn't the case, you have to train your own model from scratch.