9 ms·
GPT‑NL: a sovereign language model for the Netherlands
- HelloUsername 4mo agoPreviously posted on 02-dec-2023 https://news.ycombinator.com/item?id=38497495 https://news.ycombinator.com/item?id=38497495 3 comments
- ronsor 4mo agoTwo and a half years and still not complete? That's ridiculous.
- pedromlsreis 4mo agoAMALIA, from Portugal, going the same path! https://en.wikipedia.org/wiki/Am%C3%A1lia_(LLM) https://en.wikipedia.org/wiki/Am%C3%A1lia_(LLM)
- SiempreViernes 4mo agoThey are starting to deploy to customers now, not sure if that counts as "complete" or not. The big innovation here is that they are doing it all legally.
- Marciplan 4mo agoSupposedly this model also aims to treat publishers of all sizes well. Looking forward to its launch soon :)
- adalacelove 4mo agoMaybe it's time to acknowledge that current copyright laws do more harm than good and put another framework in place.
- jansenmac 4mo agoThis is not an open source model. In that sense I think the sovereign claim is a bit strange. It's the data providers that determine access to the model.
- frangonf 4mo agoSo it's a model that's sovereign as in sovereign kingdom of the Netherlands vs sovereign for the people's?
- embedding-shape 4mo ago"sovereign" the marketing term basically means "in-house" now, where "house" depends on who says it.
- stared 4mo agoIs it a proposal or a model? And if it is a model, how fies it fare on benchmarks?
- wrs 4mo agoThey’re building a competitive-quality model, from scratch, with fair compensation to content owners, for €13.5 million? Something’s wrong with this picture.
- Muromec 4mo agoBeing cheap is on brand for inhabitants of the sea floor. Nothing is wrong
- rollulus 4mo agoInteresting that this got posted now: the project is receiving increasingly more skepticism lately in the Dutch tech scene [0], and I think that’s fully justified. [0]: https://www.quotenet.nl/zakelijk/a71588202/techondernemers-maken-nederlandse-chatgpt-belachelijk-kan-echt-helemaal-niks/ https://www.quotenet.nl/zakelijk/a71588202/techondernemers-m...
- deleted 4mo ago[deleted]
- embedding-shape 4mo agoWhat is the exact skepticism? The only thing I could get from that was from some "tech entrepreneur": > GPT-NL was never built to compete with Claude or ChatGPT. It was trained exclusively on licensed data, and is intended more for governments and companies where privacy and compliance matter more than raw performance.” That's it? That it didn't aim to compete with SOTA models? Maybe this is something you have to start with something, then ramp up, rather do what only a select few labs been able to do, start with really big models. Especially if you're resource constrained, which since this is a government project, I really hope for the sake of the tax payers it was.
- barrenko 4mo agoI mean if you are wasting funds kind of knowing it's nowhere near remote competitive, then it's kind of a fraud.
- embedding-shape 4mo agoBut why is "competing against remote SOTA models on quality" the only thing that matters here?
- barrenko 4mo agoWhat the hell else is there? All the other stuff can be done by an intern with an 8 euro HF Pro subscription. Other than actual research, which is in a different camp.
- simianwords 4mo agoI really think countries should build a sovereign _ecosystem_ and sovereign models are an excuse to achieve it. An ecosystem is the tribal knowledge, revolving door of talent, known processes etc. If the end goal is to make a half assed Dutch speaking model, I think it won’t cut it. I don’t see anyone using it over Gemma 4b that runs on my laptop. An ecosystem is more durable and has desirable second order effects.
- dwa3592 4mo agoI don't understand countries (especially governments) wanting to have their own models when there are already pretty solid open source (weights) models out there. Countries should want control over _where_ the compute is happening rather than _what code_ is running. What's wrong with a country hosting a Kimi, Qwen or GPT-Oss on their hardware for their government work purpose?
- Achterlangs 4mo agoIt is not about the country but the language. Most llms have poor or no support for Dutch.
- tgv 4mo agoIdk which models you refer to, but I tested a bunch recently, and they performed well on Dutch. Only the smallest, such as qwen 3.6 27B, made up words and switched languages.
- dvdkon 4mo agoThere would be a bunch of value in having, say, a good 30B-class model that used my local language as well as it does English. There's lots of cases, especially in the government sphere, where local processing is a requirement and frontier-level capabilities aren't required. Making those cheap to run seems like a fine goal.
- throw310822 4mo agoCan you provide some examples of these use cases?
- bigfudge 4mo agoSupport bots and question answering with access to sensitive pii?
- stared 4mo agoI feel that not only is Europe losing its independence to the US and China, but it does not even try to take part in the race. Unlike the US, Europe has no California-level VCs. I don't expect hundreds of billions of Euros to be poured into long-shot projects. Unlike China, Europe has neither cohesive public investment at the global level nor the drive to grow. Long-term investments have a lot of words, a lot of regulations, a lot of proxy goals, but there is neither a lot of money nor urgency. It was captured by this post: https://x.com/piotrsankowski/status/2065795919623438546 https://x.com/piotrsankowski/status/2065795919623438546 So yeah, both in economy and warfare, Europe dooms itself to be in the hands of the US, China, or a mix of both.
- ews 4mo agoEurope decided to regulate the hell out of foreign AI instead of investing in their own systems. It's sad to see the European continent lost the race to create a decent startup ecosystem (no decent search engines, social networks, cloud, mobile OS) and now it seems to be hellbent in losing this battle.
- joe_mamba 4mo ago>It's sad to see the European continent lost the race to create a decent startup ecosystem What's ironic and sad at the same time is that pre-2022 Russia's Yandex(domestic Russian variant of Google) was lightyears ahead of what EU, a significantly richer and more capable block, had. IIRC, their reverse image search was so good, they had to nerf it because people were using it to find the identity of people from photos. Same for Israel, their tech sector is probably greater than the EU one combined Absolutely shameful how the EU kept managing to snatch defeat from the jaws of victory over and over.
- vanviegen 4mo agoI think much of that is because European customers (both private and business) tended to prefer American suppliers over suppliers from European countries that were not their own. That may have something to do with most people in IT being quite fluent in English, while European products were all-to-often half-heartedly translated from German/French/Spanish/Polish/Italian/Ukranian. In many cases, well-established and well-liked European services have been supplanted by American counterparts that came later and were not really better in any way. They did usually have much more money to burn though, undercutting pricing until competition was dead. I'm speaking in the past tense, because now for the first time in the couple of decades I can remember, there seems to be a somewhat commonly held preference for European suppliers.
- gnegggh 4mo agoI'm making a Dutch dictionary and would be interested to see how this model would fair in evals vs non specialized ones. I've tested a variety of models for https://hetnederlands.com https://hetnederlands.com content and differences can be big
- thatguymike 4mo ago> A total of €13.5 million has been allocated to the project. > This public investment underlines the importance of an independent, trustworthy and future‑proof Dutch language model. It does, but not in the way you think it does.
- thepasch 4mo ago> It does, but not in the way you think it does. They're training a model, not funding a startup. €13.5 million is plenty to pre- and post-train a decent model.
- matheusmoreira 4mo agoSo good to see these developments. Every country should do this. I'd even say every person should gave their own personalized AI running on their own computers. If only the costs involved were not so astronomical.
- nathanielsimard 4mo agoI think it will be cost effective at some point. Computers were limited to research institutes before the personal computer arrived.
- matheusmoreira 4mo agoI hope you're right. I really don't want a future where only corporations and governments have computers.
- 14u2c 4mo agoNvidia will certainly be pleased.
- mediaman 4mo agoWhy? That doesn't make any sense. The government would be far better off figuring out how to take commodity models and applying them to government functions where they can, with deterministic scaffolding and guardrails, to make government more efficient, optionally using RL on traces from their use to improve their performance. Imagine taking models and fine-tuning them / doing RL rollouts to help automate permit application approvals, as applied specifically to Dutch permit processes. That would be a real help to Dutch businesses! That type of applied AI is more interesting and effective now than just trying to make another foundational model that isn't going to work well or do anything of economic value.
- matheusmoreira 4mo ago> Why? Because then the USA can't just turn it off.
- 4mo ago
- sarjann 4mo agoI wonder with these stories. Why are there so many individual country efforts? We know the scale needed with scaling laws / capital / energy. Most of these countries alone can barely compete (even large groups of them would struggle. Why don't they work together on it? Companies like Airbus have already been able to do that with aircraft.
- dr_dshiv 4mo agoHow do you use it?
- sublimefire 4mo agoIt is crazy that anything Europe gets so much hate. IMO it is important to build models within the boundaries of smaller nations, using their own language. Research has to continue even if it is outside of US and China.
- transcriptase 4mo agoIt’s not that it gets hate so much as it’s akin to watching them make announcements that they’re going to make a European google/facebook/tiktok. Sure… they can, except at the end of the day it’s a bit late, regulatory burden will make it comparatively useless, and because of that nobody will ever use it. It will be spending a bunch of taxpayer dollars for press releases. The running joke is that when these “sovereign” EU models launch, they’re going to refuse to answer anything that might involve personal information such as Elon Musk’s birthday.
- Lucasoato 4mo agoI kinda agree, the best use of taxpayer money should be in reducing taxes to corporation that would like to compete in the market vs US and China, rather than making governments playing the game (since they very obviously can’t).
- data-ottawa 4mo agoThat’s on Wikipedia, it’s not PII, it’s also not going to be relevant to any meaningful IRL work. I challenge the assumption you can do meaningful work in this field without blatant disregard for intellectual property. The idea that it’s all down to training size is clearly incorrect, as every expert human learned their craft without nearly the sum total information of the internet. Clearly there are architectural wins to be found. Besides that, why would everyone just be fine with Opus level AI at best, as that’s all the US is willing to export, and I doubt China will share beyond that. Sovereign AI is more important than ever after Friday.
- arrrg 4mo agoAt least with social networks the network effect is a powerful force. Foregrounding regulatory burden in that context is nonsensical. (That does not apply in the same way to models.)
- armcat 4mo agoI keep seeing these "sovereign" LMs time and time again. In Sweden we had GPT-SW3 (https://www.ai.se/en/project/gpt-sw3 https://www.ai.se/en/project/gpt-sw3) and same story there. Instead of burning money on "sovereign" claims, national research labs should instead focus on building on top of solid baselines (like Qwen/Kimi) and finetuning frontier models with real agentic utility that can be applied across actual use cases and can be widely used by its people, basically for free. Nations should mirror what Cursor has done with Composer 2.5 for example.
- thevinter 4mo agoAnd what happens once the "solid baselines" become unavailable for a reason or the other?
- zozbot234 4mo agoYou keep building on the last available version? Fine tuning is a whole lot cheaper, easier and more useful than pretraining a model from scratch. It's a complete no brainer.
- rapidfl 4mo ago> You keep building on the last available version? yes but a sovereign can allocate some resources and a few people to stay in the loop from a first principles level. No need to wait for a rug pull. Of course, it can not compete with the frontier labs. But good to have researchers and professors "in-house". LLMs are here for the long-term.
- michaelscott 4mo agoUnfortunately in this game first principles requires massive resources, not "some". Building in-house on top of existing open weights is a good way to bootstrap this process, especially since there's nothing inherently magical or particularly expertise-heavy when it comes to weights themselves
- 4mo ago
- debarshri 4mo agoSo cute.
- GreenSalem 4mo ago[flagged]
- jermaustin1 4mo agoWhy are you being nasty? And in what world is the Netherlands a second-world nation?
- dash2 4mo agoThis really isn’t what HN is for. I think you should read the guidelines. I’ve looked at your posting history and it is all like this: relentless negativity with low information content.
- mvanbaak 4mo ago> Excluding harmful content #define(HARMFUL) [edit] Downvoters please tell me what the problem is with specifying this?
- jermaustin1 4mo agoI didn't down vote you, but you aren't really adding anything to the conversation. This type of pithy comment might be fun on Reddit, but at HN, we try to provide more constructive, and information rich, comments.
- jurschreuder 4mo agoWhat are they going to train with 13.5M really? We're a tiny company in Amsterdam in Holland and we've got "only 64x B300 to train on" so we could never make an LLM I thought, since we've got only 4M in compute. And they're going to train an LLM with all kinds of extra difficulties compared to OpenAI for just 13.5M? The very first Llama was 16M for one training.
- LaurensBER 4mo agoThis is too little, too late. Europe really need to start focussing. All these tiny niche models are perhaps fun as an academic exercise or great for the researchers resume but I highly doubt that they'll add any value or will be used for anything serious. Even if this becomes a somewhat decent model with a fantastic understanding of "gezellig", "kring verjaardag" or "pannenkoeken", how many people will interact with it before the limits of it will drive them back to a frontier model? Even if the purpose of this is government & other regulated industries, do we really want our government to use a poor model? Either do it right or don't do it at all.
- numeri 4mo agoPrices for training have dropped immensely in terms of research required, code efficiency, algorithmic/sample efficiency, and possibly also hardware (I'm not qualified to say without looking it FLOPS/dollar, or even to be certain that's the right metric here).
- wolvoleo 4mo agoWe already had GEITje but it was banned by the courts. Of course it can still be found because the entire internet is not subject to Dutch law. But it did manage to stop development :'(
- Aeolun 4mo agoA total of €13.5M has been allocated to the project. I guess we’re going for GPT2 level capability?
- Marciplan 4mo agoapparently their aim is GPT3.5
- siva7 4mo ago> GPT‑NL is developed within the Netherlands and Europe. This gives us full control over the model, the data and the choices we make. We avoid dependency on non‑European providers and invest in a sustainable AI ecosystem aligned with our laws, values and societal goals. I love it! So this is our answer to America and China denying foreigners access to their frontier models.. a massive 13,5M€ founding to develop souvereign european ai, trained exclusively on legally obtained documents and highest moral standards as defined in EU AI Act.
- deleted 4mo ago[deleted]
- jbverschoor 4mo agoNL could simply say: no more ASML machines, and no more ASM wafers.
- siva7 4mo agoYou don't wanna find out how fast american troops would land there..
- Muromec 4mo agoFaster than in Iran?
- rmccue 4mo agoASML’s EUV technology is partially based on US research and so Congress has a degree of control over it, so it’s not that simple: https://web.archive.org/web/20230116222847/https://www.nytimes.com/2021/07/04/technology/tech-cold-war-chips.html https://web.archive.org/web/20230116222847/https://www.nytim...
- jbverschoor 4mo agoThe US can only block exports. They cannot force exports. The NL-US relationship is quite toxic, for example the "Dutch America Friendship Treaty". Everything is very one-sided.
- WarmWash 4mo agoIf Europe is serious about getting home grown AI fast, three simple steps: 1. Huge tax incentives, let the companies get grossly wealthy while paying minimal taxes. Minimum 10 years with clauses protecting "retribution" taxes there after. 2. Tax incentives for the founders/shareholders, just like above. 3. Drop worker protections to a minimum, make it easy to fire people. You only want serious/dedicated employees anyway. Within 2-3 years there will be at least a trillion dollars looking to get in. Don't worry though if reading that made you mad. Its absolutely not going to happen. I can think of few things more antithetical to the European ethos than smart skilled people working 80-100hrs weeks with almost no vacation to gas their founders net worth by tens, hundreds, of billions.
- deleted 4mo ago[deleted]
- yanis_t 4mo agoExactly this. You can't have a competitive industry while at the same time heavily redistributing wealth to the point where people don't have any incentives at all.
- lpapez 4mo agoButcher worker protections and quality of life across all industries to (hypothethically) benefit a single one? No thanks. Why do you feel grinding insane hours would be beneficial to AI progress?
- Cthulhu_ 4mo agoThey can be serious about home-grown AI without needing to become a libertarian capitalist hellscape. I prefer happiness, safety and privacy over competing with the US / China.
- WarmWash 4mo agoSure, but good luck finding serious top tier AI researchers who are willing to work for $80k/yr when the US is offering them upwards of $1M/yr.
- 4mo ago
- deleted 4mo ago[deleted]
- rahimnathwani 4mo ago[dead]
- jdw64 4mo agoHonestly, I used to think the 'sovereign model' was a waste of money. But recently, with the US logic of restricting model exports, I've come to think that if things go south, they could even cut off allied nations. So now the sovereign model seems reasonable to me. That, in turn, means US influence is deteriorating. And that probably isn't such great news for American businesses.
- Dwedit 4mo agoWhat really matters is the sovereign capability to finetune the LLM models. Any model could be vetted and tested, but you need finetuning/lora training to prevent the model from being outdated.
- rdwrrr 4mo agoBurning tax money. I dare to bet this will never lead anywhere.
- holistio 4mo ago"Burning" €13.5M of public funds. That's 4000 times less than the Cursor deal from a couple days ago. I was actually surprised by how little it was.
- entropyneur 4mo agoHow about fixing whatever the hell prevents competitive private LLM vendors from appearing in Europe?
- yanis_t 4mo ago> A total of €13.5 million has been allocated to the project. This is not even funny. If you want a competitive AI industry, you need to invest much more heavily in infrastructure first, building models second.
- deleted 4mo ago[deleted]
- alper 4mo agoEurope should have a sovereign model on its content and languages that is trained with renewable energy and published as open source. This looks like a good step in that direction.
- agrijakhetarpal 4mo ago"sOvErEiGn"
- mvdh1304 4mo agooverall, the revenue sharing model is (IMO) more interesting than the fact that it is dutch. Usage of data, and sharing it with the providers of this data, is an inherent part of the creation of these models that is not discussed as much as it should be
- lejeanvaljean 4mo agoBetter work on something at Europe level
- jgbuddy 4mo agoI fear sovereignty is not a adoption-driving feature
- bmenrigh 4mo agoI think at this point what the Netherlands, and any other country that wants a good model in their language should do, is gather up every piece of text ever written in that language and license it to the big AI labs/companies for training. I'm sure there are vast libraries of books and other text that haven't been digitized and aren't a priority for the big labs.
- tantalor 4mo agoYeah except replace "license it to the companies for training" with "pay the companies to train on it"
- bmenrigh 4mo agoOh I didn’t mean at all charging them. I mean licensing in the sense of granting rights for the purpose of training. Probably most labs would be fine adding the language to the training for free as long as the dataset quality is high and it improves the results. But yes, pay them if that’s what it takes for them to use it.
- whateverboat 4mo agoI think they should just make a national security thing and gather every piece of text in every language.
- Zababa 4mo agoI feel like building datacenters and filling them with chips may be more valuable than creating sovereign models. xAI I think makes more money renting datacenters to Anthropic than with the models they trained, and they could pivot thanks to their datacenters. By making regulations easier than in the US, this could bring some computing power to Europe, which then can be used to train sovereign models, or rented to big AI labs. Also, when training models, you create talent that then could go to other countries (brain drain). Restricting that brain drain without imposing authoritarian restrictions on the movements of people seems hard, so it seems hard to keep talent as a competitive advantage. If instead the competitive advantage is datacenters with chips, power capacity building, fast path to building datacenters, I think they are easier to retain while preserving the rights of everyone involved.
- iyia 4mo ago[flagged]