16 ms·
Rio de Janeiro's "homegrown" LLM appears to be a merge of an existing model
- flowbarai 4mo ago[flagged]
- unrvl22 4mo agoThe municipality of Rio de Janeiro (via its IT company IplanRIO) released Rio-3.5-Open-397B, presented as a homegrown Qwen3.5 fine-tune that beats comparable open models on benchmarks. The linked issue argues it's actually a weighted merge of ~60% Nex-N2 Pro + ~40% Qwen3.5-397B-A17B - Nex-N2 having been released about a week earlier.
- deleted 4mo ago[deleted]
- clear-octopus 4mo ago[dead]
- Lucasoato 4mo agoSo the problem isn’t in the missing attribution to Qwen, but with the fact that they didn’t mention Nex-N2 Pro right?
- Aurornis 4mo agoThe problem is that they claimed to have made a big achievement with their home grown post training, and they expected to receive a lot of praise for it. Then researchers looked at the weights and there is no post training at all. They are now attributing both models they merged, but their excuse for the lack of post training is to claim they accidentally uploaded the wrong files.
- serial_dev 4mo agoI’d believe they accidentally uploaded the wrong files if they uploaded the correct ones. To state that they accidentally uploaded something else and then not upload the correct version means they probably do not have anything and either hope people forget about this or they are scrambling to have something that is at least close to their original claim.
- evilduck 4mo ago"Oops, we uploaded the wrong files" is the standard deflection every time people like this get caught. Look up "Reflection 70B" drama.
- DonsDiscountGas 4mo agoI didn't know model merging like that was possible. (Obviously possible from a pure software standpoint but I'm surprised it's effective)
- bwhitty 4mo agoAs another poster above linked, it’s been shown to be effective since 2022: https://arxiv.org/abs/2203.05482 https://arxiv.org/abs/2203.05482
- nightpool 4mo agoit works because Nex N2 is also a derivative of the original base Qwen model. If it was two completely unrelated models it wouldn't work.
- hypercube33 4mo agoEven merging models with themselves as shown here in the post how they got to the top of hugging face with two gpus
- baobabKoodaa 4mo agoA few years back these used to be called "Frankenstein models"
- vasco 4mo agoRio better have the best IT infrastructure and software in the world if they are spending time on LLMs. What a waste of tax payer money.
- vitorgrs 4mo agoPiaui state it's also doing a LLM it seems. But indeed it would make more sense if it was a national thing rather than local...
- elzbardico 4mo ago[flagged]
- guiraldelli 4mo agoWithout evidence, your comment is just bad mouthing. I have been involved in academia, including in Brazil, and I don't find academia there any more copycat than any other institution, including top tier ones.
- boca_honey 4mo agoThis is very easy to prove [1][2]. Brazil has that reputation in the broarder academic world, and it's for a reason. [1] https://www.sciencedirect.com/science/article/abs/pii/S1751157722000396 https://www.sciencedirect.com/science/article/abs/pii/S17511... [2] https://www.scielo.br/j/aac/a/xNytDrrrHdyK4XPcHBRJZmd/?lang=en https://www.scielo.br/j/aac/a/xNytDrrrHdyK4XPcHBRJZmd/?lang=...
- avdelazeri 4mo agoOne study about faculty hiring people they know, and the other about high school students cheating on assignments... What was the original claim again?
- _3u10 4mo agoNo, typically Brazilians go to Paraguay for their education, most of their technology comes from Paraguay too.
- AnotherGoodName 4mo agoThis is fascinating that it worked though. Can we just merge all the open weight models and get something better?
- _3u10 4mo agoNo, they need the same arch, but you can distill them into a single model. And yes, if you use the API directly Claude will often say it’s an open weight model (likely the ones it was distilled from)
- wds 4mo agoI imagine it'd work the same as merging all the good-tasting foods to get an even tastier one
- booleandilemma 4mo ago[dead]
- dindunuf 4mo agothat kinda worked in llama 1/2 era, not between different models but between finetunes of the same model. the briefly legendary Mythomax was IIRC a merge of 5+ tunes, some of which were merges themselves.
- avereveard 4mo agomost merge improve a small subset of "feeling" benchmark (too small, too specific, or out of distribution) and tend to show degradation on actual benchmark, with especially punishing result on long chain benchmarks. also only work on matching architectures (i.e. finetunes/loras of the same model)
- nylonstrung 4mo agoIf you go to Civitai this is pretty how it works in that corner of the image generation world Everything is using Stable Diffusion as underlying model, then most of the usage is merged of checkpoints
- 4mo ago
- AlienRobot 4mo agoThe model's webpage at https://huggingface.co/prefeitura-rio/Rio-3.5-Open-397B https://huggingface.co/prefeitura-rio/Rio-3.5-Open-397B says it's a merge now. It previously didn't contain this paragraph: >The model is built via a merge of https://huggingface.co/nex-agi/Nex-N2-Pro https://huggingface.co/nex-agi/Nex-N2-Pro and https://huggingface.co/Qwen/Qwen3.5-397B-A17B https://huggingface.co/Qwen/Qwen3.5-397B-A17B, proceeded by On-Policy Distillation from a stronger model. We detected an incorrect upload in the previous version, where the base merged version was upload instead of the final distilled model. We are sorry for the confusion and apologize profusely. Incidentally are people using Github issues as blogs now?
- jonchurch_ 4mo agoEdit: I didnt even notice until someone pointed out this was on the Nex-n2 repo not the rio one, now I understand the OP’s confusion! It wasnt framed as an issue which is the norm breakage I think you’re reacting to, as in they didnt ask that the readme be updated etc, but it is common now for folks to use a project’s issue tracker to name and shame them in a place they cant easily ignore. Whether that’s right, prosocial, or professional is up for debate (as well as if any single definition of etiquette can be expected in 2026 on an issue tracker). But surely you can see the optics reason why someone would take their complaint to the repo directly? It pressures the maintainers to respond, it allows for a pile on from the internet, and makes any decision to lock down a hostile thread into its own kind of statement. The maintainers should absolutely post an official response and lock the thread though, it will likely get ugly in there.
- ChoosesBarbecue 4mo agoBut this is posted on Nex's GitHub, not on "Rio de Janeiro's" GitHub. i.e. this is the maintainer posting on their own GitHub Issues.
- zinodaur 4mo agoOh no, someone is profiting off of their work without proper attribution!?!?
- internet2000 4mo agoAttribution isn't the relevant part. Lying about your lab's capabilities is.
- Planktonne 4mo agoThat's also something all the AI companies have been doing.
- dofm 4mo agoLying about model capability is right now the lingua franca of the cloud AI business model, almost; they yes-and each other's lies because they are in a position of needing to generate interest, including going as far as needing to trigger regulatory capture. (It's not news to anyone who has worked in sales-led businesses that salespeople are prone to believing the claims of other salespeople, I guess).
- selcuka 4mo ago> Lying about model capability is right now the lingua franca of the cloud AI business model Lying about your lab's capabilities != Lying about model capability Exaggerating the capabilities of a new model that you've actually trained in press bulletins can be called marketing. Merging two models and claiming that you trained a new model is plain lazy.
- deleted 4mo ago[deleted]
- low_tech_love 4mo agoThey’re using public money to “train” this.
- alfiedotwtf 4mo agoWasn’t it already obvious given the awfully familiar parameter numbers?
- intoXbox 4mo agoThat only tells what base architecture they used, but fine tuning does not increase the number of weights, it just adapts the weights to improve better on a fine tuning dataset- something they claimed they had done
- ekjhgkejhgk 4mo agoOne funny thing about incompetence is that they don't have the competence to know that their incompetence is straightforward to verify by a competent person.
- root-parent 4mo agoYou just described every single vibe coder...
- vvpan 4mo agoI think that's unfair to "vibe coding". If anybody explicitly claims to vibe coding something than they are admitting to low supervision of the code. And on the contrary you can also AI-produce code that you have supervised highly. I suppose there are people who both AI their code and push it as bespoke but I, for one, have not met such a person at our outside of work.
- root-parent 4mo ago>> but I, for one, have not met such a person at our outside of work. https://news.ycombinator.com/item?id=48516679 https://news.ycombinator.com/item?id=48516679
- carlosjobim 4mo agoWhy would they care? They get their salaries and pensions and bonuses, and the tax payer is footing the bill.
- thimabi 4mo agoI wouldn’t describe what happened here as incompetence. As a “carioca”, I am pleasantly surprised to know that the government’s IT department is involved in AI work — even without the budget to create its own models from scratch.
- arcticfox 4mo agoThis seems kind of insane though, every time I go to Rio I think of the potential of AI/technology to solve some problems and leave it even more paradisiacal... But working on their own model? Wtf? There are a million applications of existing ones there that should be followed up on instead.
- MadrasTh0rn 4mo agoNot surprised
- nom 4mo agowhy not?
- diego_moita 4mo agoIt is a recurrent Brazilian meme: Rio is known in Brazil as "terra de bandido" (gangster's land). The majority of their politicians have ties to organized crime. There is a virtual revolving door between police and crime, where people migrate from one to the other. It is like Chicago in the 20s, Naples and Medelin in the 80s or Moscow and Culiacan (Sinaloa, Mexico) today.
- alexgoodhart 4mo agoSomehow I doubt that political affiliations with crime syndicates are affecting heavily the dispositions of LLM developers. The industry itself though is one of incest.
- afh1 4mo agoHe is putting into question the character of the public workers involved in the project, not that it has anything to do with organized crime. Rio has relapsed into crime in the last decades and government workers in general have a reputation for corruption in Brazil. It's a low trust society specially north of Parana hence the lack of surprise.
- sebastianconcpt 4mo agoPoliticians don't come from outer space, they emerge locally and were raised swimming in an imaginary that has normalized the morals that eventually end up expressed at the top.
- dormento 4mo agoRio is kinda funny as a litmus test - federal government creates laws to try and curb some of the corruption, and Rio produces better and better corrupts - so far Rio is winning. BTW wasn't it a few months ago the current governor wanted to leave to be able to run as a candidate, so he asked a supreme justice to step in in as governor, since there wasn't anyone else that technically could?
- fkozlowski 4mo agoI'm honestly surprised that they even had the inclination to attempt creating a model. I guess it's bullish that a municipal IT department had the guts to try this?
- Havoc 4mo agoMerges and fine tunes are within reach of individuals with some money to burn so I’m sure a muni can do it
- axus 4mo agoI like the [dead] comment theory that they proposed a huge LLM training budget to the government, kept most of the money, and released a cheap merge to justify the grift.
- matheusmoreira 4mo agoThat's essentially Brazil's standard operating procedure. Wouldn't be surprising if that turned out to be the case. Still, I'm actually impressed that this even happened at all. "Rio de Janeiro's homegrown LLM" is the last headline I expected to read on HN.
- seba_dos1 4mo agoIt's kinda weird to claim extraordinary results in such case though, as that brings a lot of eyes to it.
- mgambati 4mo agoNothing weird. The mayor wanted something brag about. That Rio, my friend.
- dormento 4mo agoThis would be so very brazilian of them. Source: am Huelander.
- fkozlowski 4mo agoAh that makes sense
- yieldcrv 4mo agoDidn’t the last thread about this have someone from the lab or an enthusiast in Rio saying exactly that? Its a fine tune of Qwen Not a conspiracy
- daemonologist 4mo agoThe allegation here is that it's not actually a fine-tune of Qwen, but instead an undisclosed mashup (merge) of someone else's fine-tune of Qwen and the original model. Rio subsequently said that the model was in fact a merge, that they did additional fine-tuning after the merge, and that they accidentally uploaded the base merge instead of the version with additional fine-tuning. But this seems like quite an oversight...
- yieldcrv 4mo ago> But this seems like quite an oversight... Not to me, what would people like to happen? Who are those people? And why do they care?
- antonvs 4mo agoThey made a public claim to having produced a useful model, which they published. Turns out they did nothing of the sort. > why do they care? Why does anyone ever care about having their time wasted by fraudulent claims?
- yieldcrv 4mo agoContinue to explain like I’m 5 instead of the rhetoricals
- Aurornis 4mo ago[dead]
- antii 4mo ago[dead]
- jrm4 4mo ago“Well, Steve (Jobs), I think it’s more like we both had this rich neighbor named Xerox, and I broke into his house to steal the TV set, but I found out that you had already stolen it.” -- Bill Gates
- wunderlotus 4mo agolmao i really hope this is a real quote cuz it’s a banger
- ckcheng 4mo agoApparently: https://www.folklore.org/A_Rich_Neighbor_Named_Xerox.html https://www.folklore.org/A_Rich_Neighbor_Named_Xerox.html
- ckcheng 4mo agoWhat’s more funny to me is the set up to that quote: > Bill Gates had somehow manifested, alone, surrounded by ten Apple employees. … Steve started yelling at Bill, asking him why he violated their agreement. And what’s more interesting is the conclusion: > Apple filed a monumental copyright lawsuit against Microsoft in 1988, but they eventually lost on a technicality (the judge ruled that Apple inadvertently gave Microsoft a perpetual license to the Mac user interface in November 1985). Microsoft didn’t steal Apple’s GUI … Apple gave it to them.
- alexgoodhart 4mo agoThat isn’t fully true is it? Microsoft claimed that its software’s use of various visualizations related to window state was covered by the 1985 agreement, and Apple claimed that this was not true; those window states were produced by Macintosh while Microsoft’s software was being rendered in the Mac environment. > In his March 20, 1989 Order, Judge Schwarzer declined to consider whether the visual displays in issue were generated by the Microsoft application programs or by the Macintosh system software. The point arose in connection with Microsoft's argument that the 1985 Agreement licensed to Microsoft all visual displays that could possibly be called up by running the five Microsoft application programs on the Macintosh system software then or in the future. 709 F. Supp. at 929. Judge Schwarzer concluded that Microsoft's contention would "defy common sense." Id.
- deleted 4mo ago[deleted]
- deleted 4mo ago[deleted]
- hintymad 4mo ago> Every weight tensor in Rio is, to thousands of standard deviations, the same 0.6/0.4 blend of Nex and Qwen — across all 60 layers and every component of the network. Other finetunes cannot be explained as interpolations. I find it amazing how robust the current deep learning models are. A simple linear combination of every weight did not degrade the performance of the model, but enhanced it.
- deleted 4mo ago[deleted]
- woadwarrior01 4mo agoIt's is a well known idea[1], although it's still surprising that something as simple, even works. [1]: https://arxiv.org/abs/2203.05482 https://arxiv.org/abs/2203.05482
- kolanos 4mo agoThis team could have stopped here and still had something interesting (albeit not novel) to show. But the hype cycle was too tempting.
- randall 4mo ago[dead]
- x312 4mo agoThis works because Nex itself is a finetune of Qwen3.5 (https://huggingface.co/nex-agi/Nex-N2-Pro https://huggingface.co/nex-agi/Nex-N2-Pro). It's merging Qwen3.5 with a Qwen3.5 finetune. I don't believe this would work on two LLMs that have different pretraining. Even if it did you would need two LLMs that have exact same internal activation shapes, dimensions, expert counts, token vocabulary, realistically it would never happen outside of finetunes or academic experiments.
- oofbey 4mo agoCorrect. We used to think that because NN optimization is non-convex there are all these local minima. Now we know that once you get past the very early parts of training from random init, the loss surface is fairly smooth, and not really convex, but close enough in a bunch of ways - linear combinations of trained models are pretty much always valid combinations. You can think of fine tunings as deltas on the original model which can be summed together successfully. I think this paper first showed that to me: https://arxiv.org/pdf/1802.10026 https://arxiv.org/pdf/1802.10026 which was 8 years ago now.
- jordz 4mo agoCan someone please explain or link to some information about how models are merged? Is this genuinely merging weights mathematically or some kind of distillation (presumably not if they’ve done zero training as the post suggests).
- calebkaiser 4mo agoThis is a good starting point: https://huggingface.co/docs/peft/developer_guides/model_merging https://huggingface.co/docs/peft/developer_guides/model_merg... But yes, in general, merging refers to techniques that directly blend the weights of different models mathematically. It had a big moment of popularity ~2 years ago, with many so-called "Frankenmodels" popping up on leaderboards. I tend to think of merging as belonging to the same general umbrella as things like "abliteration", or other techniques that surgically modify the weights of a model without a traditional training/tuning loop. Maxime Labonne is a great person to follow if you're interested in this general area.
- jxmorris12 4mo agoThere’s nothing to read. Model A: A_1, …, A_n Model B: B_1, …, B_n C_i = A_i * p + B_i * (1 - p) In other words, it’s just a linear combination of the other models’ weights, per position.
- joe_the_user 4mo agoIt's been a while since I looked at neural networks in detail. Do all the large models have a close enough architecture that this makes sense? Do they have the same number of layers and width? I had thought that each model it's own "secret sauce" of normal and special layers (convolution, max-pooling, something-something) stacked together. Genuinely curious.
- diego_moita 4mo agoWHAT!? There are thieves in Rio de Janeiro? Oh, I am so SHOCKED, so SHOCKED! /s Explaining the joke: in Brazil, Rio de Janeiro is known as "Terra de bandido" (Gangster's Land). Kinda like Chicago in the 20's or Naples and Palermo in the 90s.
- Havoc 4mo agoNex in turn is also based on qwen so don’t think they’re too far off
- deleted 4mo ago[deleted]
- delusional 4mo agoIt's absolutely insane to me that we are now at a point where the top of the front page of hacker news is a random GitHub issue about attribution to some random LLM merge, written in just the most disgusting AI slop style. I would like to downvote this please.
- vor_ 4mo agoThere's been a noticeable drop in quality. It's often a blend of AI culture war posts and arbitrary Github links.
- Scroll_Swe 4mo ago[flagged]
- vvpan 4mo agoGross historical ovesimplifications aside I am wondering why one would use this as an opportunity to belittle a whole continent.
- Scroll_Swe 4mo agoSome continents are just better, simple as.
- antonvs 4mo agoHave you ever heard of literature?
- Scroll_Swe 4mo agoYes
- antonvs 4mo agoLook into the book "100 Years of Solitude".
- rafaquintanilha 4mo agoI have no affiliation with them but here's what I think happened: 1. They claim the official model is based on Qwen 397B. It's likely they didn't disclose Nex Pro at all because Nex itself is based on the same base model (not saying they shouldn't). 2. The improvement would come from merging the weights PLUS on-policy distillation. The confusion is that the uploaded model didn't have the distillation at all. 3. It's important to notice they didn't advertise the model besides posting it on Reddit 2 days ago. It became viral organically, over the weekend, and during Brazil's World Cup debut (Brazilians will understand). Of course the mayor of Rio took the opportunity to capitalize over the free coverage, but that wasn't done in conjunction with the researchers. 4. I don't see why they would disclose Qwen 397B as base and mention the SwiReasoning paper but not mention Nex if all they did was to merge both models. 5. In any case, what they are claiming is easily verifiable once (if) they upload the right model.
- throwa356262 4mo agoRegarding #2 https://news.ycombinator.com/item?id=48529544 https://news.ycombinator.com/item?id=48529544
- xiphias2 4mo agoThis should be at the top: they uploaded the wrong model, they fixed it
- jwitthuhn 4mo agoThey did upload the wrong model but as of the time of writing they have not fixed it. Right now, 12 hours after they took the old one down, there is simply no model present in their huggingface repo.
- xiphias2 4mo agoI guess they will upload it later, it seems like an honest mistake to me. Anyways SwiTransformer paper looks interesting and doing a post training to optimize for it looks interesting as well.
- pelasaco 4mo agoan eternal 7x1.. and I am not talking about Curaçao..
- hottrends 4mo ago[flagged]
- aaronbrethorst 4mo agoThey really missed out by not calling it Neuromancer.
- thelonelyborg 4mo agothis is probably occurring all over the world including in startups.
- jing09928 4mo ago[flagged]
- PixComicOS 4mo ago[flagged]
- FooBarWidget 4mo agoCan anyone explain to me what a merge is and why that works? It seems utterly bizarre to me that you can just merge weights. You can't make a working program by just merging machine instruction pages. Aren't weights tightly coupled to a specific architecture?
- antonvs 4mo agoIn this case both sets of weights ultimately came from the same model. The Nex model they used is a fine-time of Qwen, which was the other model they used. I'm not an expert in this area, but it's not too hard to see how a merge like that could turn out ok.
- jkwang 4mo agoThis is a concerning pattern. Rebranding merged models as "homegrown" without disclosure undermines trust in open-source AI development. The community needs better provenance tracking and transparency standards for model releases.
- blitzar 4mo agoIts stupid and hilarious when someone in Rio does it; when a techbro in silicon valley does it they get VC funding, a maserati and an entry on the 30 under 30 list.
- rgbrth 4mo agoI don't think people are saying it's stupid. It's just funny that potentially some random municipality worker is going well beyond their work scope and making contributions in the AI world. Could be from Rio, could be from any municipality anywhere in the world. The fact that the account is actually from the town hall rahter than a personal account also makes it funnier.
- rsynnott 4mo ago> and an entry on the 30 under 30 list. Ah, yes, the Nobel Prize for Fraud. (I'm seriously kind of amazed they're still publishing those.)
- RandyOrion 4mo agoPlease do not claim you trained a new model, only to got caught red-handed by others. There are already several people or groups did that, got caught, and vanished in no time. Check how the "authors" of "this model" react to this problem [1]. See how they deal with this problem by first changing their affiliation from https://iplanrio.rio.rj.gov.br https://iplanrio.rio.rj.gov.br to https://iplanrio.prefeitura.rio https://iplanrio.prefeitura.rio [2], then saying that they are sorry for being caught [3], then just remove all their affiliations once for all [4]. I think the "authors" of "this model" [5] should be held accountable until they upload new checkpoints, and the performance of the new model is verified by third-parties. P.S. To people who downvoted me, show me why you're doing this. [1] https://huggingface.co/prefeitura-rio/Rio-3.5-Open-397B/commits/main?p=2 https://huggingface.co/prefeitura-rio/Rio-3.5-Open-397B/comm... [2] https://huggingface.co/prefeitura-rio/Rio-3.5-Open-397B/commit/cd503cdc7aaac2506bf68f6d278503ddb07a83ef https://huggingface.co/prefeitura-rio/Rio-3.5-Open-397B/comm... [3] https://huggingface.co/prefeitura-rio/Rio-3.5-Open-397B/commit/a778c1ec4e21180ee55c3ea016a348e549e75f09 https://huggingface.co/prefeitura-rio/Rio-3.5-Open-397B/comm... [4] https://huggingface.co/prefeitura-rio/Rio-3.5-Open-397B/commit/44a2a981870907b0dbe3983017a0e5455c2acbd4 https://huggingface.co/prefeitura-rio/Rio-3.5-Open-397B/comm... [5] https://huggingface.co/prefeitura-rio https://huggingface.co/prefeitura-rio
- nicman23 4mo agois it any good?