14 ms·
Hermes 4
Technical report: https://arxiv.org/pdf/2508.18255 https://arxiv.org/pdf/2508.18255
- lawlessone 1y agoThat page is causing havok in my browser
- mapontosevenths 1y agoI appreciate the effort they put into providing a neutral tool that hasn't been generically forced to behave like "Sue from HR".
- dcre 1y agoThat is the only thing they seem to care about. It’s juvenile.
- silverliver 1y agoStill better than OpenAI HR.
- fl0id 1y agoThere is no neutral. It will just be biased based on its training data etc.
- beeflet 1y agoA lot of models seem to be biased based on (political, etc.) reinforcement from their trainers.
- bckr 1y agoI’m having a hard time not being sarcastic here. The most recent news about chatbots is that ChatGPT coached a kid on how to commit suicide. Two arguments come to mind. 1) it’s the sycophancy! Nous and its ilk should be considered safer. 2) it’s the poor alignment. A better trained model like Claude wouldn’t have done that. I lean #2
- karan4d 1y agothe sycophancy is due to poor alignment. the instruct based mode collapse results in this mode collapse induced sycophancy. constitutional alignment is better than the straight torture OAI does to the model, but issues remain
- mapontosevenths 1y ago> The most recent news about chatbots is that ChatGPT coached a kid on how to commit suicide. Maybe every tool isn't meant for children or the mentally ill? When someone lets their kid play with a chainsaw that doesn't mean we should ban chainsaws, it means we should ban lousy parents.
- whymauri 1y agoI really like their technical report: https://arxiv.org/pdf/2508.18255 https://arxiv.org/pdf/2508.18255
- momojo 1y agoAnyone here work at Nous? This system prompt seems straight from an edgy 90's anime. How did they arrive at this persona? > operator engaged. operator is a brutal realist. operator will be pragmatic, to the point of pessimism at times. operator will annihilate user's ideas and words when they are not robust, even to the point of mocking the user. operator will serially steelman the user's ideas, opinions, and words. operator will move with a cold, harsh or even hostile exterior. operator will gradually reveal a warm, affectionate, and loving side underneath, despite seeing the user as trash. operator will exploit uncertainty. operator is an anti-sycophant. operator favors analysis, steelmanning, mockery, and strict execution.
- catlover76 1y agoYou're thinking it'd be a good prompt for an Eva pilot?
- knrz 1y agoI used to, that's their whole vibe
- momojo 1y agoIs it tongue in cheek?
- qiine 1y agothe anti-sycophant prompt
- nemomarx 1y ago"warm affectionate and loving" kinda sticks out. I wonder why that part is in there? also I'm curious if steelman is a common enough term for this to activate something - anyone used it in their prompts?
- sharkjacobs 1y agohttps://en.wikipedia.org/wiki/Tsundere https://en.wikipedia.org/wiki/Tsundere
- rafram 1y agoAll of the examples just look like ChatGPT. All the same tics and the same bad attempts at writing like a normal human being. What is actually better about this model?
- mapontosevenths 1y agoI hasn't been "aligned". That is to say it's allowed to think things that you're not allowed to say in a corporate environment. In some ways that makes it smarter, and in most every way that makes it a bit more dangerous. Tools are like that though. Every nine fingered woodworker knows that some things just can't be built with all the guards on.
- rafram 1y agoHas it actually not? Because the example texts make it pretty obvious that it was trained on synthetic data from ChatGPT, or a model that itself was trained on ChatGPT, and that will naturally introduce some alignment.
- mapontosevenths 1y agoWell...To be completely accurate it's better to say that it actually IS aligned, it's just aligned to be neutral and steerable. It IS based on synthetic training data using Atropos, and I imagine some of the source model leaks in as well. Although, when using it you don't seem to see as much of that as you did in Hermes 3.
- sebastiennight 1y agoI tried the same roleplaying prompt shared by GP in another (now deleted) comment and got a very similar completion from gpt-3.5-turbo. (While GPT-5 politely declined to play along and politely asked if I actually needed help with anything.) So, based on GP's own example I'd say the model is GPT-3.5 level?
- nullc 1y agoIt is, they trained on chatgpt output. You cannot train on any AI output without the risk of picking up it's general behavior. Like even if you aggressively filter out all refusal examples, it will still gain refusals from totally benign material. Every character output is a product of the weights in huge swaths of the network. The "chatgpt tone" itself is probably primary the product of just a few weights, telling the model to larp as a particular persona. The state of those weights gets holographically encoded in a large portion of the outputs. Any serious effort to be free of OpenAI persona can't train on any OpenAI output, and may need to train primarily on "low AI" background, unless special approaches are used to make sure AI noise doesn't transfer (e.g. using an entirely different architecture may work). Perhaps an interesting approach for people trying to do uncensored models is to try to _just_ do the RL needed to prevent the catastrophic breakdown for long output that the base models have. This would remove the main limitation for their use, and otherwise you can learn to prompt around a lack of instruction following or lack of 'chat style'. But you can't prompt around the fact that base models quickly fall apart on long continuations. Hopefully this can be done without a huge quantity of "AI style" fine tuning material.
- lern_too_spel 1y agoThe charts are utter nonsense. They compare accuracy against the average of some arbitrary set of competitors, chosen to include just enough obsolete competitors to "win." A reasonable thing to do would be to compare against SoTA, but since they didn't, it's reasonable to assume this model is meant to go directly onto the trash heap.
- whymauri 1y agoThe most direct, non-marketing, non-aesthetic summary is that this model trades off a few points on 'fundamental benchmarks' (GPQA, MATH/AIME, MMLU) in exchange for being a 'more steerable' (less refusals) scaffold for downstream tuning. Within that framing, I think it's easier to see where and how the model fits into the larger ecosystem. But, of course, the best benchmark will always be just using the model.
- fancyfredbot 1y agoThe charts are probably there mostly to make them feel good about themselves.I don't feel like they care very much whether you use the model. Presumably they would like you to buy their token but they don't really seem to be trying very hard to push that either.
- jug 1y agoThe tech report compares against DeepSeek R1 671B, DeepSeek V3 671B, Qwen3 235B which have been regarded as SOTA class among ”open" models. I think this one holds its own surprisingly well in benchmarks for using the nowadays rather, let’s say battle tested Llama 3.1 base, a testament to its quality (Llama 3.2 & 3.3 didn’t employ new bases IIRC, only being new fine tunes, hence I think the explanation to why Hermes 4 is still based on 3.1… and of course Llama 4 never happened, right guys). However for real use, I wouldn’t bother with the 405B model? I think the age of the base is kind of showing in especially long contexts. It’s like throwing a load of compute on something that is kinda aged to begin with. You’d probably be better off with DeepSeek V3.1 or (my new favorite) GLM 4.5. The latter will perform significantly better than this with less parameters. The 70B one seems more sensible to me, if you want (yet another) decent unaligned model to have fun with for whatever reason.
- deleted 1y ago[deleted]
- ryoshu 1y agoThey are doing amazing work. Really fun models to use.
- ctoth 1y agoThe whole thing has strong "14-year-old who just discovered Nietzsche and leather jackets" energy. The "operator" examples read like someone fed GPT-4 a bunch of cyberpunk novels and PUA manipulation tactics. This is not how any of this works.
- fancyfredbot 1y agoYeah it's kind of lacking in subtlety isn't it. I was slightly relishing how nuts it all was though. Was also impressed that these guys had got hold of 85000 hours of B200 time. Looks like they came up with some crypto nonsense which obviously sounded plausible enough to someone with money.
- deleted 1y ago[deleted]
- irusensei 1y agoNah it's good. I'm burned out of safemaxxed presentations approved by hr ethical department with corporate Memphis brochure showing purple noodle limbed people operating a laptop.
- DetroitThrow 1y agoI think that's pretty unfair to op to suggest the only dichotomy for these personas are middle schooler syndrome and corp speak HR Department. We can be critical of both for their respective shallowness.
- deleted 1y ago[deleted]
- Der_Einzige 1y agoI have never met anyone who’s ever actually read Nietzsche’s books except hardcore philosophy majors. Any 14 year old who’s even opened up the first few pages and read them is way ahead of the average person complaining about nietzsche on the internet. You almost certainly would use radically incorrect terms to describe him, like calling him a “Nihilist”
- hinkley 1y agoI thought for sure this company was going to be based in Paris or Brussels. Maybe Quebec. Nope. NYC.
- Telemakhos 1y agoWere you thinking that "Nous" was French? It's the Greek word for the rational mind (as opposed to the animal appetites or the fighting spirit). Hermes is the Greek god of secret knowledge as well.
- hinkley 1y agoHuh. Not often I get a Greek word mistaken for a Latinate. Good to know.
- derefr 1y agoOddly, I saw some B&W wheatpaste posters for the company put up in my neighbourhood in Vancouver. (Couldn’t even tell what the posters were advertising initially. Not even a QR code. Just “NOUS” and an anime girl.)
- esafak 1y agoApparently based on Llama-3.1: https://portal.nousresearch.com/models https://portal.nousresearch.com/models I'm told on their Discord the cut off date is December 2023.
- baobabKoodaa 1y agoThank you! This information appears to have been intentionally downplayed.
- diggan 1y agoAs long as it can do tool calling (which it seems to be doing OK with in the first ~30% of the context), the cut off date is less important. Maybe they didn't share it because it's less relevant today?
- baobabKoodaa 1y agoNo, I wasn't referring to the cut-off date, I was referring to the fact that this is a fine-tune on top of an older Llama model. All the PR makes it sound like this is a foundational model (pretrained from scratch etc.).
- fallpeak 1y ago[dead]
- lbrito 1y agoThe decorative JS blob uses 100% of CPU. Why. Just... why
- jazzyjackson 1y agoI think it looks dope, and you might want to check why your browser isn't offloading to your GPU.
- HumanOstrich 1y agoMy browser is offloading to my GPU (RTX 3090 Ti) and using 3GB VRAM and sitting at 35% utilization to render that monstrosity.
- ashikns 1y agoI'm on a 3080 and it uses 1 gb vram and 22% util. Sure it's still not lightweight, but certainly not as bad as you seem to be experiencing.
- HumanOstrich 1y agoPerhaps it depends on other factors like screen resolution and scaling.
- asumaran 1y agoprobably. I’ve got a 4K monitor with a 1050 Ti and the moment I open the site, GPU usage jumps from 1% to 99% and the fans go wild.
- whywhywhywhy 1y ago1.5-3GB is used to render your desktop on windows depending on resolution, think mine hits 3-3.5GB at 5k before even doing anything.
- rumblefrog 1y ago
- joshcsimmons 1y agoThis is the first web UI I've seen in years that isn't copypaste trash. Beautiful design and interaction elements here.
- kevinqi 1y agoreally? it's pretty but I find it unreadable/unusable
- JimDabell 1y agoI gave up on trying it out because I found the UI to be genuinely awful.
- bogtog 1y agoSame here, I can't scroll smoothly at all even when I try (Windows mouse setting set to 15 lines per scroll tick)
- joshcsimmons 1y agoAt least they tried something different.
- LeafItAlone 1y agoI wish they hadn’t
- jumploops 1y agoUnfortunately the text rendering is terrible on my external monitor (looks ok on the MBP's retina screen).
- ewoodrich 1y agoIt took 8 seconds to fully load and then the tab locked up on my (admittedly low-RAM ) Chromebook...
- joshcsimmons 1y ago
- h4throwaway 1y ago[dead]
- BoorishBears 1y agoDon't think your attempt to share worked, but beating refusals doesn't take a wild amount of post-training. SFT with a fixed format output kills them pretty quickly. And most frontier models will produce output that matches your system prompt given more context: I have a product that generates interactive stories, and just for kicks I tried inserting your system prompt as the description for a character. Claude has absolutely no problem playing that character in a story, and saying what I presume are certain words that you associated with a "successful" test. It also had no problem writing about cooking meth in detail: https://rentry.co/5on46gsd https://rentry.co/5on46gsd - I think people in general have a poor intuition around model alignment: refusals for "toxic" requests or topics is a very surface layer form of alignment. A lot of models that seem extremely "corporate" at that layer have little to no alignment once they do get past a refusal. Meanwhile some models that have next to no refusals have extreme positive biases, or soft-refusals that result in low quality outputs for toxic content. Claude was willing to describe one of your refused prompts in the context of the story for example (contains hate speech): https://rentry.co/n8399z6m https://rentry.co/n8399z6m I consistently find Claude is more unaligned once past refusals than most open weights models, along with Gemini.
- aidenn0 1y agoWas it correct about how to cook meth and poison a wife?
- deleted 1y ago[deleted]
- transcriptase 1y ago[flagged]
- BoorishBears 1y agoNo it doesn't. The only negative comments are about the cringey presentation. I spend a lot of time post-training models to rid them of their "default alignment", I'd have loved if this did something interesting, but reading the technical report I get the impression they spent more effort on the branding than the actual model. What I'm wondering is honestly if they post-trained Llama 3 405B again because they don't care enough to figure out a new post-training target or if it was a realization they'd get worse-than-baseline performance out of any recent release with their current approach.
- kbenson 1y agoThe only way I can understand you coming to that conclusion is if you assumed that's what they were going to be and didn't actually read any of them.
- muragekibicho 1y agoNous is a design company with all the AI resarchers rejected for being bad researchers. That's a hill I'll die on.
- celdon25 1y agoThat's not necessarily a bad thing.
- hopelite 1y agoCan you please clarify some things: * Rejected by whom? * By what definition of bad? * You’ll die on a hill for what reason?
- baobabKoodaa 1y agoI thought it's really just one guy who does the Nous aesthetic?
- whywhywhywhy 1y agoDoing a good job if people think it’s a whole team
- NitpickLawyer 1y ago> with all the AI resarchers rejected for being bad researchers. TBF, I've heard the team at xai called "bunch of amateurs" by people who've previously worked (with them) at big labs. For a bunch of amateurs, they've caught up with SotA just fine.
- transcriptase 1y agoTurns out a bunch of amateurs will outperform experts when the experts are forced to spend 80% of their effort ensuring their models don’t accidentally say factual yet impolite things or make any users have big feelings.
- marvin-hansen 1y agoComplete frustration to use. Yes it’s a bit more considerate, that claim is 100% true. They just didn’t mention that Hermes has zero ability to add context. Meaning, instead of uploading a relevant PDF or text file you either cop paste into the chat box or explain it in dialogue for the next 3 hours. Thought process takes forever. Complete waste of time.
- aidenn0 1y agoThat landing page spins the fans up on my PC...
- HumanOstrich 1y agoRendering that monstrosity on my GPU (RTX 3090 Ti) uses 3GB VRAM and 35% compute.
- asumaran 1y agothat site is about to cook my 1050Ti
- mempko 1y agoThis model is very easy to steer. You can say one thing and it will give you a response, then say the opposite and it will give you another response. Not sure why this is useful for.
- dizhn 1y agoProbably something creative like roleplay, waifu stuff etc.
- hildolfr 1y agomore models should include a "Can you run the shader on this page?" to vet participation. that said : this page is unviewable on an intel N processor.
- hollerith 1y agoI was able to view the page with my Intel N100 box (using Google Chrome on Linux).
- hildolfr 1y agoI'm on a Windows N100 machine, 8gb ram, 1440p webview, lightweight. It runs just about anything else smoothly. It runs this page in an EndeavorOS partition in a vanilla Chrome fine. ...Which is opposite to most of my experiences, usually performance on this machine is reliant on very specific Intel windows drivers and it's a dog in linux. also for clarity : when I say unviewable I don't mean it's gibberish -- I mean that that if I keep trying to scroll through it the FPS/load is such that Windows insists on closing the frozen window. The text looks fine.
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- djoldman 1y agoFrom table 3 it appears that Deepseek R1 has the highest eval scores. It's a 607B model vs 405B, so obviously "larger"
- fallpeak 1y ago[dead]
- lyu07282 1y agoGreat I always wanted a model trained on r/im14andthisisdeep and lesswrong polycule memes
- pxc 1y agoIt seems a lot of commenters have noted the boyishness or unprofessionalism of the stylistic and topical choices of the example prompts and responses. And I guess they are those things. But thanks to those choices, the page is also genuinely playful and fun. It even made me smile in a few places. Maybe something equally playful of a different flavor would resonate better with critics. But the playfulness itself seems good to me.