7 ms·
Cognition launches new SWE-2 model, Rivaling Fable 5.1 and GPT-Astra
- nujan_dev 2mo ago[dead]
- mydreamof 2mo agoSeems like benchmaxing? For example for Terminal-Bench 4 it doesn't have great results. And why not show other benchmarks?
- harmonic18374 2mo agoProbably, FrontierCode is made by Cognition itself. The model also seems worse in every way than DeepSeek v4.1 Flash, launched today. Also the submitter's account is very new which makes me suspicious of self-promotion.
- _doctor_love 2mo agoSWE-1.5 was surprisingly good when I used it last. I feel like Cognition is one of the solid players that’s flying a bit under the radar while Anthropic and OpenAI race to IPO.
- _doctor_love 2mo agoOdd to get downvotes simply for sharing my experience. Like it or not, Cognition has a good frontier model and they are building serious products. They are working hard which is how you become successful. Sorry if that ruffles your feathers.
- Tsarp 2mo ago"SWE-2 is post-trained from Kimi K3"
- airstrafer 2mo agoYeah, I'd expect model performance to be super spiky on SWE work, at least they admit it with the name of the model. It's distilled from an already-distilled model. Maybe still worth it if their "64% cheaper" figure holds.
- Tsarp 2mo agoWith the Devin subscription even at the 20$ plan, they offered unlimited SWE 1.7 usage. Wondering if they do the same for SWE 2.
- Bolwin 2mo agoI don't think you know what distill means
- airstrafer 2mo agoI guess I don't. Does post-training from another (larger) model not fall under the umbrella of distillation? I'd imagine it leads to the same spiky-ness issues...?
- FergusArgyll 2mo agoDistilling you don't have the actual model weights of the teacher. All you have are the teachers answers to a lot of questions. You then teach your own smaller model to answer more similarly to the big teacher model. Fine tuning you have the actual model weights of the original model, you then train that model to answer in a different (or better) way.
- scronkfinkle 2mo agoPlease correct me if I'm wrong, but this appears to require Devin to use? I'm disappointed to see I need to use a bespoke platform to interact with this agent, to the point that I probably won't be trying it.
- samyok 2mo agoSWE-2 is free to use for users like yourself for the next month, and almost all usage should be supported via our CLI (https://docs.devin.ai/cli https://docs.devin.ai/cli) :) Disclaimer: I work at Cognition, although was not involved in SWE-2
- scronkfinkle 2mo agoBut I don't want to use your CLI. I already have my own harnesses and workflows. The friction is too high to "just try out" a new model like this. It would be preferable if I can evaluate it over, say, open router like all the other models and then decide from there if it's worth downloading a bespoke tool chain for only 1 lab's models
- jkelleyrtp 2mo agoIt’s preferable to keep inference capacity available for users using main Devin products than openrouter atm. Might change in the future. Even OAI is cutting off new plan signups to keep up with demand.
- randomblock1 2mo agoI just gave it a try and it doesn't appear to be free, it used up some of my on demand usage. It does say 75% off though. Seems like for Pro subscribers SWE-1.7 is free, maybe SWE-2 is free for them?
- samyok 2mo agoDid you use it via the CLI or Desktop? It's 75% off in cloud and free to run on your device.
- monkeydust 2mo agoAs an Econ graduate, pretty cool seeing Pareto in the "AI-bro" zeitgeist. Slightly surreal watching a 1906 welfare economics idea get rediscovered as a plotting convention. The original, if anyone fancies 579 pages of Italian: https://archive.org/details/manualedieconomi00pareuoft https://archive.org/details/manualedieconomi00pareuoft. There is an English translation somewhere.
- arrowleaf 2mo agoPareto leaked out of the sociology/econ bubble a long time ago :) Pareto principle, Pareto efficiency, Pareto distribution have been in the pop-sci buzzwords for quite awhile, I probably encountered it first in the 4-Hour Workweek. I don't think you can read a self-help book without the author introducing it as a groundbreaking principle to live your life by.
- ccapitalK 2mo agoPareto frontiers are pretty commonly invoked to describe tradeoffs in computer science and have been for quite a while. I remember the term being used in one of my early algorithms courses to describe the tradeoff between data structures with fast writes, ones with fast reads and ones that tried to balance the two. IIRC cognition boasted about hiring a lot of competitive programmers and algorithms experts back when they released Devin, so it tracks that they'd use the term.
- walrus01 2mo agoIt's interesting watching people throw about pareto frontiers sort of like how RF nerds approach the shannon limit (in a practical real world sense of the term, like charting possible modulations/data rates on a two way satellite modem's manufacturer datasheet).
- postalcoder 2mo agoIf you're looking for reason to be skeptical, look no further than the massive delta between the Terminal Bench 2.1 (92.8%) and the Terminal Bench 4 score (27.3%). Terminal Bench 4 was released a couple weeks ago, so the difference you're seeing between the two scores can be interpreted as "how well does this model generalize to new problems"? More crudely: "how benchmaxxed is this model?"
- enraged_camel 2mo agoYeah, this echoes my thoughts. I will be very surprised if a model with 2.8T parameters reaches the intelligence and capabilities of 10T parameter models. RL can take things far, but not that far.
- nullbio 2mo agoClosed weights AND benchmaxxed. Somehow this company raised 2bil at a 48bil valuation. Pure insanity. I feel bad for their investors (not really, but... Still). Andreessen Horowitz is being played like a fiddle.
- thereitgoes456 2mo agoThe Cursor acquisition shows that it’s possible for these valuations to be justified. But Cursor was more successful and bent the truth much less. While I wouldn’t expect anything good for Cognition’s fate, it’s a much safer bet than Thinking Machines, SSI, and some others. Though they’ll be in big trouble if the more talented Chinese labs stop letting them repackage their work.
- selectodude 2mo agoThe cursor acquisition just shows that there’s always a dumber shithead out there. Though vaporizing Elon Musks money is about as pure of a good as there is out there these days.
- 2mo ago
- TheJCDenton 2mo ago> SWE-2 is post-trained from Kimi K3 On the one hand I would have expected a completely new model, on the other hand it's an RL-ed K3 go Fable 5 capabilities, which demonstrate that this is probably possible, which is nice.
- htrp 2mo agowhy are all American AI models basically Kimi in a trench coat
- ianm218 2mo agoNo one in the US is going to fund pretty good open source with VC money. US has OpenAI/ Anthropic/ Google/ meta/ SpaceX atleast trying to make frontier foundation models 2 are using VC money + cash flow, last 3 are mainly cash flow + equity and debt. China has state banks and similar willing to fund lower margin open source labs.
- Roark66 2mo agoI'd be pretty surprised if someone told me few years ago communist China, state banks would become the main founders of open source compute and our last hope against monopolists like Musk, the whole bunch at OpenAI and so on. To be fair Zuck is releasing fairly capable open source models, but Chinese labs are way ahead.
- nijave 2mo agoIt's (one of) the best freely available?
- mxmilkiib 2mo agoI thought it would be GLM based, as Devin has had free GLM-5.2 for a while now
- nullbio 2mo agoWhere are the model stats? Is this open-weights? If not, why would I use this over DeepSeek Flash 4.1? I think these competing labs need to realize that no one wants another closed-weight model provider... We aren't even happy with the two we have right now, and their days are entirely numbered. If DeepSeek 4.1 flash is really as good as it's benching, we're probably a month away from 1/3rd of users moving off the closed-weight models in favor of something they have more control over (or is cheaper). The big labs love to release their new model and quantize after the first week. You don't have that problem using dirt cheap API rates on OpenRouter. DS 4.1 flash is also faster than fast mode Astra. OAI's subscription rates are good value, but now these new open-weight models are nearly as cheap on API usage rates. I honestly can't wait for the day we're not beholden to the two big labs anymore. No wonder there's so much fear pumping happening at the moment from Anthropic and their funded NGOs.
- eru 2mo ago> If DeepSeek 4.1 flash is really as good as it's benching, we're probably a month away from 1/3rd of users moving off the closed-weight models in favor of something they have more control over (or is cheaper). Only in the world where the incumbents don't react. Eg if they saw lots of users moving away, they'd drop prices or do something else.
- bayesianbot 2mo ago1M cached tokens on deepseek is $0.006, the big labs can't sell anywhere close to this, they have funders expecting returns and huge overhead. btw I've had a ton of fun with the new deepseek today, I was waiting for my OpenAI 5h limit reset and decided to give it some problems for fun, got pretty great results. Tried some harder problems and still got great results. I don't expect it to be Sol class or anything but I really didn't expect it to be anywhere near this good so we'll see where it ends up. And it's really fun throwing crazy amount of tokens at the wall for ~free instead of watching the subscription limits tick closer while your agents churn away.
- cmrdporcupine 2mo ago
- ltsSmitty 2mo agoWell written and good diagrams. No idea the verity of the TMBB (trust me bro benchmarks) but it was pleasing to look at
- llmslave 2mo agoAt work I setup a cloud worker, where i can spin up as many concurrent agents I want, with unlimited fable 5.1 (thanks employer!!). I now just work from my phone, and speak into the agents as they run. I dont write code and I dont write documents. I work on very complicated distributed systems. I dont open my laptop most days. Its a legacy brick I carry around. Some of my coworkers are still doing things by hand, and are working long hours to produce 25% of the output (when considering hours worked). I stay quiet with my setup. We are in the end times for this job for the people that can see clearly how to automate their own job
- hnedeotes 2mo agothat's why after 1 year of product development of these AI 20x maxxed speed, we reached AGI 'wizards', there's really no difference in output, outstanding bugs no longer get solved and sites still suck, even doing things that were just regular development 20 years ago. Are you sure they aren't only producing 2.5% of your output that you manage just by farting into your phone? Are you sure it's 25% really? Seems way to high, days when I have diarrhoea my AI agents move even faster
- llmslave 2mo agoplease keep thinking this so i can relax with my automated job
- hnedeotes 2mo agoIt's not you, it's X... but what would you expect of a nepo-baby economy of little swines. This is like the nepo wet-dream on steroids. Incompetence and delulu
- deleted 2mo ago[deleted]
- mogwire 2mo agoIf you are too dumb or too lazy to figure out what this guy has, then you are the problem and will be looked at as a relic. Watching the AI slop my sales reps put in their emails is disgusting but the reply telling them how great of a job they are doing and how insightful their email was says differently. Many people are laughing to the bank while you are still running `--help` to figure out how to run a complex command.
- eyeris 2mo agoWonder if this was the model that drove factoring the rsa-260 The write-up from yesterday was by somebody from cognition using Devin to translate existing cpu sieving methods to gpu and to optimize the gpu sieve.
- bobtheborg 2mo agoSWE 1.6 was great for small tasks. Very fast and good enough. 1.7 was unusable for me. Took more time thinking than GLM 5.2 and seemed to be generally running in circles. I tried it but abandoned it. Looking forward to 2 -- maybe it'll be usable
- bluelightning2k 2mo agoI like Cognition as a company and hope they succeed. Seemingly excellent engineering org. I used to really like Windsurf. (Now Devin. Kind of? But also now Antigravity.) I still use it as my editor but haven't touched the agent for a while simply due to the rise of Codex.
- pkilgore 2mo agoNot sure it matters when devin is the most consistently shit product I've used. And yes, I tried again, they wasted the money on the billboards.
- AznHisoka 2mo agoI have not met a single person/company that uses Devin… does anyone here actually use it?
- PolCPP 2mo agoI pay for it (mostly because they grandfathered me from the old prices)! I used to use windsurf as my main editor until they changed their pricing model. Now i use it just to burn my weekly tokens on fable/astra if i remember to that on a task and that's it.
- chris_st 2mo agoI use it, and have been happy with it for the most part. Like sibling, I use it for GPT-5.6-Sol and Opus work, and use their free models (GLM 5.2 for the past few months, trying SWE-2 now).
- dominotw 2mo agomy friends at infosys are being trained on it
- 0l 2mo agoDeeply unserious company so not surprising
- fschuett 2mo agoI only used their "DeepWiki" automatic docs, they are pretty decent at getting an overview of a large project and are relatively accurate, with diagrams and anything. Haven't tried out their coding agent stuff.
- hightrix 2mo ago
- gruez 2mo agoCognition, the same company that a few years ago demoed a coding bot purporting to be able to autonomously complete upwork tasks, but upon closer inspection was going off the rails and not even completing what was asked? https://www.youtube.com/watch?v=tNmgmwEtoWE https://www.youtube.com/watch?v=tNmgmwEtoWE As others have mentioned this is post trained from Kimi k3, which is already quite capable, so it can't be that bad, but any claimed improvements in performance should be taken with a grain of salt.
- notfromhere 2mo agoWell the models did get better but yeah their early product was godawful
- fishtoaster 2mo agoA friend recently pushed me to try out their coding platform, Devin, after I decided to move away from Cursor. I had the same reaction: "What, the con artists from like 2024?" But after some cajoling, I gave it a shot and was pleasantly surprised. I guess they learned their lessons, grew up, and are doing good work now, maybe?
- htrp 2mo agoa reminder that money does enable you to make mistakes and buys you the ability to correct from them
- unshavedyak 2mo ago> and buys you the ability to correct from them Or at the very least, make more mistakes.
- darkwizard42 2mo agoI think the evolution of the harness and ability to preserve loop context outside the context window has made running these kinds of agentic experiences easier. sorry so many buzzwords to say, the capabilities to do this kind of work are more accessible and easier to manage, so now it works! Good to see, and agree they were severely overhyping their product back then.
- m3kw9 2mo agoI'm using Codex, Gemini etc, they all have desktop apps and have a plan, how do i use SWE-2? Thats is a problem they have. I'm not about to switch out my workflow and plans with a shiny LLM that looks benchmaxxed and graph maxxed.
- CyLith 2mo agoI think I'm probably in the minority here, but for my line of work, the software engineering and coding is only a small part of the work. I write simulation software, so a deep understanding of physics, math, and how they can be applied to the software is absolutely crucial. I'm assuming this model is tuned to be more focused on SWE topics, and the very reason we seek "multidisciplinary" hires is the also why I actually need a jack-of-all-trades model to back my coding agents.
- skrhee 2mo agoI'm also in simulation software! Wondering which models you are finding helpful, the models I'm using for general SWE skills are horrible at our simulations and even basic physics/engineering calculation and intuition
- CyLith 2mo agoI use Claude Opus 4.8 almost exclusively. I have had fairly good experiences with it. One time it derived an entirely novel simulation method different than anything in literature by combining its knowledge about how problems in other fields with similar underlying mathematical structure are solved. That was a bit of a Jacobian Conjecture moment for me.
- mohamedkoubaa 2mo agoI'm in the same field and I find GPT to be better at understanding physics conceptually but Claude is better at writing numerical code. I use cursor so many of my sessions start in GPT and switch to Claude.
- wqash71 2mo agoThe horrible website is made by Claude or Cognition is distilled. I'm so tired of it all.
- Take8435 2mo agoPost made by account 2 days ago.
- wy35 2mo agoIs there an implication here I'm missing?
- alansaber 2mo agoFair enough that they did a "propoganda and censorship" eval but not sure why i'd care about that in my highly juiced SWE kimi FT.
- sbseitz 2mo agoWhy doesn't clickbait trash like this get moderated ?
- handoflixue 2mo agoBecause it's not clickbait trash / doesn't violate the site rules? Dang is pretty good at enforcing stuff. There's a flag button and you can email reports if you're really bothered.
- pelorat 2mo agoUnless it can do CAD via computer-use how can you say it rivals GPT-Astra?
- andai 2mo agoTheir benchmark used to show other metrics, like output tokens and time, but now only shows cost: https://cognition.com/frontiercode https://cognition.com/frontiercode Which is too bad, since all of the gains here appear to be from massively reduced output tokens? The model SWE-2 is based on, Kimi K3, is cheaper per token than Sol, but costs more per task (ArtificialAnalysis) due to using way more tokens. Whereas, based on the graphs, SWE-2 appears even more token-efficient than Sol! That might have been worth showing off, if true.
- logicallee 2mo agoall this while Cognition doesn't even make the top page of search results, that's some impressive stealth.[1] (just to be clear, I am a far-left activist who spends most of my time working on funding Social Security Trust Funds (OASI & DI Solvency), which could impact my search results - this was while I was logged in.) [1]https://www.google.com/search?q=what%27s+cognition+in+ai https://www.google.com/search?q=what%27s+cognition+in+ai or https://imgur.com/a/UdxtnGg https://imgur.com/a/UdxtnGg
- captainregex 2mo agoI am skeptical. Lived experience is what matters and I don’t have anyone in my life (Devin shop) saying good things about SWE other than it’s free. Hope I’m wrong and it’s not so bad this time
- yipinwong 2mo agoIncest in human biology causes mutations that's bad in the long term. Same for AI models trained on Kimi-3 or other models like Chinese models do. They suffer from the same issue.
- microdrum 2mo agoDo they have anything as good as Amp (which is able to use free models)? Amp has been in the lead for almost a year now and doesn't seem to be relinquishing it.
- gigatexal 2mo agoNot avail on open router?
- deleted 2mo ago[deleted]
- gexla 2mo agoIf everything basically rivals Fable, then why is everything still using it for comparison?
- MaxikCZ 2mo agoHave you spent at least 10 seconds thinking about it or are you asking just out of spite?
- gexla 2mo agoLet me grab my calculator and add up the time I have spent reading about model releases since Fable has been released. It seems they all place themselves relative to Fable. I'm sure that time has added up to far greater than 10 seconds. At some point, it ceased to be a meaningful differentiation. This is especially true when I put the model through real usage.
- thimble_io 2mo ago92.8% on TB2.1 dropping to 27.3% on TB4 is the only number that matters. The rest is marketing.
- palguna26 2mo agoAt this point, i firmly believe all companies are building benchmaxed models, which perform well on older benchmarks but struggle on new one, terminal bench is the best example.