6 ms·
Step 3.5 Flash – Open-source foundation model, supports deep reasoning at speed
- kristianp 8mo agoRecent model released a couple of weeks ago. "Mixture of Experts (MoE) architecture, it selectively activates only 11B of its 196B parameters per token". Beats Kimi K2.5 and GLM 4.7 on more benchmarks than it loses to them. Edit: there are 4 bit quants that can be run on an 128GB machine like a GB10 [1], AI Max+ 395, or mac studio. [1] https://forums.developer.nvidia.com/t/running-step-3-5-flash-on-single-spark/359457/12 https://forums.developer.nvidia.com/t/running-step-3-5-flash...
- Alifatisk 8mo ago> Beats Kimi K2.5 and GLM 4.7 on more benchmarks than it loses to them. Does this really mean anything? I for example, tend to ignore certain benchmarks that are focused towards agentic tasks because that is not my use case. Instruction following, long context reasoning and non-hallucinations has more weight to me.
- mycall 8mo agoQ4_K_S @ 116 GB IQ4_NL @112 GB Q4_0 @ 113 GB Which of these would be technically better? [1] https://huggingface.co/bartowski/stepfun-ai_Step-3.5-Flash-GGUF https://huggingface.co/bartowski/stepfun-ai_Step-3.5-Flash-G...
- KerrAvon 8mo agoof those, Q4_K_S is better
- wmf 8mo agoThat reverse x axis sure is confusing.
- esafak 8mo agoI imagine they thought they'd look better this way. I don't think they do.
- __mharrison__ 8mo agoWas going to comment the same thing... Not sure what they are thinking there.
- SilverElfin 8mo agoSo who exactly is StepFun? What is their business (how do they make money)? Each time I click “About Stepfun” somewhere on their website, it sends me to a generic landing page in a loop.
- deaux 8mo agoMight want to give it a search.
- 0x1997 8mo agohttps://en.wikipedia.org/wiki/StepFun https://en.wikipedia.org/wiki/StepFun
- SilverElfin 8mo agoThanks. Do they sell any of these products today or is it more like research? I am not able to find anything relating to pricing on their website. Just a chatbot.
- 0x1997 8mo agoPrincing can be found on their docs website https://platform.stepfun.ai/docs/en/pricing/details https://platform.stepfun.ai/docs/en/pricing/details
- kristopolous 8mo agoThey've been around a couple years. This is the first model that has really broken into the anglosphere. Keep a tab on aihubmix, the Chinese openrouter, if you want to stay on top of the latest models. They keep track of things like the Baichuan, Doubao, baai (beijing academy), Meituan, 01.AI (yi), xiaomi, etc... Much larger chinese coverage than openrouter
- Havoc 8mo ago>first model that has really broken into the anglosphere. Do you know of a couple of interesting ones that haven't yet?
- danieltanfh95 8mo agoHallucinates like crazy. use with caution. Tested it with a simple "Find me championship decks for X pokemon", "How does Y deck work". Opus 4.6, Deepseek and Kimi all performed well as expected.
- mickeyp 8mo agoI mean, is it possible the latter models used Search? Not saying Stepfun's perfect (it is not.) Gemini especially and unsurprisingly uses search a lot and it is ridiculously fast, too.
- esafak 8mo agoI would use a medium-sized model for execution not its knowledge.
- danieltanfh95 8mo agoIt is just search. If it can't even execute search then what else would it be good at?
- agentifysh 8mo agowhat country is behind this one ?
- personalcompute 8mo agoStep 3.5 Flash was made by Chinese company StepFun - https://en.wikipedia.org/wiki/StepFun https://en.wikipedia.org/wiki/StepFun
- culi 8mo agoIt's nice to see more focus on efficiency. All the recent new model releases have come along with massive jumps in certain benchmarks but when you dig into it it's almost always paired with a massive increase in token usage to achieve those results (ahem Google Deep Think ahem). For AI to truly be transformational it needs to solve the electricity problem
- tankenmate 8mo agoAnd not just token usage, expensive token usage; when it comes to tokens/joule not all tokens are equal. Efficient use of MoE architectures does have an impact on tokens/joule and tokens/sec.
- mzl 8mo agoI like the intelligence per watt and intelligence per joule framing in https://arxiv.org/abs/2511.07885 https://arxiv.org/abs/2511.07885 It feels like a very useful measure for thinking about long-term sustainable variants of AI build-outs.
- mohsen1 8mo agoSWE-bench Verified is nice but we need better SWE benchmarks. Making a fair benchmark is a lot of work and a lot of money needed to run it continuously. Most of "live" benchmarks are not running enough with recent models to give you a good picture of which models win. The idea of a live benchmark is great! There are thousands of GitHub issues that are resolved with a PR every day.
- cbracketdash 8mo agoHelp us out with Terminal Bench 3.0! https://docs.google.com/document/d/1pe_gEbhVDgORtYsQv4Dyml8uaR7PZBEyVZnBUrs1z0M/edit?pli=1&tab=t.0 https://docs.google.com/document/d/1pe_gEbhVDgORtYsQv4Dyml8u...
- sinenomine 8mo agoWorks impressively well with pi.dev minimal agent.
- janalsncm 8mo agoNumber of params isn’t really the relevant metric imo. Top models don’t support local inference. More relevant is tokens per dollar or per second.
- dakolli 8mo agoIts an open source model, why wouldn't it be relevant for people who want to self host.....
- lm28469 8mo agoIt does since you can run this model locally on a < $3k machine
- qeternity 8mo agoNumber of parameters is at least a proxy for model capability. You can achieve incredible tok/dollar or tok/sec with Qwen3 0.6b. It just won't be very good for most use cases.
- janalsncm 8mo agoModel capability is the other axis on their chart. So they could have put Qwen 0.6b there, it would be in the bottom right corner. I know what they are trying to do. They are attempting show a kind of pareto frontier but it’s a little awkward.
- prmph 8mo agoInteresting. Each time a Chinese model makes the news, I wonder: How come no major models are coming from Japan or Europe?
- jstummbillig 8mo agoHave you heard of Mistral? I would consider Mistral major, albeit not frontier.
- tonis2 8mo agoCause Europe only good at writing fines for other tech companies
- rester324 8mo agoYou would be surprised to see how much the japanese IT industry is behind the times (a decade at least IMO). There is only a very limited startup culture here (both in size and talentpool and business ideas), there is no real risk taking venture capital market here (maybe Masayoshi Son is the exception here, but again he tends to invest in the US mostly) and most software companies use very very very outdated management practices. On top of that most software development had been/has been outsourced to India, Vietnam, China, etc, so management see no value in software talent... SW engineers' social recognition here are mostly on the level of accountants. Under such circumstances japan will never have a chance to contribute to AI meaningfully (other than niche academic research)
- KerrAvon 8mo agoSeems like the Japanese have had this major blind spot in software engineering since the 90's. Even Sony didn't bother to use what they learned from the PlayStations to produce their own TV OS, outsourcing it to Google. It's as if the 5th generation stuff not working out just burned out that circuit in Japan entirely.
- Tepix 8mo agoThe Koreans have released some good models lately. And Mistral is also release open weights models that aren't too shabby.
- anentropic 8mo ago> 51.0% on Terminal-Bench 2.0, proving its ability to handle sophisticated, long-horizon tasks with unwavering stability I don't know anything about TerminalBench, but on the face of it a 51% score on a test metric doesn't sound like it would guarantee 'unwavering stability' on sophisticated long-horizon tasks
- pitched 8mo agoThat score is on par with Gemini 3 Flash but these scores look much more affected by the agent used than the model, from scrolling through the results.
- varispeed 8mo agoGemini 3 Flash is pure rubbish. It can easily get into loop mode and spout information no different than Markov chain and repeat it over and over.
- networked 8mo ago51% doesn't tell you much by itself. Benchmarks like this are usually not graded on a curve and aren't calibrated so that 100% is the performance level of a qualified human. You could design a superhuman benchmark where 10% was the human level of performance. Looking at https://www.tbench.ai/leaderboard/terminal-bench/2.0 https://www.tbench.ai/leaderboard/terminal-bench/2.0, I see that the current best score is 75%, meaning 51% is ⅔ SOTA.
- andai 8mo agoThis is interesting, TFA lists Opus at 59. Which is the same as Claude Code with opus on the page you linked here. But it has Droid agent with Opus scoring 69. Which means the CC harness harness loses Opus 10 points on this benchmark. I'm reminded of https://swe-rebench.com/ https://swe-rebench.com/ where Opus actually does better without CC. (Roughly same score but half the cost!)
- YetAnotherNick 8mo agoTerminalBench is like the worst named benchmark. It has almost nothing to do with terminal, but random tools syntax. Also it's not agentic for most tasks if the model memorized some random tool command line flags.
- Mashimo 8mo agoHoly moly, I made a simple coding promt and the amount of reasoning output could fill a small book. > create a single html file with a voxel car that drives in a circle. Compared to GLM 4.7 / 5 and kimi 2.5 it took a while. The output was fast, but because it wrote so I had to wait longer. Also output was .. more bare bones compared to others.
- Tepix 8mo agoThat's been my experience as well. Huge amounts of reasoning. The model itself is good but even if you get twice as many tokens as with another model, the added amount of reasoning may make it slower in the end.
- esafak 8mo agoThat's how it compensates for its small size. To accomplish a task of certain difficulty either you know more and think less, or vice versa.
- amelius 8mo agoDoes it pass the carwash test?
- oblio 8mo agoWhat's that?
- amelius 8mo agohttps://news.ycombinator.com/item?id=47031580 https://news.ycombinator.com/item?id=47031580
- earth2mars 8mo agoYes it did well. Also some other word problems it did well too. Reasoning seems good. But maybe not a great code model
- tallesborges92 8mo agoI’ve been using this model for a while, and it’s very fast. It spent some time thinking but does fewer calls. For example, yesterday I asked the agent to find the Gemini quota limit for their API, and it took 27 seconds and just 2 calls, Opus 4.6 took 33 seconds, but 5 calls with less thinking
- tarruda 8mo agoThis is probably one of the most underrated LLMs releases in the past few months. In my local testing with a 4-bit quant (https://huggingface.co/ubergarm/Step-3.5-Flash-GGUF/tree/main/IQ4_XS https://huggingface.co/ubergarm/Step-3.5-Flash-GGUF/tree/mai...), it surpasses every other LLM I was able to run locally, including Minimax 2.5 and GLM-4.7, though I was only able to run GLM with a 2-bit quant. Some highlights: - Very context efficient: SWA by default, on a 128G mac I can run the full 256k context or two 128k context streams. - Good speeds on macs. On my M1 Ultra I get 36 t/s tg and 300 t/s pp. Also, these speeds degrade very slowly as context increases: At 100k prefill, it has 20 t/s tg and 129 t/s pp. - Trained for agentic coding. I think it is trained to be compatible with claude code, but it works fine with other CLI harnesses except for Codex (due to the patch edit tool which can confuse it). This is the first local LLM in the 200B parameter range that I find to be usable with a CLI harness. Been using it a lot with pi.dev and it has been the best experience I had with a local LLM doing agentic coding. There are a few drawbacks though: - It can generate some very long reasoning chains. - Current release has a bug where sometimes it goes into an infinite reasoning loop: https://github.com/ggml-org/llama.cpp/pull/19283#issuecomment-3870270263 https://github.com/ggml-org/llama.cpp/pull/19283#issuecommen... Hopefully StepFun will do a new release which addresses these issues. BTW StepFun seems to be the same company that released ACEStep (very good music generation model). At least StepFun is mentioned in ComfyUI docs https://docs.comfy.org/tutorials/audio/ace-step/ace-step-v1 https://docs.comfy.org/tutorials/audio/ace-step/ace-step-v1
- terhechte 8mo agoDid you try an MLX version of this model? In theory it should run a bit faster. I'm hesitant to download multiple versions though.
- tarruda 8mo agoHaven't tried. I'm too used to llama.cpp at this point to switch to something else. I like being able to just run a model and automatically get: - OpenAI completions endpoint - Anthropic messages endpoint - OpenAI responses endpoint - A slick looking web UI Without having to install anything else.
- 8mo ago
- octoclaw 8mo ago[dead]
- lostmsu 8mo agoAny pelicans from non-quantized variants?
- hedgehog 8mo agoIn a quick test using a few of my standard test prompts a few observations: 1) the trace was very verbose and written in an odd style reminiscent of chat or those annoying one-sentence-per-paragraph LinkedIn posts; 2) token output rate is very high on the hosted version; 3) conformance to instructions and output quality was better than most of the leading models I've tested (e.g. Opus 4.5)
- lm2s 8mo agoLoved reading the reasoning[0] for the recent "Walk or drive to the carwash" trick. [0] https://gist.github.com/lm2s/c4e3260c3ca9052ec200b19af9cfd709 https://gist.github.com/lm2s/c4e3260c3ca9052ec200b19af9cfd70... Not sure if it's directly accessible, but here's the link: https://stepfun.ai/chats/213451451786883072 https://stepfun.ai/chats/213451451786883072
- XCSme 8mo agoNot so bad, it's above MiniMax M2.5 with reasoning in my tests (mostly because MiniMax m2.5 seems to reason forever): https://aibenchy.com/compare/?left=stepfun-step-3-5-flash-free-medium&right=minimax-minimax-m2-5-medium https://aibenchy.com/compare/?left=stepfun-step-3-5-flash-fr...