14 ms·
Qwen3-Coder: Agentic coding in the world
- jddj 1y agoOdd to see this languishing at the bottom of /new. Looks very interesting. Open, small, if the benchmarks are to be believed sonnet 4~ish, tool use?
- danielhanchen 1y agoYe the model looks extremely powerful! I think they're also maybe making a small variant as well, but unsure yet!
- sourcecodeplz 1y agoYes they are: "Today, we're announcing Qwen3-Coder, our most agentic code model to date. Qwen3-Coder is available in multiple sizes, but we're excited to introduce its most powerful variant first: Qwen3-Coder-480B-A35B-Instruct." https://huggingface.co/Qwen/Qwen3-Coder-480B-A35B-Instruct https://huggingface.co/Qwen/Qwen3-Coder-480B-A35B-Instruct
- danielhanchen 1y agoOh yes fantastic! Excited for them!
- fotcorn 1y agoIt says that there are multiple sizes in the second sentence of the huggingface page: https://huggingface.co/Qwen/Qwen3-Coder-480B-A35B-Instruct https://huggingface.co/Qwen/Qwen3-Coder-480B-A35B-Instruct You won't be out of work creating ggufs anytime soon :)
- danielhanchen 1y ago:)
- stuartjohnson12 1y agoQwen has previously engaged in deceptive benchmark hacking. They previously claimed SOTA coding performance back in January and there's a good reason that no software engineer you know was writing code with Qwen 2.5. https://winbuzzer.com/2025/01/29/alibabas-new-qwen-2-5-max-model-takes-on-deepseek-in-ai-benchmarks-xcxwbn/ https://winbuzzer.com/2025/01/29/alibabas-new-qwen-2-5-max-m... Alibaba is not a company whose culture is conducive to earnest acknowledgement that they are behind SOTA.
- swyx 1y ago> there's a good reason that no software engineer you know was writing code with Qwen 2.5. this is disingenous. there are a bunch of hurdles to using open models over closed models and you know them as well as the rest of us.
- omneity 1y agoAlso dishonest since the reason Qwen 2.5 got so popular is not so much paper performance.
- stuartjohnson12 1y agoThose hurdles exist because they're worse for most people. You think Cursor wouldn't spin up their own Qwen inference cluster or contract with someone who can if doing so would give them SOTA code editing performance against Claude?
- stocksinsmocks 1y agoThere is also paranoia that the Chinese government may compel their tech companies to play dirty tricks on their users. Yet without a trace of irony the critics have nothing to say about this not-so-secret practice for US based technology companies.
- pxc 1y agoClearly the thing we should want is a healthy, international AI ecosystem characterized both by cooperation and by competition, so that we are free to choose between models developed under different conditions, for compliance with different laws, subject to different cultures and biases, pressured by different interests, etc. To the extent that there's a solution, the solution is choice!
- mohsen1 1y agoOpen weight models matching Cloud 4 is exciting! It's really possible to run this locally since it's MoE
- danielhanchen 1y agoYe! Super excited for Coder!!
- ilaksh 1y agoWhere do you put the 480 GB to run it at any kind of speed? You have that much RAM?
- Cheer2171 1y agoYou can get a used 5 year old Xeon Dell or Lenovo Workstation and 8x64GB of ECC DDR4 RAM for about $1500-$2000. Or you can rent a newer one for $300/mo on the cloud
- sourcecodeplz 1y agoEveryone keeps saying this but it is not really useful. Without a dedicated GPU & VRAM, you are waiting overnight for a response... The MoE models are great but they need dedicated GPU & VRAM to work fast.
- jychang 1y agoWell, yeah, you're supposed to put in a GPU. It's a MoE model, the common tensors should be on the GPU, which also does prompt processing. The RAM is for the 400gb of experts.
- danielhanchen 1y agoYou don't actually need 480GB of RAM, but if you want at least 3 tokens / s, it's a must. If you have 500GB of SSD, llama.cpp does disk offloading -> it'll be slow though less than 1 token / s
- generalizations 1y ago> Additionally, we are actively exploring whether the Coding Agent can achieve self-improvement How casually we enter the sci-fi era.
- deleted 1y ago[deleted]
- yakz 1y agoI don’t get the feeling that the amount of money being spent is at all casual.
- jasonjmcghee 1y agoWe have self driving cars, humanoid robots, and thinking machines. I think we're there.
- jasonthorsness 1y agoWhat sort of hardware will run Qwen3-Coder-480B-A35B-Instruct? With the performance apparently comparable to Sonnet some of the heavy Claude Code users could be interested in running it locally. They have instructions for configuring it for use by Claude Code. Huge bills for usage are regularly shared on X, so maybe it could even be economical (like for a team of 6 or something sharing a local instance).
- sourcecodeplz 1y agoWith RAM you would need at least 500gb to load it but some 100-200gb more for context too. Pair it with a 24gb GPU and the speed will be 10t/s, at least, I estimate.
- danielhanchen 1y agoOh yes for the FP8, you will need 500GB ish. 4bit around 250GB - offloading MoE experts / layers to RAM will definitely help - as you mentioned a 24GB card should be enough!
- vFunct 1y agoDo we know if the full model is FP8 or FP16/BF16? The hugging face page says BF16: https://huggingface.co/Qwen/Qwen3-Coder-480B-A35B-Instruct https://huggingface.co/Qwen/Qwen3-Coder-480B-A35B-Instruct So likely it needs 2x the memory.
- danielhanchen 1y agoI think it's BF16 trained then quantized to FP8, but unsure fully - I was also trying to find out if they used FP8 for training natively!
- jychang 1y agoQwen uses 16bit, Kimi and Deepseek uses FP8.
- rbren 1y agoGlad to see everyone centering on using OpenHands [1] as the scaffold! Nothing more frustrating than seeing "private scaffold" on a public benchmark report. [1] https://github.com/All-Hands-AI/OpenHands https://github.com/All-Hands-AI/OpenHands
- swyx 1y agomore info on AllHands from robert (above) https://youtu.be/o_hhkJtlbSs https://youtu.be/o_hhkJtlbSs
- KaoruAoiShiho 1y agoHow is cognition so incompetent? They got hundreds of millions of dollars but now they're not just supplanted by Cursor and Claude Code but also by their literal clone, an outfit that was originally called "OpenDevin".
- samrus 1y agoThe AI space is attracting alot of grifters. Even the initial announcement for devin was reaking of elon musk style overpromising. Im sure the engineers are doing the best work they can. I just dont think leadership is as interested in making a good product as they are in creating a nice exit down the line
- ramon156 1y agoAre you purposefully ignoring Zed?
- incomingpain 1y agoI just finally got devstral working well. Openhands is clearly the best ive used so far. Even gemini cli is lesser.
- flakiness 1y agoThe "qwen-code" app seems to be a gemini-cli fork. https://github.com/QwenLM/qwen-code https://github.com/QwenLM/qwen-code https://github.com/QwenLM/qwen-code/blob/main/LICENSE https://github.com/QwenLM/qwen-code/blob/main/LICENSE I hope these OSS CC clones converge at some point. Actually it is mentioned in the page: we’re also open-sourcing a command-line tool for agentic coding: Qwen Code. Forked from Gemini Code
- rapind 1y agoI currently use claude-code as the director basically, but outsource heavy thinking to openai and gemini pro via zen mcp. I could instead use gemini-cli as it's also supported by zen. I would imagine it's trivial to add qwen-coder support if it's based on gemini-cli.
- bredren 1y agoHow was your experience using Gemini via Zen? I’ve instead used a Gemini via plain ol’ chat, first building a competitive, larger context than Claude can hold then manually bringing detailed plans and patches to Gemini for feedback with excellent results. I presumed mcp wouldn’t give me the focused results I get from completely controlling Gemini. And that making CC interface via the MCP would also use up context on that side.
- rapind 1y agoI just use it for architecture planning mostly when I want more info and to feed more info to claude. Tougher problems where 3 brains are better.
- apwell23 1y agowhat is the benefit of outsourcing to other models. do you see any noticable differences?
- bredren 1y agoThere are big gains to be had by having one top tier model review the work of another. For example, you can drive one model to a very good point through several turns, and then have the second “red team” the result of the first. Then return that to the first model with all of its built up context. This is particularly useful in big plans doing work on complex systems. Even with a detailed plan, it is not unusual for Claude code to get “stuck” which can look like trying the same thing repeatedly. You can just stop that, ask CC to summarize the current problem and attempted solutions into a “detailed technical briefing.” Have CC then list all related files to the problem including tests, then provide the briefing and all of the files to the second LLM. This is particularly good for large contexts that might take multiple turns to get into Gemini. You can have the consulted model wait to provide any feedback until you’ve said your done adding context. And then boom, you get a detailed solution without even having to directly focus on whatever minor step CC is stuck on. You stay high level. In general, CC is immediately cured and will finish its task. This is a great time to flip it into planning mode and get plan alignment. Get Claude to output an update on its detailed plan including what has already been accomplished then again—-ship it to the consulting model. If you did a detailed system specification in advance, (which CC hopefully was originally also working from) You can then ask the consulting model to review the work done and planned next steps. Inevitably the consulting model will have suggestions to improve CC’s work so far and plans. Send it on back and you’re getting outstanding results.
- rapind 1y agoI just checked and it's up on OpenRouter. (not affiliated) https://openrouter.ai/qwen/qwen3-coder https://openrouter.ai/qwen/qwen3-coder
- danielhanchen 1y agoI'm currently making 2bit to 8bit GGUFs for local deployment! Will be up in an hour or so at https://huggingface.co/unsloth/Qwen3-Coder-480B-A35B-Instruct-GGUF https://huggingface.co/unsloth/Qwen3-Coder-480B-A35B-Instruc... Also docs on running it in a 24GB GPU + 128 to 256GB of RAM here: https://docs.unsloth.ai/basics/qwen3-coder https://docs.unsloth.ai/basics/qwen3-coder
- mathrawka 1y agoLooks like the docs have a typo: Recommended context: 65,536 tokens (can be increased) That should be recommended token output, as shown in the official docs as: Adequate Output Length: We recommend using an output length of 65,536 tokens for most queries, which is adequate for instruct models.
- danielhanchen 1y agoOh thanks - so the output can be any length you like - I'm actually also making 1 million context length GGUFs as well! https://huggingface.co/unsloth/Qwen3-Coder-480B-A35B-Instruct-1M-GGUF https://huggingface.co/unsloth/Qwen3-Coder-480B-A35B-Instruc...
- sgammon 1y agoCool, thanks! I'd like to try it
- danielhanchen 1y agoIt just got uploaded! I made some docs as well on how to run it at https://docs.unsloth.ai/basics/qwen3-coder https://docs.unsloth.ai/basics/qwen3-coder
- Jayakumark 1y agoWhat will be the approx token/s prompt processing and generation speed with this setup on RTX 4090?
- 1y ago
- pxc 1y ago> Qwen3-Coder is available in multiple sizes, but we’re excited to introduce its most powerful variant first I'm most excited for the smaller sizes because I'm interested in locally-runnable models that can sometimes write passable code, and I think we're getting close. But since for the foreseeable future, I'll probably sometimes want to "call in" a bigger model that I can't realistically or affordably host on my own computer, I love having the option of high-quality open-weight models for this, and I also like the idea of "paying in" for the smaller open-weight models I play around with by renting access to their larger counterparts. Congrats to the Qwen team on this release! I'm excited to try it out.
- segmondy 1y agosmall models can never match bigger models, the bigger models just know more and are smarter. the smaller models can get smarter, but as they do, the bigger models get smart too. HN is weird because at one point this was the location where I found the most technically folks, and now for LLM I find them at reddit. tons of folks are running huge models, get to researching and you will find out you can realistically host your own.
- nico 1y ago> and now for LLM I find them at reddit. tons of folks are running huge models Very interesting. Any subs or threads you could recommend/link to? Thanks
- ActorNightly 1y agoBasically just run ollama and run the quantized models. Don't expect high generation speeds though.
- segmondy 1y agojoin us at r/LocalLlama
- BriggyDwiggs42 1y agoThe small model only needs to get as good as the big model is today, not as the big model is in the future.
- vFunct 1y agoMuch faster than Claude Sonnet 4 with similar results.
- thenaturalist 1y agoCare to share more specifics/ your comparison case?
- campers 1y agoLooking forward to using this on Cerebras!
- deleted 1y ago[deleted]
- indigodaddy 1y agoHow does one keep up with all this change? I wish we could fast-forward like 2-3 years to see if an actual winner has landed by then. I feel like at that point there will be THE tool, with no one thinking twice about using anything else.
- int_19h 1y agoWhy do you believe so? The leaderboard is highly unstable right now and there are no signs of that subsiding. I would expect the same situation 2-3 years forward, just possibly with somewhat different players.
- blibble 1y agodon't bother at all assuming it doesn't all implode due to a lack of profitability, it should be obvious
- aitchnyu 1y agoThe underlying models are apparently profitable. Inference costs are in a exponential fall that makes Gordon Moore faint. OpenRouter shows Anthropic, AWS, Google host Claude at same rates, apparently nobody is price dumping. That said, code+git+agent is only acceptable way for technical staff to interact with AI. Tools with sparkles button can go to hell. https://a16z.com/llmflation-llm-inference-cost/ https://a16z.com/llmflation-llm-inference-cost/ https://openrouter.ai/anthropic/claude-sonnet-4 https://openrouter.ai/anthropic/claude-sonnet-4
- blibble 1y agoif I dropped 99.999999% of my costs I'd be Google level profitable too
- segmondy 1y agoOne keeps up with it, by keeping up with it. Folks keep up with latest social media gossip, the news, TV shows, or whatever interests them. You just stay on it. Weekend I got to running Kimi K2, last 2 days I have been driving Ernie4.5-300B, Just finished downloading the latest Qwen3-235b this morning and started using it this evening. Tonight I'll start downloading this 480B, might take 2-3 days with my crappy internet and then I'll get to it. Obsession?
- mogili 1y agoI'm waiting on this to be released on Groq or Cerebras for high speed vibe coding.
- adenta 1y agoI was only getting like 200 tk/s with groq on K2, was expecting it to be faster tbh. I think the bottleneck is file read/write tooling right now
- nnx 1y agoThis suggests adding a `QWEN.md` in the repo for agents instructions. Where are we with `AGENTS.md`? In a team repo it's getting ridiculous to have a duplicate markdown file for every agent out there.
- singhrac 1y agoI just symlink to AGENTS.md, the instructions are all the same (and gitignore the model-specific version).
- drewbitt 1y agoCLAUDE.md MISTRAL.md GEMINI.md QWEN.md GROK.md .cursorrules .windsurfrules .copilot-instructions Saw a repo recently with probably 80% of those
- falcor84 1y agoIt would be funny to write conflicting instructions on these, and then unleash different coding agents on the same repo in parallel, and see which one of them first identifies the interference from the others and rewrites their instructions to align with its own.
- yard2010 1y agoLol you can even tell each model to maliciously and secretly sabotage other agents and see which one wins.
- yencabulator 1y agoCore War NG! https://en.wikipedia.org/wiki/Core_War https://en.wikipedia.org/wiki/Core_War
- sunaookami 1y agoI just make a file ".llmrules" and symlink these files to it. It clutters the repo root, yes...
- mattigames 1y ago
- lvl155 1y agoCan someone please make these CLI from Rust/Ratatui.
- mostlyk 1y agoI made one using Rust predating Gemini-CLI https://github.com/MostlyKIGuess/Yappus-Term https://github.com/MostlyKIGuess/Yappus-Term , but it's more of a search tool than coding. Closest you get is https://github.com/opencode-ai/opencode https://github.com/opencode-ai/opencode in GO.
- crvdgc 1y agoOpenAI's Codex has a Rust and Ratatui implementation. I believe it's now the default verison. (Previously the TypeScript implementation was the default.) https://github.com/openai/codex/tree/main/codex-rs https://github.com/openai/codex/tree/main/codex-rs
- yencabulator 1y agoAmazon's Claude Code clone is in Rust and Apache-2/MIT: https://github.com/aws/amazon-q-developer-cli https://github.com/aws/amazon-q-developer-cli
- sunaookami 1y agoThank god I already made an Alibaba Cloud account last year because this interface sucks big time. At least you get 1 mio. tokens free (once?). Bit confusing that they forked the Gemini CLI but you still have to set environment variables for OpenAI?
- NitpickLawyer 1y agoAt this point the openai compatible API is the de facto standard. You probably want to set both the base_url and api key, so you can test with 3rd party providers.
- nisten 1y agoI've been using it all day, it rips. Had to bump up toolcalling limit in cline to 100 and it just went through the app no issues, got the mobile app built, fixed throug hthe linter errors... wasn't even hosting it with the toolcall template on with the vllm nightly, just stock vllm it understood the toolcall instructions just fine
- ramoz 1y agoNice, what model & on what hardware?
- nxobject 1y agoWelp, time to switch aider models for the _second_ time in a week...
- manmal 1y agoHow good is it at editing files? Many write/replace errors?
- MaxikCZ 1y agoIm interested in more info? Where do you host it? Whats the hardware, and exact model? What t/s do you get? What is the codebase size? Etc pls, thank you
- apwell23 1y agoso are you tell where you hosted it?
- jijji 1y agoI'm confused why would this LLM require API keys to openAI?
- niea_11 1y agoThe env variables names are misleading. They don't require api keys to OpenAI. Apparently, their tool can connect to any open ai compatible api and that's how you configure your crendentials. You can point it to openrouter or nebius.com to use other models.
- pzo 1y agodoes anyone understand pricing ? On OpenRouter (https://openrouter.ai/qwen/qwen3-coder https://openrouter.ai/qwen/qwen3-coder) you have: Alibaba Plus: input: $1 to $6 output: $5 to $60 Alibaba OpenSource: input: $1.50 to $4.50 output: $7.50 to $22.50 So it doesn't look that cheap comparing to Kimi k2 or their non coder version (Qwen3 235B A22B 2507). What's more confusing this "up to" pricing that supposed to can reach $60 for output - with agents it's not that easy to control context.
- barrenko 1y agoI wanted to start using the Alibaba cloud for a personal project six months ago, couldn't make sense of the pricing and just gave up, so it's not new in my humble experience...
- pzo 1y agoFound out this on simonw blog [0]: "his is the first model I've seen that sets different prices for four different sizes of input" [1] It seems now a every expensive model to run with alibaba as provider. You only get this low price for input <32k. For input <256k both gemini 2.5 pro and o3 is cheaper. [0] https://simonwillison.net/ https://simonwillison.net/ [1] https://static.simonwillison.net/static/2025/qwen3-coder-plus-prices.jpg https://static.simonwillison.net/static/2025/qwen3-coder-plu...
- zkmon 1y agoAt my work, here is a typical breakdown of time spent by work areas for a software engineer. Which of these areas can be sped up by using agentic coding? 05%: Making code changes 10%: Running build pipelines 20%: Learning about changed process and people via zoom calls, teams chat and emails 15%: Raising incident tickets for issues outside of my control 20%: Submitting forms, attending reviews and chasing approvals 20%: Reaching out to people for dependencies, following up 10%: Finding and reading up some obscure and conflicting internal wiki page, which is likely to be outdated
- sandos 1y agoVery similar to my job, although its very variable: some weeks I do write / debug code 80-90% of the time.
- mhl47 1y agoIn theory, nearly all of them? Many of those things could be improved today without AI but e.g. raising Incidents for issues outside of your control could also give you a suggestion already that you just have to tick off. Not saying we are there yet but hard to imagine it's not possible.
- zkmon 1y agoRaising incidents is not about suggestions. Things like build pipelines run into issues, someone from Ops need to investigate, and maybe bump up some pods or apply some config changes on their end. Or some wiki page has conflicting information, someone need to update it with correct information after checking with the relevant other people, policies and standards. The other people might be on vacation and their delegate misguides as they are not aware of the recently changed process. It's probably messier than you think.
- theptip 1y agoIn the short term, I think humans will be doing more of technical / product alignment, f2f calls (especially with non-technical folks), digesting illegible requirements, etc. Coding, debugging builds, paperwork, doc chasing are all tasks that AI is improving on rapidly.
- Imanari 1y agoI've been using it within Claude Code via ccr[0] and it feels very similar to Claude 4. [0] https://github.com/musistudio/claude-code-router https://github.com/musistudio/claude-code-router
- deleted 1y ago[deleted]
- kindkang2024 1y ago[dead]
- Alifatisk 1y agoWow, these companies in the llm field is so quick to catch up. From everyone offering their own chat model to openai-compitable schema to allowing extensions and IDEs do the work to agentic tasks and now most of them offering their own cli
- zelphirkalt 1y agoSo far none of these models can write even a slightly complicated function well for me. I tried Mistral, ChatGPT, Qwen Coder 2, Claude, ... they apparently all fail when the solution requires to make use of continuations and such. Probably, because they don't have enough examples in their training data or something. Example: Partition a linked list in linear time. None of these models seems to be able to get, that `reverse` or converting the whole list to a vector are in themselves linear operations and therefore forbid themselves. When you tell them to not use those, they still continue to do so and blatantly claim, that they are not using them. Á la: "You are right, ... . The following code avoids using `reverse`, ... : [code that still uses reverse]" And in languages like Python they will cheat, because Python's list is more like an array, where random access is O(1). This means they only work well, when you are doing something quite mainstream, where the amount of training data is a significantly strong signal in the noise. But even there they often struggle. For example I found them somewhat useful for doing Django things, but just as often they gave bullshit code, or it took a lot of back and forth to get something useful out of them. I think it is embarrassing, that with sooo much training data, they are still unable to do much more than going by frequency in training data when suggesting "solutions". They are "learning" differently than a human being. When a human being sees a new concept, they can often apply that new concept, even if that concept does not happen to be needed that often, as long as they remember the concept. But in these LLMs it seems they deem everything that isn't mainstream irrelevant.
- frumiousirc 1y ago> [code that still uses reverse] I get this kind of lying from Gemini 2.5 Flash sometimes. It's super frustrating and dissolves all the wonder that accumulated when the LLM was giving useful responses. When it happens, I abandon the session and either figure out the problem myself or try a fresh session with more detailed prompting.
- sourcecodeplz 1y agoI use it more like documentation, I know it can't really invent things for me.
- logicchains 1y ago
- oliver_zenn 1y ago[dead]
- veselin 1y agoAnybody knows if one can find an inference provider that offers input token caching? It should be almost required for agentic use - first speed, but also almost all conversations start where the previous ended, so cost may end up quite higher with no caching. I would have expected good providers like Together, Fireworks, etc support it, but I can't find it, except if I run vllm myself on self-hosted instances.
- karolist 1y agoI have 4x3090 (96GB) and 128GB DDR4 RAM, can I run unsloth on this machine and utilize all 4 GPUs?
- incomingpain 1y agoNow I await the distilled options. I wonder if there's a python expert that can be isolated.
- chisleu 1y agoI tried using the "fp8" model through hyperbolic but I question if it was even that model. It was basically useless through hyperbolic. I downloaded the 4bit quant to my mac studio 512GB. 7-8 minutes until first tokens with a big Cline prompt for it to chew on. Performance is exceptional. It nailed all the tool calls, loaded my memory bank, and reasoned about a golang code base well enough to write a blog post on the topic: https://convergence.ninja/post/blogs/000016-ForeverFantasyFreshFoundation.md https://convergence.ninja/post/blogs/000016-ForeverFantasyFr... Writing blog posts is one of the tests I use for these models. It is a very involved process including a Q&A phase, drafting phase, approval, and deployment. The filenames follow a certain pattern. The file has to be uploaded to s3 in a certain location to trigger the deployment. It's a complex custom task that I automated. Even the 4bit model was capable of this, but was incapable of actually working on my code, prefering to halucinate methods that would be convenient rather than admitting it didn't know what it was doing. This is the 4 bit "lobotomized" model though. I'm excited to see how it performs at full power.
- jug 1y agoI checked this website along with API pricing on OpenRouter, and this one beats Gemini 2.5 Pro (…Preview-0506 in their chart, but with a good margin so probably the non-preview too) at half Google’s API price. Nice. Admittedly their own posted benchmark, but still. If it even just competes with it, it’s a win. Edit: I ran my fun test on it and it unfortunately failed. > ”How can I detect whether a user is running in a RemoteApp context using C# and .NET? That is, not a full RDP desktop session, but a published RemoteApp as if the app is running locally. The reason I’m asking is that we have an unfortunate bug in a third party library that only shows up in this scenario, and needs a specific workaround when it happens.” It started by trying to read hallucinated environment variables that just aren’t there. Gemini 2.5 Pro had the same issue and IIRC also Claude. The only one I have seen give the correct answer that is basically ”You can’t. There’s no official method to do this and this is intentional by Microsoft.” along with a heuristic to instead determine the root launching process which is thus far (but not guaranteed to be) RDPINIT.EXE rather than EXPLORER.EXE as in typical desktop or RDP scenarios… has been OpenAI o3. o3 also provided additional details about the underlying protocol at play here which I could confirm with external sources to be correct. I like my query because it forces the LLM to actually reply with that you just can’t do this, there’s no ”sign” of it other than going by a completely different side-effect. They are usually too eager to try to figure out a positive reply and hallucinate in the process. Often, there _are_ these env vars to read in cases like these, but not here.
- _peregrine_ 1y agoPretty solid at SQL generation, too. Just tested in our generation benchmark: https://llm-benchmark.tinybird.live/ https://llm-benchmark.tinybird.live/ Not quite as good as Claude but by the best Qwen model so far and 2x as fast as qwen3-235b-a22b-07-25 Specific results for qwen3-coder here: https://llm-benchmark.tinybird.live/models/qwen3-coder https://llm-benchmark.tinybird.live/models/qwen3-coder