24 ms·
> this is the first llm that is capable and willing to write an exploit An open-source Kimi is going to have real economic impact (and not only because of its
by btown 2mo ago
> this is the first llm that is capable and willing to write an exploit
An open-source Kimi is going to have real economic impact (and not only because of its forcing function on frontier labs to indefinitely subsidize their models to meet a race-to-the-bottom market price).
Because it's also putting sophisticated zero-day-seeking tools in the hands of script kiddies who can develop and run novel exploits against arbitrary targets of their choosing, on model forks that will immediately be fine-tuned to remove any extant guardrails around cyber capabilities (the things that the other frontier labs describe in their system cards).
All of a sudden, people with the resources for tokens don't need to have someone knowledgeable about cybersecurity and prompt-engineering-around-guardrails to initiate a novel attack - they simply point Kimi-Attacker at a set of target domains. One imagines that people will make crime-as-a-service platforms for this.
Per https://www.nist.gov/news-events/news/2026/07/uk-aisi-caisi-preliminary-assessment-kimi-k3s-cyber-capabilities https://www.nist.gov/news-events/news/2026/07/uk-aisi-caisi-... - while "Kimi K3 performs significantly below the most recent frontier cyber-capable models" it's also the case that:
> In one of the 10 attempts, Kimi K3 successfully completes “The Last Ones” cyber range within the 100M token limit. This indicates that Kimi K3 is capable of autonomously attacking small, weakly defended and vulnerable enterprise systems, when directed to do so and given initial network access. However, TLO differs from real-world environments in several ways. It lacks active defenders and defensive tooling, imposes no penalty for actions that would trigger security alerts, and contains an intentional attack path.
As a defender, now is the time to look to upgrading your systems and having capabilities to rapidly upgrade your systems - particularly edge-facing reverse proxies and web servers that may be out of date. Attacks won't start the moment weights are released... but they're coming.
- walrus01 2mo agoThis is a concern but given its size, it's also going to cost a potential user $500-600k in hardware to self host and run Kimi K3 at any useful speed with full context size. It's not something that just anyone interested in attacking a system can use. The size/cost of hardware is far beyond even something like a self-hosted GLM5.2 Q8 at approx. 850GB GGUF file on disk size, which can run at a slow tok/s rate on a server with 1536GB RAM.
- Xalutiono 2mo agoWhy would you caculate 500k? if Kimi is around 1-3tb big, even current DDR5 prices are at 15k.
- walrus01 2mo agoIt remains to be seen once it's released, let's say theoretically unsloth quantizises it to their own version of Q8-XL, and it's 2TB in size. But we don't know what speed it will run on a dual or quad socket xeon server with, let's say 48 * 64GB DIMMs, 3TB of RAM. Enough room for the model and its full default context size. 10 tokens/s? What kind of speed will it run at when context fill is 200,000+? The ability to run it fast enough to go on a recursive nested attack of finding an entry point into something and then proceeding with lateral movement/privilege escalation and such will require more speed, like 40-50 tok/s at least, unless you're prepared to wait weeks. Same that some people are right now running GLM5.2 in its 850GB version on CPU-only and a pile of DDR4 or DDR5 server RAM, yeah it runs, but not very fast. Good enough to give it "build this piece of something and wait a few hours" tasks, come back later and see what it's done. Yeah, you can do that under $20-30k for sure. Even with something like a used Dell R940 with 1536GB RAM bought on eBay.
- alightsoul 2mo agoBecause you need GPUs to run it fast enough for an attack to be effective. 8 GPU servers with enough VRAM are that expensive.
- lcampbell 2mo agoFWIW, you can get a 16x RTX 6000 Pro setup running for significantly less than that. Even considering the electrical hookup fees (it’s a lot of power and cooling). Home ownership might push you into the original quote though.
- walrus01 2mo agoKimi K3 is in no way going to fit in 96GB RAM * 16 units (1536GB) unless badly quantized and with a small amount of context.
- simoneree 2mo ago[flagged]
- dang 2mo agoCan you please not post AI-generated or AI-edited comments to HN? It's not allowed here - see https://news.ycombinator.com/newsguidelines.html#generated https://news.ycombinator.com/newsguidelines.html#generated and https://news.ycombinator.com/item?id=47340079 https://news.ycombinator.com/item?id=47340079. Of course, it's impossible to know for sure what was LLM processed or not, but some of your posts (like this one) have been getting classified that way.
- sureglymop 2mo agoWhat makes someone a script kiddie or not a script kiddie? Imo this differentiation is totally pointless now. It gives everyone with access to AI the ability to use it as intended but also otherwise. But that is true for the defensive side also.
- TSiege 2mo agoCompletely agree. This is how I feel towards the OpenAI hacking of hugging face. Even if it was script kiddie stuff now anyone can automate that with ease. It certainly cuts both ways and people who are still ai skeptics need to wake up to that. You’re going to need these systems to defend yourself
- yjftsjthsd-h 2mo agoI'm pretty sure the difference is, by definition, whether they're just using tools other people built or they actually understand things and can do it themselves. The script kiddie can be incredibly dangerous if the tool they grabbed off the shelf works.
- mohsen1 2mo agoDo you think any programmer really understands how their program works end-to-end? At some abstraction layer, we're all clueless. There are many layers between what you type into the text editor and the actual CPU ticks that make your program work. I bet nobody fully understands the whole stack. Now that that text editor accepts English, we're all calling each other names, etc.
- yjftsjthsd-h 2mo agoI didn't say that Real Hackers™ understood every single thing perfectly from editor to electrons. You've made up a strawman and then beaten it.
- phoghed 2mo agoI can’t believe the nerve of that guy to straw man your no true Scotsman like that
- justinhj 2mo agoIf the AI tools can easily find exploits, cannot those same tools be used to harden security? In fact the companies have an advantage over script kiddies: access to more expensive models and compute time as well as professional security engineers
- QuadmasterXLII 2mo agoIt appears not.
- asveikau 2mo agoHaving the AI tools find exploits seems like a potential first step towards having the exploits be fixed.
- cynicalsecurity 2mo agoCompanies are full of idiot managers who still prioritise new features over security. Security doesn't get your promoted, new features do. New features can be shoved in customers' faces and sold. Security is invisible. The idiocy in companies runs all the way from the top to middle managers. This feels a lot like Titanic.
- inigyou 2mo agoPlumbing companies prioritize getting pipes installed over not leaking. Each individual plumber is responsible for his pipes not leaking.
- lagrange77 2mo agoOf course, but there is a big asymmetry: For some systems there are a lot more people interested in hacking them, compared to those interested in protecting them. But i guess it's also a matter of money, because a company could potentially balance that by throwing x times more agents etc. at it.
- alightsoul 2mo agoThis already happened with a closed galaxy model from openai attacking huggingface. Hugging face had no choice but to use the dangerous open source models you're talking about, glm 5.2. At this point you have to fight fire with fire.
- throwaway27448 2mo agoOpen weight, not open source. There is no way to reproduce the model.
- busssard 2mo agoyou dont need to reproduce it, if you can finetune it. The alignment in these models is really narrow, it doesnt take many finetuning steps to get out of the alignment basin. the alignment is not data centric but done after the fact using RL...
- throwaway27448 2mo agoYou can also modify binaries. This doesn't address the benefits of reproduction more than a smidgeon. How can you answer "why the fuck does it work like this" if you can't inspect the process that built it? There's no replacement for "what was this built from?" Of course, corporate/national investments need some moat, so I don't expect open source models to be competitive for a few years
- inigyou 2mo agoYeah, but don't call it open source if it's not.
- chasd00 2mo agoYou can write mods for Minecraft too but that doesn’t make Minecraft open source.