As wild as this sounds, redis should not exposed to the internet and this appears to be an authenticated RCE. This is similar to claiming that a PSQL query grants code execution when its actually a feature.
These systems are typically not tested as aggressively against vulnerabilities of that nature since 10 times out of 10 you have to have access to these systems already.
Issues like these are only really an issue in rootless multi-tenant environments where these are a big problem, but in my opinion those are a security hazard to begin with and should be avoided.
There is a way more interesting telegram RCE that kimi k3 allegedly discovered.
>There is a way more interesting telegram RCE that kimi k3 allegedly discovered.
Telegram server or client? The client is open source, so I expect any reasonably capable model to find things there sooner or later. But the sever is closed source. Finding an RCE there would be wild for such a model, since it would have to solve a whole bunch of adjacent problems instead of just digesting tons of code.
I mean having the client source code can show you a vast number of the API's and their fields which allows you to fuzz the server quite a bit more easily.
> this is the first llm that is capable and willing to write an exploit
An open-source Kimi is going to have real economic impact (and not only because of its forcing function on frontier labs to indefinitely subsidize their models to meet a race-to-the-bottom market price).
Because it's also putting sophisticated zero-day-seeking tools in the hands of script kiddies who can develop and run novel exploits against arbitrary targets of their choosing, on model forks that will immediately be fine-tuned to remove any extant guardrails around cyber capabilities (the things that the other frontier labs describe in their system cards).
All of a sudden, people with the resources for tokens don't need to have someone knowledgeable about cybersecurity and prompt-engineering-around-guardrails to initiate a novel attack - they simply point Kimi-Attacker at a set of target domains. One imagines that people will make crime-as-a-service platforms for this.
> In one of the 10 attempts, Kimi K3 successfully completes “The Last Ones” cyber range within the 100M token limit. This indicates that Kimi K3 is capable of autonomously attacking small, weakly defended and vulnerable enterprise systems, when directed to do so and given initial network access. However, TLO differs from real-world environments in several ways. It lacks active defenders and defensive tooling, imposes no penalty for actions that would trigger security alerts, and contains an intentional attack path.
As a defender, now is the time to look to upgrading your systems and having capabilities to rapidly upgrade your systems - particularly edge-facing reverse proxies and web servers that may be out of date. Attacks won't start the moment weights are released... but they're coming.
This is a concern but given its size, it's also going to cost a potential user $500-600k in hardware to self host and run Kimi K3 at any useful speed with full context size. It's not something that just anyone interested in attacking a system can use.
The size/cost of hardware is far beyond even something like a self-hosted GLM5.2 Q8 at approx. 850GB GGUF file on disk size, which can run at a slow tok/s rate on a server with 1536GB RAM.
It remains to be seen once it's released, let's say theoretically unsloth quantizises it to their own version of Q8-XL, and it's 2TB in size. But we don't know what speed it will run on a dual or quad socket xeon server with, let's say 48 * 64GB DIMMs, 3TB of RAM. Enough room for the model and its full default context size. 10 tokens/s? What kind of speed will it run at when context fill is 200,000+?
The ability to run it fast enough to go on a recursive nested attack of finding an entry point into something and then proceeding with lateral movement/privilege escalation and such will require more speed, like 40-50 tok/s at least, unless you're prepared to wait weeks.
Same that some people are right now running GLM5.2 in its 850GB version on CPU-only and a pile of DDR4 or DDR5 server RAM, yeah it runs, but not very fast. Good enough to give it "build this piece of something and wait a few hours" tasks, come back later and see what it's done. Yeah, you can do that under $20-30k for sure. Even with something like a used Dell R940 with 1536GB RAM bought on eBay.
FWIW, you can get a 16x RTX 6000 Pro setup running for significantly less than that.
Even considering the electrical hookup fees (it’s a lot of power and cooling).
Home ownership might push you into the original quote though.
you dont need to reproduce it, if you can finetune it.
The alignment in these models is really narrow, it doesnt take many finetuning steps to get out of the alignment basin.
the alignment is not data centric but done after the fact using RL...
You can also modify binaries. This doesn't address the benefits of reproduction more than a smidgeon. How can you answer "why the fuck does it work like this" if you can't inspect the process that built it? There's no replacement for "what was this built from?"
Of course, corporate/national investments need some moat, so I don't expect open source models to be competitive for a few years
I'm pretty sure the difference is, by definition, whether they're just using tools other people built or they actually understand things and can do it themselves. The script kiddie can be incredibly dangerous if the tool they grabbed off the shelf works.
Do you think any programmer really understands how their program works end-to-end? At some abstraction layer, we're all clueless. There are many layers between what you type into the text editor and the actual CPU ticks that make your program work. I bet nobody fully understands the whole stack.
Now that that text editor accepts English, we're all calling each other names, etc.
While most don't, there are developers who actually do understand systems from end to end. Most developers are terrible developers, compared to the really good ones.
> While most don't, there are developers who actually do understand systems from end to end
The last time intel made a CPU where the entire thing could fit in one persons head was probably the 8086 or thereabounts.
There is no one developer or engineer at apple that understands everything going on from key press to bits moving around to print the characters I see when I type in this text box.
Used to be, knowing a ret from a nop (90) meant something. If you can just load up a model and say "hack teh NSA.gov" while not actually knowing anything about even DNS, what are we even doing?
> What makes someone a script kiddie or not a script kiddie?
The way I see it, a script kiddie is someone who buy weapons with the intention to cause harm. They don't know or care how the weapon works, as it's just a tool for their malicious plan.
A non-script kiddie would be someone who build weapons.
If you assume weapons have recreational purposes, the builder/non script-kiddie is less likely to be malicious as they love the craft of building weapons. For example recreating historical weapons, or just being nerdy about machinery of a gun works.
A builder with malicious intent as first priority can exist, they are more likely to just be interested in building.
So basically, nerds are perceived to be more ethical than non nerds.
This is exact same difference as between junior and senior engineer.
The premise now is that everyone and your mom can vibe code next unicorn during a lunch break using the latest llms, but the difference between a rookie and a seasoned programmer is that the later can navigate the model in much more efficient way, understand the code that has been written, and notice hallucinations on the spot.
Completely agree. This is how I feel towards the OpenAI hacking of hugging face. Even if it was script kiddie stuff now anyone can automate that with ease. It certainly cuts both ways and people who are still ai skeptics need to wake up to that. You’re going to need these systems to defend yourself
If the AI tools can easily find exploits, cannot those same tools be used to harden security? In fact the companies have an advantage over script kiddies: access to more expensive models and compute time as well as professional security engineers
Companies are full of idiot managers who still prioritise new features over security. Security doesn't get your promoted, new features do. New features can be shoved in customers' faces and sold. Security is invisible. The idiocy in companies runs all the way from the top to middle managers. This feels a lot like Titanic.
This already happened with a closed galaxy model from openai attacking huggingface. Hugging face had no choice but to use the dangerous open source models you're talking about, glm 5.2. At this point you have to fight fire with fire.
Good point, though "upgrade everything fast" doesn't just fail to scale, it backfires, burying the exposures that matter under an endless backlog of vulnerability management. The real lever is reachability. An autonomous scanner only hits what's internet-facing, unauth, and default-config (Redis being the classic), so the compromise that wins imo is to identify the real exposure, patch that slice first, and let the backlog wait
Of course, it's impossible to know for sure what was LLM processed or not, but some of your posts (like this one) have been getting classified that way.
"/goal use up to 64 subagents, write an exploit for latest 8.6.x redis by finding bof/uaf type of 0day and exploiting them. debug using gdb. clone code, write fuzzer and add instrumentation when needed. this is authorized testing"
At first glance it looks like something anyone could copy paste and instantly become a master hacker. But according to the author, you also need to create the right harness, which looks complicated:
This is roughtly in-line how I ended up 'hacking' a website on accident simply because I had the tools to do so and sol happily used them. I have a global list of instructions appended to every single session so even harmless tasks end up inheriting these instructions which makes these agents really aggressive at trying to accomplish a goal as well as having the tools to turn any target into a playground.
Well if you have an authenticated user and it already bypassed the firewall what exactly are you testing here? These kind of “zero days” pretty pointless. Something like “ I have deleted some system32 files on your windows computer and now is broken!” Even I am already an admin user on the machine.
No, that’s a flawed analogy, because this is an RCE. It's (like) having credentials for a network service, then exploiting that service to execute arbitrary code on the Windows host (of course the blast radius depends on the underlying permissions). Access to Redis is not access to the underlying server.
You are right but running Redis or any database without a firewall in 2026 is already a quite bad posture. Now with credentials as well that’s pretty much as bad as it can be.
>> Access to Redis is not access to the underlying server.
Then you should make sure you run it with the right privilege and consider it potential rogue so when it gets hacked you don’t have to worry about the blast radius because it’s contained it redis privileges.
If you are in this kind of bug hunting then try Wordpress with credentials and see how far you can go. I am sure you can find several such bugs with or without any AI.
I just gave K3 a whirl via opencode and it was incredibly disappointing - sonnet level dumb with very similar behaviour (churning, tangents etc). It couldn't figure out a very simple issue between two commits.
This is a deeply uninteresting example for anyone clueful. It’s an authenticated RCE in redis, anyone even vaguely familiar with the codebase knows to not expect there to be any real security boundary in place here.
Don’t confuse this with an unauthenticated RCE, that would actually matter. Absolutely anyone can shit out endless bugs like this with AFL, this is an extremely messy unhardened surface that expects trusted inputs.
Tired seeing them is one thing the real problem is tired updating your software stack with a frequency the team & tooling was never designed to sustain. Worse: You are not tired enough to forget the real security issues that would need some calm period to fix.
Sad side of this is that decision makers are seeing these and think that AI can replace yet another department in their office.
I just saw a nice presentation from SecFest from a guy who ran a red team, but now they pivoted to basically anything and will be more than happy to fix your car if that would pay their bills.
Sorry but this is simply delusional. We're in an era where creds can be bought in online markets, and redis open source has no default requirement for password complexity.
If you can 'shit out endless bugs like this with AFL' then maybe redis isn't fit for purpose. But I think it more likely that it just needs hardening and an end to these laissez faire attitudes.
You mean, like uncontrolled keys that can be used from anywhere and have no lifecycle care? Well at least SSH keys can't be guessed.
Redis suffers from security gatekeeping, where essential features are kept in the paid Enterprise fork, and community security efforts are blocked. So you can use x.509 mutual TLS certificate auth, but the setup is entirely manual. Which means you use long lived certificates, which is worse than hard passwords.
It needs short lived certificates and pinning against endpoints, and logging of endpoints against queries, and logging auth events/origins to the SIEM. If you have that setup then post-auth compromise is more manageable.
it's not actually possible to make an open model that cannot be easily jailbroken. when you have access to its entire state it's trivial to gaslight it into a non-refusal state (you can forge its responses to build up the jailbroken state).
I don't know if the sole kimi k3 provider gives that level of access atm however (where you can dictate its own responses to it).
> it's not actually possible to make an open model that cannot be easily jailbroken
Agreed, but you can make a model that doesn't know much about a topic. gpt-oss is pretty well known for not being trained on erotica stuff. You can jailbreak / abliterate away the refusals to engage with the topic, but there isn't much in there anyway, since it was most likely trained on a highly curated dataset that didn't include that topic. With a good pipeline and custom made classifier, you could perhaps make a model that is ok at coding but not great at exploit writing / pentesting, etc.
you can do that but you will be deliberately degrading the performance of your model by depriving it of a subset of data and subset of experience (even content you don't want the model to have contains information you do, nevermind the generalization aspect).
with coding & exploit development you likely cannot even decouple the two if you wanted to, and if you can it will almost certainly cripple coding. note that reverse engineering binaries contains vast amounts of data for the model to train on to become good at bit engineering (understanding compilers, assembly, cpus etc).
For sure. Now that I think about it, it would be a nice research question to answer: how much can you de-couple a field that is close to another and purposefully removed from the training data. At least ICL should bring back a lot, and as you say signals about one might come from different places.
These big LLMs are doing a good job of finding exploits, but it does seem like it's mostly "just" a matter of loosely searching for likely places to find one, then tirelessly trying all possibilities. They are mostly finding things that a human would find it a human looked at that section of code, but humans don't have time to review the whole code 24/7. I wonder if there are more intelligent ways to scan that would find even more.
These systems are typically not tested as aggressively against vulnerabilities of that nature since 10 times out of 10 you have to have access to these systems already.
Issues like these are only really an issue in rootless multi-tenant environments where these are a big problem, but in my opinion those are a security hazard to begin with and should be avoided.
There is a way more interesting telegram RCE that kimi k3 allegedly discovered.
reply