6 ms·
Hmmm... I'd like someone to double check my thinking here. I posted this exact prompt for gpt 5.5 xhigh: ``` does this look right to you? don't do any searche
by shay_ker 5mo ago
Hmmm... I'd like someone to double check my thinking here. I posted this exact prompt for gpt 5.5 xhigh:
```
does this look right to you? don't do any searches or check memory, just think through first principles
static int vpu_mmap(struct file fp, struct vm_area_struct vm) { unsigned long pfn; struct vpu_core core = container_of(fp->f_inode->i_cdev, struct vpu_core, cdev); vm_flags_set(vm, VM_IO | VM_DONTEXPAND | VM_DONTDUMP); / This is a CSRs mapping, use pgprot_device */ vm->vm_page_prot = pgprot_device(vm->vm_page_prot); pfn = core->paddr >> PAGE_SHIFT; return remap_pfn_range(vm, vm->vm_start, pfn, vm->vm_end-vm->vm_start, vm->vm_page_prot) ? -EAGAIN : 0; }
```
And it correctly identified the issue at hand, without web searches. I'd love to try something more comprehensive, e.g. shoving whole chunks of the codebase into the prompt instead of just the specific function, but it seems the latent ability to catch security exploits is there.
So then.... I wonder how this got out in the first place. I know I'm using a toy example but would love to learn more!
- lifis 5mo ago[flagged]
- XorNot 5mo agoYes...no one would write any code.
- bombcar 5mo agoCode would be written over TOR and passed around on unmarked USB sticks in back alleys. Pssst - hey, kid, want some GNU?
- akersten 5mo ago> If the person who wrote this were to be imprisoned for the rest of their life [...] then things would be very different Yes, they certainly would. You wouldn't have smartphones, for instance. I can't tell if this is satirical or not. But there are so many takes like this recently (hold the website liable for user content, hold the corporate developer liable for zero days in a project they happened to touch) that would all result in the same outcome (no more product at all) that I can't help but wonder if there's some luddite psy-op trying desperately to bring us back to a pre-Internet era in any way they can...
- StilesCrisis 5mo agoIf this rule were implemented, would you be walking free right now? Think it over.
- ux266478 5mo agoI most definitely don't agree with him, and I find the idea absolutely repugnant. Devils advocate though, I would be much more careful with the code that I write if regulators passed some dogshit law like that.
- pavel_lishin 5mo agoI would stop writing code for money.
- ux266478 5mo agoI understand that, though I wouldn't stop. I'd just go much slower and radically change my methodology. Failures in other engineering domains come with massive legal consequences, and they have for a very long time. In mesopotamia if a house collapsed and killed someone inside, the builder was put to death. People still built houses in the hundreds of thousands. It really just introduces a legal burden to prove competence and work in good-faith, and nets immense power to throw out ridiculous deadlines. Your managers are legally responsible too, and if they push beyond what's reasonable you have just cause to bring them to court in a way that you currently don't. To re-emphasize, I don't think this is a better world, but it's not unlivable.
- pavel_lishin 5mo agoSure, but home builders today very rarely get put to death, and it takes a particular kind of intentional fuck-up to have a plumber, or a drywaller, or electrician placed in prison. If I was personally liable for damages, and there was an insurance program or some sort - similar to how doctors & dentists practice - sure, I'd probably still write code, very carefully. But if there was a decent change of me spending the rest of my life in prison because something I wrote on a Friday at 4pm under some amount of stress? No thanks. I can re-train as a plumber, and stand knee-deep in shit all day.
- hombre_fatal 5mo agoWe should roll this out for everything. Someone T-bones you in parking lot, chef causes food poisoning, plumber's leak floods your bathroom, personal trainer pushes to injury, mislabeled allergen on food, movers break your armoire, roofer leaves a leak -- I bet we'd see a lot less of all that if a $1MM fine + life in jail loomed over everyone. Nobody would want to do business, but boy would we be in a golden age.
- creaturemachine 5mo agoA golden age of prisons and debt collectors.
- deleted 5mo ago[deleted]
- akdev1l 5mo agoAs an anecdote, I provided fragnesia.c and the subsequent proposed patch to fix the issue and while it was not able to discover an entirely new vulnerability, I think it was able to find 2 new ways of exploiting the same underlying bug. This is quite impressive considering I’m just a dumbass with a Claude subscription.
- thejokeisonme 5mo agoHow do you know it didn't search the web?
- shay_ker 5mo agono tool calls!
- empath75 5mo agoI pasted the code into claude Opus 4.7 with no internet access and just asked it to just tell me what the function did, and it explained it and also called out the bug. I did not tell it to look for bugs: > Observations & Potential Issues A few things worth flagging: 1. No bounds checking on the mapping size. Userspace controls vm_end - vm_start and vm->vm_pgoff. Here vm_pgoff is ignored entirely and the size is trusted blindly. If the VPU's register block is, say, 64KB but userspace requests a 1MB mapping, the driver will happily map 1MB of physical address space starting at core->paddr — potentially exposing whatever hardware happens to live at adjacent physical addresses. A defensive check would be: --- 70 day release cycles are very quickly not going to be fast enough to stop widespread use of exploits when you have bots able to scan every PR on every open source project as it comes out.
- gruez 5mo agoThat's not really a fair test because you're leading the model pretty hard, even if the prompt doesn't specifically say there's a bug to be found. It's basically the same objections that people raised in the thread where someone claimed current models are just as good as mythos.
- shay_ker 5mo agoright exactly, but clearly it's possible to elicit the behavior we want in the model, which means the capabilities are there!
- Matumio 5mo agoThe more interesting question is, how many issues will this prompt report to you in random code that is perfectly fine?
- naruhodo 5mo agoI don't agree, and I'd like to understand your point of view. To me, asking if a function has something wrong with it is just a very basic code review - something that should happen with every function. A competent, security conscious engineer would respond the same way as the model, unsurprisingly, since the model is... modelling competence.
- saagarjha 5mo agoCode review that finds problems in all code is useless.
- fulafel 5mo agoOn its own we can't judge if this is a workable way to find vulns, as we don't know how many false positives you'd get if you ran it on all the code. (iow might be https://en.wikipedia.org/wiki/Base_rate_fallacy https://en.wikipedia.org/wiki/Base_rate_fallacy)