3 ms·
OP here: for those interested is not that an LLM found the bugs the fuzzer found them. My take on this matter was to implement as much information theory algori
by dclavijo 1mo ago
OP here: for those interested is not that an LLM found the bugs the fuzzer found them.
My take on this matter was to implement as much information theory algorithms as possible, and try to extract as much structure with statistical importance from the binary being fuzzed. Also port as much features from other fuzzers and whie papers on the mater (llms are good at connecting dots across vast codebases and papers).
I honestly can not take full credit for this work since I made it with AI, but I has taken two months of my time and 1100+ commits.
My developing process was to use several models from several vendors not just Claude that decouples it from a single vendor/model and throws to the flor that llms regurgitate verbatim code.
Also the interesting part is the developing pipeline I have had setup my own cicd with my own tool impactguard whitch saved me a couple of times and hard rules on the agent.md(70% to 80% of those rules i wrote them by hand).
The pytest testing battery is also interesting, I adopted TDD and to me since I adopted it seems that llms make less bugs.
Yes I know the code and the readme might look like ai slop as pointed out earlier but is efective at finding bugs. At the end of the day is all economy: you spend a lot of tokens once and keep the fuzzer forever, not the same as paying every time for tokens to find bugs.
As pointed out in the readme, this fuzzer trades speed for edge novelty, maybe there is it's niche.
Also we found earlier another bug with this fuzzer https://code.ffmpeg.org/FFmpeg/FFmpeg/issues/23945 https://code.ffmpeg.org/FFmpeg/FFmpeg/issues/23945.
For the concerned IMO: rather than the results the methodology is more important.
I welcome constructive criticism and feedback. Any input is useful to me.
- nixpulvis 1mo agoI can't speak to the quality of the fuzzer since I haven't used it or looked at it thoroughly, but it does seem to cover a lot of ground on features and interesting concepts. I'll definitely be reading more into what you have here.