4 ms·

This describes my life as an open source maintainer at the moment! In the first 10 years of the rclone project we received about 20 security disclosures throug

by nickcw 1mo ago

This describes my life as an open source maintainer at the moment! In the first 10 years of the rclone project we received about 20 security disclosures through GitHub. We had to deal with over 40 in the last month! That has taken a huge amount of my time, even using AI tools to triage and come up with fixes for review. The hit rate for those security disclosures is pretty good - about 75% of them have a nugget of something which needs looking at. The configurations for rclone have got increasingly unlikely so I'm hoping they will dry up eventually. I was considering just merging the fixes straight to master just to make my life easier rather than holding a dozen independent security fixes on branches and merging them at the point release and hoping not to have too many conflicts to fix up. I've decided to stick with the process for the moment. GitHub assigns CVEs for the advisories. Before the AI apocalypse they took 2-3 days for an assignment but now it they are running at 3-4 weeks so I have to send the point releases out with CVE-PENDING in the changelog which isn't ideal. Not sure what the solution is, but it is definitely a problem for us.

Kubuxu 1mo ago
As long as you are not running a paid bounty program. Otherwise now you are getting 40 per day.
zmgsabst 1mo ago
I can’t comment on if it applies to your workflow, but one process I’ve used is to aggregate and land ~10 security patches at a time. Eg, - grab a group of (related) bugs/defects/vulns - fix them on a branch like bug-batch-XXX - run that group through the verification, landing in main, CI/CD flow to amortize process cost - repeat as needed to process backlog My experience is that process often has irreducible time (eg, two days due to reviews by various parties); but that time slot can be shared between several bugs in a single PR — especially if you have several related to the same feature.
dataviz1000 1mo ago
> even using AI tools to triage Can you discuss this? I might be able to help.
dannyw 1mo ago
Thank you for making and maintaining rclone. It is truly a blessing.
- jiggawatts 1mo ago
  I came across rclone at work because "Copy or move data to Azure Storage by using AzCopy v10" is a lie, it literally can't `move` files, only `copy` them. I can't express in polite words how pathetic it is to see the only official blob storage bulk transfer CLI tool from a multi-trillion-dollar company fail to do the simplest, most essential functionality after ten major revisions. Meanwhile, rclone Just Works(tm). Thank you from me too!
  - stavros 1mo ago
    Do not try and move the files; that's impossible. Instead, only try to realize the truth... Delete them after you copy.
    - rq1 1mo ago
      Bend the spoon* * for people who don’t know the ref
      - dingdongditchme 1mo ago
        The star only confuses me more...?
        zdragnar 1mo ago
        It's a pointer to a reference you don't have. Also, a line from a scene in the Matrix.
      - deleted 1mo ago
        [deleted]
    - TomatoCo 1mo ago
      Right? If you move between file system or device I would expect "move" to be impossible. At best you could copy a logical block and then delete a logical block. If you share a filesystem then perhaps a logical block is a few hundred kilobytes. If you don't then I'd expect the smallest logical block to be the file itself.
caminanteblanco 1mo ago
Thank you so much for maintaining rclone! That project is literally the only thing that makes Google Drive an acceptable experience on Linux.
cyanydeez 1mo ago
I feel like someone somewhere is working on an opensource GPU compute resource pool you guys could pull from as part of a donation type framework. I got a bunch of local gpu resources just relaxing, and if I could load up a a binary to provide compute for X hours a day overnight or whatever, that'd be cool.
- nickcw 1mo ago
  I got a 6 month free subscription of Claude Code Max under Anthropic's open source program which has been very helpful.
- gazarsgo 1mo ago
  http://github.com/buzz/block http://github.com/buzz/block is quickly becoming a reasonable approach to shared agent compute, though it's still a bit more hands-on than the older 'compute donation' projects like SETI or Folding@Home.
  - cpach 1mo ago
    404?
    - majorchord 1mo ago
      https://github.com/block/buzz https://github.com/block/buzz
yjftsjthsd-h 1mo ago
> GitHub assigns CVEs for the advisories. Before the AI apocalypse they took 2-3 days for an assignment but now it they are running at 3-4 weeks so I have to send the point releases out with CVE-PENDING in the changelog which isn't ideal. A strange bottleneck; anyone know why that would be so slow?
- htrp 1mo ago
  human triage in review
- bobmcnamara 1mo ago
  Trying to keep the rest of GH online?
- jamietanna 1mo ago
  https://github.blog/security/supply-chain-security/inside-the-advisory-database-and-what-happens-when-vulnerability-volume-breaks-records/ https://github.blog/security/supply-chain-security/inside-th... goes into it more As mentioned by another commenter, they intentionally have a human reviewer in the process before a GitHub Security Advisory (GHSA) becomes "official" post-publish (this is separate to getting CVE ID assigned, if wanted)
x______________ 1mo ago
> Not sure what the solution is Possibly some sort of ai agent code review system that churns through code looking for these vulns before the code is published. It feels like it's all about who has the resources to find bugs at the moment but that it should be a standard to catch issues before prod moving forward..
- pixl97 1mo ago
  Yea, AI bug finding over entire projects, at least on SOTA models is super expensive, hence those with the resources setup to find the most bugs in an automated way. That said there are a number of people and companies working on more focused means of driving the LLM to look were bugs would be the most dangerous and in doing so reduce the token spend of each bug found. In some ways the better you are at security stuff the more you can reduce your spend by better driving the LLM to problem spots.
ftchd 1mo ago
> I was considering just merging the fixes straight to master just to make my life easier I felt that. The problem with doing that is you hate yourself afterwards so not a good solution either, gotta do it properly. Thanks for working on rclone, Nick!
Groxx 1mo ago
You can add CVE info later in a git note, if you want something git-managed to point to it. Might take some habit-tweaking to include notes in log output though, to see them later.
darig 1mo ago
[dead]
dingdongditchme 1mo ago
Thank you for the insight! Are you going back to the changelog and editing the CVE-PENDING? I think this would be a justifiable edit of the release history that I'm not sure is possible on github.
- nickcw 1mo ago
  Yes I update the CVE-PENDING once the CVEs are in to keep the record straight.
thedonncha 1mo ago
Same with some of the projects I maintain. The reports came in for one project over a few months and then once those dried up, and they did become increasingly unlikely or appear to be edge cases, the reports started to accumulate for another project. Great to get these issues addressed but it's exhausting.
chews 1mo ago
Thank you good human, you're being that little block in the xkcd comic.
jamietanna 1mo ago
If it helps, in the Renovate project we don't usually request a full CVE ID, but use the GitHub Security Advisory (GHSA) ID You can use those numbers in changelogs as soon as they're published There is some human review, which takes time, before the GHSA is "GitHub-reviewed" which then allows security scanners to pick it up Then, the CVE ID can be assigned to it after-the-fact if need be, but the GHSA should be a sufficient starting point

by nickcw 1mo ago

Kubuxu 1mo ago

As long as you are not running a paid bounty program. Otherwise now you are getting 40 per day.

zmgsabst 1mo ago

I can’t comment on if it applies to your workflow, but one process I’ve used is to aggregate and land ~10 security patches at a time. Eg, - grab a group of (related) bugs/defects/vulns - fix them on a branch like bug-batch-XXX - run that group through the verification, landing in main, CI/CD flow to amortize process cost - repeat as needed to process backlog My experience is that process often has irreducible time (eg, two days due to reviews by various parties); but that time slot can be shared between several bugs in a single PR — especially if you have several related to the same feature.

dataviz1000 1mo ago

> even using AI tools to triage Can you discuss this? I might be able to help.

dannyw 1mo ago

Thank you for making and maintaining rclone. It is truly a blessing.

jiggawatts 1mo ago
I came across rclone at work because "Copy or move data to Azure Storage by using AzCopy v10" is a lie, it literally can't `move` files, only `copy` them. I can't express in polite words how pathetic it is to see the only official blob storage bulk transfer CLI tool from a multi-trillion-dollar company fail to do the simplest, most essential functionality after ten major revisions. Meanwhile, rclone Just Works(tm). Thank you from me too!
- stavros 1mo ago
  Do not try and move the files; that's impossible. Instead, only try to realize the truth... Delete them after you copy.
  - rq1 1mo ago
    Bend the spoon* * for people who don’t know the ref
    - dingdongditchme 1mo ago
      The star only confuses me more...?
      - zdragnar 1mo ago
        It's a pointer to a reference you don't have. Also, a line from a scene in the Matrix.
    - deleted 1mo ago
      [deleted]
  - TomatoCo 1mo ago
    Right? If you move between file system or device I would expect "move" to be impossible. At best you could copy a logical block and then delete a logical block. If you share a filesystem then perhaps a logical block is a few hundred kilobytes. If you don't then I'd expect the smallest logical block to be the file itself.

caminanteblanco 1mo ago

Thank you so much for maintaining rclone! That project is literally the only thing that makes Google Drive an acceptable experience on Linux.

cyanydeez 1mo ago

I feel like someone somewhere is working on an opensource GPU compute resource pool you guys could pull from as part of a donation type framework. I got a bunch of local gpu resources just relaxing, and if I could load up a a binary to provide compute for X hours a day overnight or whatever, that'd be cool.

nickcw 1mo ago
I got a 6 month free subscription of Claude Code Max under Anthropic's open source program which has been very helpful.
gazarsgo 1mo ago
http://github.com/buzz/block http://github.com/buzz/block is quickly becoming a reasonable approach to shared agent compute, though it's still a bit more hands-on than the older 'compute donation' projects like SETI or Folding@Home.
- cpach 1mo ago
  404?
  - majorchord 1mo ago
    https://github.com/block/buzz https://github.com/block/buzz

yjftsjthsd-h 1mo ago

> GitHub assigns CVEs for the advisories. Before the AI apocalypse they took 2-3 days for an assignment but now it they are running at 3-4 weeks so I have to send the point releases out with CVE-PENDING in the changelog which isn't ideal. A strange bottleneck; anyone know why that would be so slow?

htrp 1mo ago
human triage in review
bobmcnamara 1mo ago
Trying to keep the rest of GH online?
jamietanna 1mo ago
https://github.blog/security/supply-chain-security/inside-the-advisory-database-and-what-happens-when-vulnerability-volume-breaks-records/ https://github.blog/security/supply-chain-security/inside-th... goes into it more As mentioned by another commenter, they intentionally have a human reviewer in the process before a GitHub Security Advisory (GHSA) becomes "official" post-publish (this is separate to getting CVE ID assigned, if wanted)

x______________ 1mo ago

> Not sure what the solution is Possibly some sort of ai agent code review system that churns through code looking for these vulns before the code is published. It feels like it's all about who has the resources to find bugs at the moment but that it should be a standard to catch issues before prod moving forward..

pixl97 1mo ago
Yea, AI bug finding over entire projects, at least on SOTA models is super expensive, hence those with the resources setup to find the most bugs in an automated way. That said there are a number of people and companies working on more focused means of driving the LLM to look were bugs would be the most dangerous and in doing so reduce the token spend of each bug found. In some ways the better you are at security stuff the more you can reduce your spend by better driving the LLM to problem spots.

ftchd 1mo ago

> I was considering just merging the fixes straight to master just to make my life easier I felt that. The problem with doing that is you hate yourself afterwards so not a good solution either, gotta do it properly. Thanks for working on rclone, Nick!

Groxx 1mo ago

You can add CVE info later in a git note, if you want something git-managed to point to it. Might take some habit-tweaking to include notes in log output though, to see them later.

darig 1mo ago

[dead]

dingdongditchme 1mo ago

Thank you for the insight! Are you going back to the changelog and editing the CVE-PENDING? I think this would be a justifiable edit of the release history that I'm not sure is possible on github.

nickcw 1mo ago
Yes I update the CVE-PENDING once the CVEs are in to keep the record straight.

thedonncha 1mo ago

Same with some of the projects I maintain. The reports came in for one project over a few months and then once those dried up, and they did become increasingly unlikely or appear to be edge cases, the reports started to accumulate for another project. Great to get these issues addressed but it's exhausting.

chews 1mo ago

Thank you good human, you're being that little block in the xkcd comic.

jamietanna 1mo ago

If it helps, in the Renovate project we don't usually request a full CVE ID, but use the GitHub Security Advisory (GHSA) ID You can use those numbers in changelogs as soon as they're published There is some human review, which takes time, before the GHSA is "GitHub-reviewed" which then allows security scanners to pick it up Then, the CVE ID can be assigned to it after-the-fact if need be, but the GHSA should be a sufficient starting point