9 ms·

Openrsync: An implementation of rsync, by the OpenBSD team

492 points by sph 5mo ago

triggis 5mo ago
[flagged]
- ranger_danger 5mo ago
  https://social.treehouse.systems/@thesamesam/116662824873341085 https://social.treehouse.systems/@thesamesam/116662824873341...
  - akerl_ 5mo ago
    +1 to this. Other than people's reflexive anger or fear about AI coming for their code, I don't see anything to suggest that these are bugs that are due to the inclusion of AI vs bugs in a program with a bunch of complex interop with the filesystem and network.
    - triggis 5mo ago
      In any case, it's important to identify projects that are beginning to actively vibecode and clearly express position on this issue on various platforms so that authors and maintainers receive feedback. Even if this particular bug was not written by LLM in this particular case, it's not a fact that the release does not include other regressions and that subsequent vibecoded versions will not include them & new ones.
      - stsquad 5mo ago
        Do not going harassing developers because you think they are doing it wrong. If you can do better and don't want to actually contribute to the upstream you are always free to fork it.
        margalabargala 5mo ago
        That's literally what this thread is about.
        akerl_ 5mo ago
        Openrsync is a project that’s existed for a long time and is a great example of people building what they want. This thread has people suggesting it’s good or important to go tell open source devs they’re using bad tools. The former is great. The latter is not.
      - akerl_ 5mo ago
        No. It's not important. It's actually pretty shitty to go around looking for projects and then telling the maintainers you disagree with how they develop.
      - skeledrew 5mo ago
        > it's not a fact that the release does not include other regressions and [...] Are you listening to yourself? The same exact thing also has applied, applies and will continue to apply to manually written code, in perpetuity. There's nothing new under the sun here; regressions happen when there's change, and the only way to mitigate is to have healthy feedback loops.
      - applfanboysbgon 5mo ago
        Friendly reminder that volunteer maintainers owe you literally not a single goddamn thing. I absolutely want no AI slop in my commercial products that I pay money for, but your feedback is not important to people you are not paying to develop software for you. They gave away not only their software but the source code for free; if you have a problem with it, fork it. Which is something you can do with their generous allowance, and that is an allowance any maintainer can instead choose to not bother themselves with if publishing their code for free leads themselves to dealing with entitled internet commenters harassing them with complaints.
        teddyh 5mo ago
        > Friendly reminder that volunteer maintainers owe you literally not a single goddamn thing. Technically true. However, I also do not owe them my silence.
        deleted 5mo ago
        [deleted]
    - 48terry 5mo ago
      Yeah, nothing to suggest that here. We went from some commits every so often (because rsync was basically finished software) with a handful of issues a month, to "a bajillion Claude commits then a release" with a pile of issues within weeks, at least one of which is about how the goddamn thing doesn't build. I can't possibly see a correlation. Weird how 3.4.2. didn't have a similar deluge of issue reports, though.
jmclnx 5mo ago
I have not checked with OpenBSD 7.9, but as of 7.8 it did not support --exclude or -z. But outside of that openrsync works great. (EDIT: --exclude is now supported on 7.9. Not sure when that was added, nice!) But seems avoiding "slop" is getting very hard. I saw postfix now has a bit of AI code in it. https://mastodon.sdf.org/@mrmasterkeyboard@mastodon.social/116663069350165149 https://mastodon.sdf.org/@mrmasterkeyboard@mastodon.social/1...
- nineteen999 5mo ago
  Somewhat ironic Postfix has a record of no root/RCE in the default install, where opensmptd hasn't (CVE-2020-7247). Time will tell if it stays that way.
- Bender 5mo ago
  Exclude is very commonly used in automation jobs to avoid duplicating big git repos and other big files. I think that would be a show stopper for a number of people.
  - jmclnx 5mo ago
    I just tried openrsync(1) on OpenBSD 7.9, --exclude now works. I have not tried using exclude in openrsync in a while, but I can see it now works on OpenBSD 7.9!
- agwa 5mo ago
  Where do you see that about Postfix? I followed the links and the only thing I see is that AI is being used to find bugs, not write code.
  - jmclnx 5mo ago
    >Claude assisted code found in external/ibm-public/postfix/dist... That is from the original post in the thread. Is that really due to LLM ? I do not know since I avoid AI as much as I can. But the person also posted this link too: https://github.com/NetBSD/src/commit/f764ddf4062e855f73fe2e31b9c378ddd0155a4e https://github.com/NetBSD/src/commit/f764ddf4062e855f73fe2e3...
skeledrew 5mo ago
This attempt to avoid things that use AI is increasingly looking like some weird kind of reverse whack-a-mole where each targeted hole becomes radioactive after. Just grabbing some popcorn to watch.
- ranger_danger 5mo ago
  I feel bad for people with the real name Claude.
  - formerly_proven 5mo ago
    It took me quite some time to realize what an utterly presumptuous product name Claude Code actually is, but only because Shannon is rarely mentioned with his first name. It's golden calf levels of hubris, even more so if you consider how incapable it was on release. It's like renaming calc.exe Einstein. Incredibly poor taste, but entirely in line with AI tech bro mentality.
    - kstrauser 5mo ago
      That linkage never occurred to me, or, I suspect, them. Claude use to be a reasonably common name. I have an uncle Claude. Why do you believe they named it after Shannon in particular?
      - homebrewer 5mo ago
        It seems to be a widely repeated "fact" which can't be traced to anything particularly authoritative: https://archive.is/pt5fQ https://archive.is/pt5fQ https://britannica.com/topic/Claude-AI https://britannica.com/topic/Claude-AI Looks like the 2023 NYT article started it, and it uses this as reference: > depending on which employee you ask, was either a nerdy tribute to the 20th-century mathematician Claude Shannon Personally I always associated it with the silent protagonist from GTA3. https://gta.fandom.com/wiki/Claude https://gta.fandom.com/wiki/Claude
    - bombcar 5mo ago
WD-42 5mo ago
What's the deal with the name? Openrsync implies to me that it's an open source alternative to a closed source program. But the original Rsync is GPL? Is this just the pushover license making it "more open"?
- jtickle 5mo ago
  OpenBSD folks would consider the GPL to be less open due to the requirement to apply the GPL to any derivative works.
  - ranger_danger 5mo ago
    And GNU folks would say the GPL is actually the more open choice because it forces the project to stay open. Two different ways of thinking about it I guess... it's nice to have choices and I don't think one is more or less "correct", more a matter of opinion/taste I guess.
    - gilrain 5mo ago
      > more open choice because it forces the project A true morality must be based on consent, not coercion. Humanity may not be there yet, and therein lies the argument for force (and thus copyleft); but the ultimate goal should always be to reduce its necessity.
      - kennywinker 5mo ago
        Is this not the paradox of tolerance restated in different terms? BSD license is unrestricted, it tolerates taking open source and closing it, thus always being at risk of things closing down. GPL license doesn’t tolerate taking from open source and closing it, thus ensuring things stay open.
        KZerda 5mo ago
        The BSD license is why we have Valkey and not a purely closed-source Redis. It would have been much easier to perform the rugpull if Redis had initially been GPLed.
Bender 5mo ago
The actual work of porting is matching the security features provided by OpenBSD's pledge(2) and unveil(2). These are critical elements to the functionality of the system. Without them, your system accepts arbitrary data from the public network. https://justine.lol/pledge/ https://justine.lol/pledge/ I am not seeing pledge on Alpine Linux in edge. Have people been testing Pledge on Linux? Did I perhaps misunderstand the risk of using Openrsync without pledge? Or is this article just for OpenBSD users?
- e12e 5mo ago
  From above your quote: > The only officially-supported operating system is OpenBSD, as this has considerable security features. And below your quote: > This is possible (I think?) with FreeBSD's Capsicum, but Linux's security facilities are a mess, and will take an expert hand to properly secure. It is portable in the sense that it compiles and runs, not in the sense that it has the same security features. I'd love to see pledge/unveil on (upstream) Linux - but I'm not holding my breath.
  - Bender 5mo ago
    Ok that makes more sense, thankyou.
  - papercrane 5mo ago
    > I'd love to see pledge/unveil on (upstream) Linux - but I'm not holding my breath There is Landlock now, I believe it would be possible to implement unveil and pledge on top of that.
    - isityettime 5mo ago
      One of HN's favorite hackers has done that: https://justine.lol/pledge/ https://justine.lol/pledge/ There was a discussion here about it a few years ago: https://news.ycombinator.com/item?id=32096801 https://news.ycombinator.com/item?id=32096801
      - 5mo ago
tptacek 5mo ago
rsync has specific running modes for the super-user. It also pumps arbitrary data from the network onto your file-system. openrsync is about 10 000 lines of C code: do you trust me not to make mistakes? No, but that's why almost nobody runs it outside of strict trust boundaries. This security section would make more sense if rsync was like curl, which routinely deals with hostile counterparties. If the other side of your rsync is hostile, you probably have bigger problems! (I'm not an rpki person so I don't know if there's some part of that problem domain that changes this equation. I'm not dunking on the project, just saying this snagged me in the README).
- cperciva 5mo ago
  No, but that's why almost nobody runs it outside of strict trust boundaries. This security section would make more sense if rsync was like curl, which routinely deals with hostile counterparties. If the other side of your rsync is hostile, you probably have bigger problems! I disagree. While rsync is most often used to transfer data between "friendly" systems, it's inherently crossing a security boundary. It's important to make sure that an attacker can't leverage it to transform the breach of one system into the breach of multiple systems.
  - eikenberry 5mo ago
    It is almost universally hooked up using ssh tunneling so ssh takes care of the security boundary and ssh is well trusted.
    - cperciva 5mo ago
      That solves the traffic tampering problem, but not the "malicious peer" problem. You want to be able to sync files without accidentally sharing root privileges.
- delusional 5mo ago
  > almost nobody runs it outside of strict trust boundaries. I guess you can define "strict" however you want, but from what I saw ~10 years ago, most linux distros handled mirroring with rsync. That's a lot of usage in a pretty core part of the foundational open source ecosystem.
denysvitali 5mo ago
There's also a Go implementation by Michael Stapelberg / the Gokrazy team: https://github.com/gokrazy/rsync https://github.com/gokrazy/rsync
- mtlynch 5mo ago
  The gokrazy team being Michael Stapelberg : ) https://github.com/gokrazy/rsync/graphs/contributors https://github.com/gokrazy/rsync/graphs/contributors
  - denysvitali 5mo ago
    I mean, yes, mostly him (and he gave multiple presentations about this topic, very interesting!). - but there are other contributors too :)
    - dolmen 5mo ago
      stapelberg: 329 commits jonas-fi: 15 commits others: 1 commit each The bus factor is 1.
thefilmore 5mo ago
This is the version used in macOS since 15.0.
- mrdomino- 5mo ago
  Was it 15.0? I seem to recall it coming in one of the minor point releases in the 15.x line - and I remember it breaking some scripts mysteriously. EDIT: ah, fun: they did include it in 15.0, but they decided to save the breaking change that removed backwards compatibility for 15.4. https://apple.stackexchange.com/a/479297 https://apple.stackexchange.com/a/479297
chasil 5mo ago
There is also a (stub) web page: https://www.openrsync.org/ https://www.openrsync.org/ The problem with this fragmentation of rsync is that Apple and Android will prefer it, but the Linux and greater GPL world will adhere to the original implantation due to inertia. Power users will just have to know the quirks of each version. The only way to stop this is for the original author(s) to release this under a BSD license. Edit: For those assuming equivalent/identical behavior, study these words carefully: "accepts only a subset of rsync's command-line arguments."
- qalmakka 5mo ago
  Basically like GNU Tar/CPIO and BSD Tar/CPIO. I've largely standardised towards using the bsd variant everywhere (especially since now even Windows ships it and it handles lots of other archive formats using the `tar` command) but it's always a pain to install it everywhere
  - yjftsjthsd-h 5mo ago
    Yeah, I'm leaning towards strongly preferring bsdtar since it's happy to work on e.g. zip files:) Does it have any real downsides?
    - SoftTalker 5mo ago
      Limits on the length of path and file names in the archive. Edit: I see that they switched from ustar to pax as the default format in OpenBSD 7.6, so I guess this isn't true any longer.
- spauldo 5mo ago
  It's really no different than every other BSD utility (and SysV utility, if you're running one of those) being different than the GNU ones. We've coped with it for fifty years at this point.
- yjftsjthsd-h 5mo ago
  > The only way to stop this is for the original author(s) to release this under a BSD license. Would that stop it? My understanding was that at least OpenBSD tended do redo things for technical reasons, not just licensing
Panino 5mo ago
I've been using openrsync here and there since it was announced and it's definitely improved over time. I'm looking forward to when I can use it exclusively. The one place in my usage where it doesn't match Samba rsync is with the following: openrsync --rsync-path=openrsync -av -e ssh /etc/services example.com:/tmp/services I would expect openrsync to create a remote file /tmp/services, but instead it creates /tmp/services/services. Normal directory mirroring as in -av -e ssh /path/to/src/ example.com:/path/to/dst/ works as it does with Samba rsync.
- genxy 5mo ago
  Was there already a /tmp/services directory on the dest? One of the biggest points of confusion with rsync is how directories and trailing slashes are handled.
  - Panino 5mo ago
    > Was there already a /tmp/services directory on the dest? No. And just to make sure, I ran a quick 'rm -rf /tmp/services' on the remote host, then re-ran openrsync on the client. Same result. This is OpenBSD 7.9 on both sides. And I 100% agree about trailing slashes.
  - anyfoo 5mo ago
    I hear that a lot, but I familiarized myself with it once and ever since it makes a lot of sense to me. Source ending in “/“: You want what’s inside. Source not ending in “/“: You want the thing (i.e. directory itself). For the destination, it does not matter whether it ends in “/“ or not, but for consistency I like adding a “/“ anyway (I want to put thing inside the directory).
    - linsomniac 5mo ago
      Yes, the nice thing about dest having a trailing "/" is that if it exists and is NOT a directory, you are alerted right away.
  - eichin 5mo ago
salvesefu 5mo ago
For those needing context for the development of this package; this project is presently being developed as part of a RPKI validator. https://medium.com/@jobsnijders/a-proposal-for-a-new-rpki-validator-openbsd-rpki-client-1-15b74e7a3f65 https://medium.com/@jobsnijders/a-proposal-for-a-new-rpki-va...
SubiculumCode 5mo ago
I'm going to ask a question. I could ask chatgpt. I could Google it. I am asking a question because it is human to do so. Ubuntu's packaged rsync, is it Samba rsync? Why reimplement it?
- hideout_berlin 5mo ago
  samba is different topic
- eichin 5mo ago
  it's tridge rsync; samba is another project by the same guy. (rsync was originally a PhD thesis...)
  - bombcar 5mo ago
    tridge is mighty in the Linux world, he arguably greatly contributed to Linux as a server (via Samba to save on windows NT client user licenses), rsync, and IIRC reverse-engineering the bitlocker protocol thereby starting the gitstorm.
- tredre3 5mo ago
  Ubuntu's rsync is samba rsync. It's not part of the samba project per se, but it is made by the same guy and the official url is https://rsync.samba.org/ https://rsync.samba.org/ so it's entirely fair to call it samba rsync in my opinion.
onedognight 5mo ago
They don’t support any recent rsync protocol, so there’s no 64bit timestamp support, so you can never actually sync metadata across newer filesystems.
throwaway27448 5mo ago
I'm confused. Isn't rsync already free software? What are we doing here. Why are we trying to cuck ourselves for capital. I like open bsd but this just seems like burning cash
- somat 5mo ago
  My understanding is that much of the point of openrsync is to create a second implementation of a protocol so the standards bodies don't balk at including it in their standards. Or to put it more concretely, people working on the rpki standard(who happened to also be openbsd devs) wanted to use rsync to transfer bulk data. The standards body was hesitant, while rsync is ostensibly a documented protocol, there was only one implementation. So in true openbsd fashion they rolled up their sleeves and wrote that second implementation. On use, there is nothing wrong with openrsync, however it may never hit feature parity with rsync, that is not a goal of the project, they want a specific subset of rsync features to support their rpki needs. If anyone else finds this useful that is great. So I suspect users will be those who want a bsd licensed rsync(apple) or them who are willing to give up features for openbsd quality code(myself).
  - throwaway27448 5mo ago
    Why would we explicitly develop code legally available to capital?
    - reverius42 5mo ago
      Are you new to the concept of open source? https://en.wikipedia.org/wiki/Open_source https://en.wikipedia.org/wiki/Open_source There's some discussion here that might help answer your question: https://en.wikipedia.org/wiki/Open_source#Economics https://en.wikipedia.org/wiki/Open_source#Economics
    - avadodin 5mo ago
      Most people using open licenses over right–restricting GPL–style licenses do so for philosophical reasons. They don't necessarily think they are better for getting reciprocal contributions from evil corporations even though many do argue that today, especially after GPLv3. See the redis licensing debacle and why and how it was resolved. If I understand it correctly, the FSF could technically sell all of GNU tools to Microsoft which could in turn keep them closed source including all contributions over the past 40 years. Not without losing all of their remaining credibility, but still.
trollbridge 5mo ago
Given the sudden spike in vibe-coded commits to the rsync codebase, and regressions that’s introduced, this is very good news.
- port11 5mo ago
  I was prepared to dismiss your comment because rsync has always been rock solid, but indeed upgrading broke my backup script. The latest issue on GitHub documents plenty of bugs introduced in the last 2 patches, including a monstrous ~9k LOC commit that was probably pointless. LLMs make writing code faster/easier, but the thinking was always the important bit. I’ve no idea why you’d muck up such a long-standing, reliable piece of software.
  - prmoustache 5mo ago
    When LLM allows you to produce code n% faster, it also allows you to introduce bugs n% faster. I find it quite strange that people do not seem to be aware of that...I think many started worshipping the tool as if it was some kind of divinity and lost all objctivity. This doesn't bode well for the future if people aren't able to review code anymore.
    - brianwawok 5mo ago
      Don’t need people to review code, need more LLMs. People aren’t that magical.
      - port11 5mo ago
        Maybe one day. Right now I can have Opus carefully draft a plan, Sonnet execute it, and have Opus review the code. It will complain about things that it had dismissed in the plan. Example: it recently flagged ID derivation from a signing key. Opus itself suggested it! When reminded of that, it did this new 4.8-style “my bad, and here’s why I was right in the first place.” Humans are still badass.
- asdsg 5mo ago
LAC-Tech 5mo ago
A few comments here suggest rsync is undergoing some "churn" which tbh is highly undesirable for a command line utility. Might switch over. As an aside I really love the stuff openBSD puts out. If they ever succeed in making a modern journalling filesystem I will probably switch over.
TZubiri 5mo ago
Might be offtopic, but i think when looking for alternatives the options should be broad, questioning the category of the app, not just clones and forks. If you are considering migrating away from github, don't just consider gitlab and gitea, consider just git, and so on.. If you are considering migrating away from rsync, consider dd. You need to configure the folders you want to backup as mountable partitions or disks, but it does byte for byte copies instead of 100kloc fuckery.
BoingBoomTschak 5mo ago
Since I switched my VPS to OpenBSD (base only), I've been "forced" to handle openrsync and it has been mostly a drop-in. Except a problem where I couldn't mix `--exclude glob` and `--delete` on the client, no problem to report. Thanks for the various vibecoding posts, I wasn't aware of the extent of it. Some prior HN discussion: https://news.ycombinator.com/item?id=48334021 https://news.ycombinator.com/item?id=48334021
deleted 5mo ago
[deleted]
mike_hock 5mo ago
> If the source or destination is on a remote server, the client then fork(2)s and starts the server openrsync on the remote host over ssh(1). The client and the server subsequently communicate over socketpair(2) pipes. How is that supposed to work? I guess they mean something like, the forked child forwards the connection to the parent using a socket pair, or just connects its stdout/stdin to the socketpair "pipe" (socket) and execs ssh. But that's like saying you're going to Australia by car when you mean you're driving to the airport.
gorgoiler 5mo ago
All software anneals into a final form that is incontrovertibly correct. Each of these OpenBSD rewrites feels like the realization of such a final form! Code is like a math proof: sketched first on a napkin, then on a blackboard, and then finally typeset in a paper. Each step tidies up the ideas to improve the communication of intent, and the final version should be self evidently stable and/or correct. In the old days you’d see code stability emerge as the “v2” edition of some piece of software. Mozilla to Phoenix to Firebird/fox. Linux 2.2 to 2.4. Python 2.6 to 3.x. The design patterns are carried over but the implementations are revamped for more stable, more legible, and more maintainable code. I don’t mind that vibe coding is the latest form of this phenomenon. We have all been “vibe coding” for decades really. Code like this crap: T = “hello world” def foo_2(): FONT = “Perpetua.ttf” w = text(T, Font) w2 = w.translate((50,0,0)) w3 = w.translate((0,0,20)) show(w3) …gets eventually rewritten, once it works, into: def render(phrase: str, font: Font) -> Shape: return w.text(phrase, font) font = Fonts.load(“Perpetua”) text = render(“hello world”, font).translate((50, 0, 20)) show(text) Before, we’d hack a v0, tidy it up sufficiently for it to be worth of review as v1, then, come back much later and rearrange the innards (in a far more sensible way) as v2. With LLMs — especially in the hands of those who can’t read or won’t read the actual code — we are seeing a lot more version zeroes in the world. Thank you OpenBSD for giving us, albeit surprisingly for rsync, a nice v2.
neves 5mo ago
Why do they need to reimplement a robust piece of difference as rsync?
- Double_a_92 5mo ago
  To make it have a different license which can be used in closed-source projects.

492 points by sph 5mo ago

triggis 5mo ago

[flagged]

ranger_danger 5mo ago
https://social.treehouse.systems/@thesamesam/116662824873341085 https://social.treehouse.systems/@thesamesam/116662824873341...
- akerl_ 5mo ago
  +1 to this. Other than people's reflexive anger or fear about AI coming for their code, I don't see anything to suggest that these are bugs that are due to the inclusion of AI vs bugs in a program with a bunch of complex interop with the filesystem and network.
  - triggis 5mo ago
    In any case, it's important to identify projects that are beginning to actively vibecode and clearly express position on this issue on various platforms so that authors and maintainers receive feedback. Even if this particular bug was not written by LLM in this particular case, it's not a fact that the release does not include other regressions and that subsequent vibecoded versions will not include them & new ones.
    - stsquad 5mo ago
      Do not going harassing developers because you think they are doing it wrong. If you can do better and don't want to actually contribute to the upstream you are always free to fork it.
      - margalabargala 5mo ago
        That's literally what this thread is about.
        akerl_ 5mo ago
        Openrsync is a project that’s existed for a long time and is a great example of people building what they want. This thread has people suggesting it’s good or important to go tell open source devs they’re using bad tools. The former is great. The latter is not.
    - akerl_ 5mo ago
      No. It's not important. It's actually pretty shitty to go around looking for projects and then telling the maintainers you disagree with how they develop.
    - skeledrew 5mo ago
      > it's not a fact that the release does not include other regressions and [...] Are you listening to yourself? The same exact thing also has applied, applies and will continue to apply to manually written code, in perpetuity. There's nothing new under the sun here; regressions happen when there's change, and the only way to mitigate is to have healthy feedback loops.
    - applfanboysbgon 5mo ago
      Friendly reminder that volunteer maintainers owe you literally not a single goddamn thing. I absolutely want no AI slop in my commercial products that I pay money for, but your feedback is not important to people you are not paying to develop software for you. They gave away not only their software but the source code for free; if you have a problem with it, fork it. Which is something you can do with their generous allowance, and that is an allowance any maintainer can instead choose to not bother themselves with if publishing their code for free leads themselves to dealing with entitled internet commenters harassing them with complaints.
      - teddyh 5mo ago
        > Friendly reminder that volunteer maintainers owe you literally not a single goddamn thing. Technically true. However, I also do not owe them my silence.
        deleted 5mo ago
        [deleted]
  - 48terry 5mo ago
    Yeah, nothing to suggest that here. We went from some commits every so often (because rsync was basically finished software) with a handful of issues a month, to "a bajillion Claude commits then a release" with a pile of issues within weeks, at least one of which is about how the goddamn thing doesn't build. I can't possibly see a correlation. Weird how 3.4.2. didn't have a similar deluge of issue reports, though.

jmclnx 5mo ago

I have not checked with OpenBSD 7.9, but as of 7.8 it did not support --exclude or -z. But outside of that openrsync works great. (EDIT: --exclude is now supported on 7.9. Not sure when that was added, nice!) But seems avoiding "slop" is getting very hard. I saw postfix now has a bit of AI code in it. https://mastodon.sdf.org/@mrmasterkeyboard@mastodon.social/116663069350165149 https://mastodon.sdf.org/@mrmasterkeyboard@mastodon.social/1...

nineteen999 5mo ago
Somewhat ironic Postfix has a record of no root/RCE in the default install, where opensmptd hasn't (CVE-2020-7247). Time will tell if it stays that way.
Bender 5mo ago
Exclude is very commonly used in automation jobs to avoid duplicating big git repos and other big files. I think that would be a show stopper for a number of people.
- jmclnx 5mo ago
  I just tried openrsync(1) on OpenBSD 7.9, --exclude now works. I have not tried using exclude in openrsync in a while, but I can see it now works on OpenBSD 7.9!
agwa 5mo ago
Where do you see that about Postfix? I followed the links and the only thing I see is that AI is being used to find bugs, not write code.
- jmclnx 5mo ago
  >Claude assisted code found in external/ibm-public/postfix/dist... That is from the original post in the thread. Is that really due to LLM ? I do not know since I avoid AI as much as I can. But the person also posted this link too: https://github.com/NetBSD/src/commit/f764ddf4062e855f73fe2e31b9c378ddd0155a4e https://github.com/NetBSD/src/commit/f764ddf4062e855f73fe2e3...

skeledrew 5mo ago

This attempt to avoid things that use AI is increasingly looking like some weird kind of reverse whack-a-mole where each targeted hole becomes radioactive after. Just grabbing some popcorn to watch.

ranger_danger 5mo ago
I feel bad for people with the real name Claude.
- formerly_proven 5mo ago
  It took me quite some time to realize what an utterly presumptuous product name Claude Code actually is, but only because Shannon is rarely mentioned with his first name. It's golden calf levels of hubris, even more so if you consider how incapable it was on release. It's like renaming calc.exe Einstein. Incredibly poor taste, but entirely in line with AI tech bro mentality.
  - kstrauser 5mo ago
    That linkage never occurred to me, or, I suspect, them. Claude use to be a reasonably common name. I have an uncle Claude. Why do you believe they named it after Shannon in particular?
    - homebrewer 5mo ago
      It seems to be a widely repeated "fact" which can't be traced to anything particularly authoritative: https://archive.is/pt5fQ https://archive.is/pt5fQ https://britannica.com/topic/Claude-AI https://britannica.com/topic/Claude-AI Looks like the 2023 NYT article started it, and it uses this as reference: > depending on which employee you ask, was either a nerdy tribute to the 20th-century mathematician Claude Shannon Personally I always associated it with the silent protagonist from GTA3. https://gta.fandom.com/wiki/Claude https://gta.fandom.com/wiki/Claude
  - bombcar 5mo ago

WD-42 5mo ago

What's the deal with the name? Openrsync implies to me that it's an open source alternative to a closed source program. But the original Rsync is GPL? Is this just the pushover license making it "more open"?

jtickle 5mo ago
OpenBSD folks would consider the GPL to be less open due to the requirement to apply the GPL to any derivative works.
- ranger_danger 5mo ago
  And GNU folks would say the GPL is actually the more open choice because it forces the project to stay open. Two different ways of thinking about it I guess... it's nice to have choices and I don't think one is more or less "correct", more a matter of opinion/taste I guess.
  - gilrain 5mo ago
    > more open choice because it forces the project A true morality must be based on consent, not coercion. Humanity may not be there yet, and therein lies the argument for force (and thus copyleft); but the ultimate goal should always be to reduce its necessity.
    - kennywinker 5mo ago
      Is this not the paradox of tolerance restated in different terms? BSD license is unrestricted, it tolerates taking open source and closing it, thus always being at risk of things closing down. GPL license doesn’t tolerate taking from open source and closing it, thus ensuring things stay open.
      - KZerda 5mo ago
        The BSD license is why we have Valkey and not a purely closed-source Redis. It would have been much easier to perform the rugpull if Redis had initially been GPLed.

Bender 5mo ago

The actual work of porting is matching the security features provided by OpenBSD's pledge(2) and unveil(2). These are critical elements to the functionality of the system. Without them, your system accepts arbitrary data from the public network. https://justine.lol/pledge/ https://justine.lol/pledge/ I am not seeing pledge on Alpine Linux in edge. Have people been testing Pledge on Linux? Did I perhaps misunderstand the risk of using Openrsync without pledge? Or is this article just for OpenBSD users?

e12e 5mo ago
From above your quote: > The only officially-supported operating system is OpenBSD, as this has considerable security features. And below your quote: > This is possible (I think?) with FreeBSD's Capsicum, but Linux's security facilities are a mess, and will take an expert hand to properly secure. It is portable in the sense that it compiles and runs, not in the sense that it has the same security features. I'd love to see pledge/unveil on (upstream) Linux - but I'm not holding my breath.
- Bender 5mo ago
  Ok that makes more sense, thankyou.
- papercrane 5mo ago
  > I'd love to see pledge/unveil on (upstream) Linux - but I'm not holding my breath There is Landlock now, I believe it would be possible to implement unveil and pledge on top of that.
  - isityettime 5mo ago
    One of HN's favorite hackers has done that: https://justine.lol/pledge/ https://justine.lol/pledge/ There was a discussion here about it a few years ago: https://news.ycombinator.com/item?id=32096801 https://news.ycombinator.com/item?id=32096801
    - 5mo ago

tptacek 5mo ago

rsync has specific running modes for the super-user. It also pumps arbitrary data from the network onto your file-system. openrsync is about 10 000 lines of C code: do you trust me not to make mistakes? No, but that's why almost nobody runs it outside of strict trust boundaries. This security section would make more sense if rsync was like curl, which routinely deals with hostile counterparties. If the other side of your rsync is hostile, you probably have bigger problems! (I'm not an rpki person so I don't know if there's some part of that problem domain that changes this equation. I'm not dunking on the project, just saying this snagged me in the README).

cperciva 5mo ago
No, but that's why almost nobody runs it outside of strict trust boundaries. This security section would make more sense if rsync was like curl, which routinely deals with hostile counterparties. If the other side of your rsync is hostile, you probably have bigger problems! I disagree. While rsync is most often used to transfer data between "friendly" systems, it's inherently crossing a security boundary. It's important to make sure that an attacker can't leverage it to transform the breach of one system into the breach of multiple systems.
- eikenberry 5mo ago
  It is almost universally hooked up using ssh tunneling so ssh takes care of the security boundary and ssh is well trusted.
  - cperciva 5mo ago
    That solves the traffic tampering problem, but not the "malicious peer" problem. You want to be able to sync files without accidentally sharing root privileges.
delusional 5mo ago
> almost nobody runs it outside of strict trust boundaries. I guess you can define "strict" however you want, but from what I saw ~10 years ago, most linux distros handled mirroring with rsync. That's a lot of usage in a pretty core part of the foundational open source ecosystem.

denysvitali 5mo ago

There's also a Go implementation by Michael Stapelberg / the Gokrazy team: https://github.com/gokrazy/rsync https://github.com/gokrazy/rsync

mtlynch 5mo ago
The gokrazy team being Michael Stapelberg : ) https://github.com/gokrazy/rsync/graphs/contributors https://github.com/gokrazy/rsync/graphs/contributors
- denysvitali 5mo ago
  I mean, yes, mostly him (and he gave multiple presentations about this topic, very interesting!). - but there are other contributors too :)
  - dolmen 5mo ago
    stapelberg: 329 commits jonas-fi: 15 commits others: 1 commit each The bus factor is 1.

thefilmore 5mo ago

This is the version used in macOS since 15.0.

mrdomino- 5mo ago
Was it 15.0? I seem to recall it coming in one of the minor point releases in the 15.x line - and I remember it breaking some scripts mysteriously. EDIT: ah, fun: they did include it in 15.0, but they decided to save the breaking change that removed backwards compatibility for 15.4. https://apple.stackexchange.com/a/479297 https://apple.stackexchange.com/a/479297

chasil 5mo ago

There is also a (stub) web page: https://www.openrsync.org/ https://www.openrsync.org/ The problem with this fragmentation of rsync is that Apple and Android will prefer it, but the Linux and greater GPL world will adhere to the original implantation due to inertia. Power users will just have to know the quirks of each version. The only way to stop this is for the original author(s) to release this under a BSD license. Edit: For those assuming equivalent/identical behavior, study these words carefully: "accepts only a subset of rsync's command-line arguments."

qalmakka 5mo ago
Basically like GNU Tar/CPIO and BSD Tar/CPIO. I've largely standardised towards using the bsd variant everywhere (especially since now even Windows ships it and it handles lots of other archive formats using the `tar` command) but it's always a pain to install it everywhere
- yjftsjthsd-h 5mo ago
  Yeah, I'm leaning towards strongly preferring bsdtar since it's happy to work on e.g. zip files:) Does it have any real downsides?
  - SoftTalker 5mo ago
    Limits on the length of path and file names in the archive. Edit: I see that they switched from ustar to pax as the default format in OpenBSD 7.6, so I guess this isn't true any longer.
spauldo 5mo ago
It's really no different than every other BSD utility (and SysV utility, if you're running one of those) being different than the GNU ones. We've coped with it for fifty years at this point.
yjftsjthsd-h 5mo ago
> The only way to stop this is for the original author(s) to release this under a BSD license. Would that stop it? My understanding was that at least OpenBSD tended do redo things for technical reasons, not just licensing

Panino 5mo ago

I've been using openrsync here and there since it was announced and it's definitely improved over time. I'm looking forward to when I can use it exclusively. The one place in my usage where it doesn't match Samba rsync is with the following: openrsync --rsync-path=openrsync -av -e ssh /etc/services example.com:/tmp/services I would expect openrsync to create a remote file /tmp/services, but instead it creates /tmp/services/services. Normal directory mirroring as in -av -e ssh /path/to/src/ example.com:/path/to/dst/ works as it does with Samba rsync.

genxy 5mo ago
Was there already a /tmp/services directory on the dest? One of the biggest points of confusion with rsync is how directories and trailing slashes are handled.
- Panino 5mo ago
  > Was there already a /tmp/services directory on the dest? No. And just to make sure, I ran a quick 'rm -rf /tmp/services' on the remote host, then re-ran openrsync on the client. Same result. This is OpenBSD 7.9 on both sides. And I 100% agree about trailing slashes.
- anyfoo 5mo ago
  I hear that a lot, but I familiarized myself with it once and ever since it makes a lot of sense to me. Source ending in “/“: You want what’s inside. Source not ending in “/“: You want the thing (i.e. directory itself). For the destination, it does not matter whether it ends in “/“ or not, but for consistency I like adding a “/“ anyway (I want to put thing inside the directory).
  - linsomniac 5mo ago
    Yes, the nice thing about dest having a trailing "/" is that if it exists and is NOT a directory, you are alerted right away.
- eichin 5mo ago

salvesefu 5mo ago

For those needing context for the development of this package; this project is presently being developed as part of a RPKI validator. https://medium.com/@jobsnijders/a-proposal-for-a-new-rpki-validator-openbsd-rpki-client-1-15b74e7a3f65 https://medium.com/@jobsnijders/a-proposal-for-a-new-rpki-va...

SubiculumCode 5mo ago

I'm going to ask a question. I could ask chatgpt. I could Google it. I am asking a question because it is human to do so. Ubuntu's packaged rsync, is it Samba rsync? Why reimplement it?

hideout_berlin 5mo ago
samba is different topic
eichin 5mo ago
it's tridge rsync; samba is another project by the same guy. (rsync was originally a PhD thesis...)
- bombcar 5mo ago
  tridge is mighty in the Linux world, he arguably greatly contributed to Linux as a server (via Samba to save on windows NT client user licenses), rsync, and IIRC reverse-engineering the bitlocker protocol thereby starting the gitstorm.
tredre3 5mo ago
Ubuntu's rsync is samba rsync. It's not part of the samba project per se, but it is made by the same guy and the official url is https://rsync.samba.org/ https://rsync.samba.org/ so it's entirely fair to call it samba rsync in my opinion.

onedognight 5mo ago

They don’t support any recent rsync protocol, so there’s no 64bit timestamp support, so you can never actually sync metadata across newer filesystems.

throwaway27448 5mo ago

I'm confused. Isn't rsync already free software? What are we doing here. Why are we trying to cuck ourselves for capital. I like open bsd but this just seems like burning cash

somat 5mo ago
My understanding is that much of the point of openrsync is to create a second implementation of a protocol so the standards bodies don't balk at including it in their standards. Or to put it more concretely, people working on the rpki standard(who happened to also be openbsd devs) wanted to use rsync to transfer bulk data. The standards body was hesitant, while rsync is ostensibly a documented protocol, there was only one implementation. So in true openbsd fashion they rolled up their sleeves and wrote that second implementation. On use, there is nothing wrong with openrsync, however it may never hit feature parity with rsync, that is not a goal of the project, they want a specific subset of rsync features to support their rpki needs. If anyone else finds this useful that is great. So I suspect users will be those who want a bsd licensed rsync(apple) or them who are willing to give up features for openbsd quality code(myself).
- throwaway27448 5mo ago
  Why would we explicitly develop code legally available to capital?
  - reverius42 5mo ago
    Are you new to the concept of open source? https://en.wikipedia.org/wiki/Open_source https://en.wikipedia.org/wiki/Open_source There's some discussion here that might help answer your question: https://en.wikipedia.org/wiki/Open_source#Economics https://en.wikipedia.org/wiki/Open_source#Economics
  - avadodin 5mo ago
    Most people using open licenses over right–restricting GPL–style licenses do so for philosophical reasons. They don't necessarily think they are better for getting reciprocal contributions from evil corporations even though many do argue that today, especially after GPLv3. See the redis licensing debacle and why and how it was resolved. If I understand it correctly, the FSF could technically sell all of GNU tools to Microsoft which could in turn keep them closed source including all contributions over the past 40 years. Not without losing all of their remaining credibility, but still.

trollbridge 5mo ago

Given the sudden spike in vibe-coded commits to the rsync codebase, and regressions that’s introduced, this is very good news.

port11 5mo ago
I was prepared to dismiss your comment because rsync has always been rock solid, but indeed upgrading broke my backup script. The latest issue on GitHub documents plenty of bugs introduced in the last 2 patches, including a monstrous ~9k LOC commit that was probably pointless. LLMs make writing code faster/easier, but the thinking was always the important bit. I’ve no idea why you’d muck up such a long-standing, reliable piece of software.
- prmoustache 5mo ago
  When LLM allows you to produce code n% faster, it also allows you to introduce bugs n% faster. I find it quite strange that people do not seem to be aware of that...I think many started worshipping the tool as if it was some kind of divinity and lost all objctivity. This doesn't bode well for the future if people aren't able to review code anymore.
  - brianwawok 5mo ago
    Don’t need people to review code, need more LLMs. People aren’t that magical.
    - port11 5mo ago
      Maybe one day. Right now I can have Opus carefully draft a plan, Sonnet execute it, and have Opus review the code. It will complain about things that it had dismissed in the plan. Example: it recently flagged ID derivation from a signing key. Opus itself suggested it! When reminded of that, it did this new 4.8-style “my bad, and here’s why I was right in the first place.” Humans are still badass.
asdsg 5mo ago

LAC-Tech 5mo ago

A few comments here suggest rsync is undergoing some "churn" which tbh is highly undesirable for a command line utility. Might switch over. As an aside I really love the stuff openBSD puts out. If they ever succeed in making a modern journalling filesystem I will probably switch over.

TZubiri 5mo ago

Might be offtopic, but i think when looking for alternatives the options should be broad, questioning the category of the app, not just clones and forks. If you are considering migrating away from github, don't just consider gitlab and gitea, consider just git, and so on.. If you are considering migrating away from rsync, consider dd. You need to configure the folders you want to backup as mountable partitions or disks, but it does byte for byte copies instead of 100kloc fuckery.

BoingBoomTschak 5mo ago

Since I switched my VPS to OpenBSD (base only), I've been "forced" to handle openrsync and it has been mostly a drop-in. Except a problem where I couldn't mix `--exclude glob` and `--delete` on the client, no problem to report. Thanks for the various vibecoding posts, I wasn't aware of the extent of it. Some prior HN discussion: https://news.ycombinator.com/item?id=48334021 https://news.ycombinator.com/item?id=48334021

deleted 5mo ago

[deleted]

mike_hock 5mo ago

> If the source or destination is on a remote server, the client then fork(2)s and starts the server openrsync on the remote host over ssh(1). The client and the server subsequently communicate over socketpair(2) pipes. How is that supposed to work? I guess they mean something like, the forked child forwards the connection to the parent using a socket pair, or just connects its stdout/stdin to the socketpair "pipe" (socket) and execs ssh. But that's like saying you're going to Australia by car when you mean you're driving to the airport.

gorgoiler 5mo ago

All software anneals into a final form that is incontrovertibly correct. Each of these OpenBSD rewrites feels like the realization of such a final form! Code is like a math proof: sketched first on a napkin, then on a blackboard, and then finally typeset in a paper. Each step tidies up the ideas to improve the communication of intent, and the final version should be self evidently stable and/or correct. In the old days you’d see code stability emerge as the “v2” edition of some piece of software. Mozilla to Phoenix to Firebird/fox. Linux 2.2 to 2.4. Python 2.6 to 3.x. The design patterns are carried over but the implementations are revamped for more stable, more legible, and more maintainable code. I don’t mind that vibe coding is the latest form of this phenomenon. We have all been “vibe coding” for decades really. Code like this crap: T = “hello world” def foo_2(): FONT = “Perpetua.ttf” w = text(T, Font) w2 = w.translate((50,0,0)) w3 = w.translate((0,0,20)) show(w3) …gets eventually rewritten, once it works, into: def render(phrase: str, font: Font) -> Shape: return w.text(phrase, font) font = Fonts.load(“Perpetua”) text = render(“hello world”, font).translate((50, 0, 20)) show(text) Before, we’d hack a v0, tidy it up sufficiently for it to be worth of review as v1, then, come back much later and rearrange the innards (in a far more sensible way) as v2. With LLMs — especially in the hands of those who can’t read or won’t read the actual code — we are seeing a lot more version zeroes in the world. Thank you OpenBSD for giving us, albeit surprisingly for rsync, a nice v2.

neves 5mo ago

Why do they need to reimplement a robust piece of difference as rsync?

Double_a_92 5mo ago
To make it have a different license which can be used in closed-source projects.