7 ms·

In straightforward terms, author of the most popular YAML parser (https://lib.rs/crates/serde_yaml https://lib.rs/crates/serde_yaml) has suddenly given up on it

by pornel 3y ago

In straightforward terms, author of the most popular YAML parser (https://lib.rs/crates/serde_yaml https://lib.rs/crates/serde_yaml) has suddenly given up on it, and marked it as deprecated and unmaintained, without warning or designating another maintainer. It's not exactly a left-pad situation, because the package remains fully functional (and crates.io doesn't allow deletions), but the package is used in 4000 other crates. Audit and auto-update tools will be complaining about use of an unmaintained crate.

xeromal 3y ago
Sometimes you just say fuck it.
- zozbot234 3y ago
  > Sometimes you just say fork it. FTFY. And vendoring a dependency is really just a hidden fork.
  - pengaru 3y ago
    > FTFY. And vendoring a dependency is really just a hidden fork. It's not a fork until you start diverging from upstream with local modifications.
  - mwcampbell 3y ago
    That raises the question: what is the current etiquette around deciding whether to pester a dependency's author to include your fixes or just fork it, especially if it looks like you will be the biggest user of that dependency? I'm dealing with that question for a dependency that I want to introduce in one of my crates, but ideally not until at least one of my own fixes is merged.
    - foverzar 3y ago
      Obviously fork it, implement your feature, battle-test it, fix and make it more generic, battle-test it again, then ask the maintainer if they would be interested in a pull request. I mean, implementing your fixes in the first place means you are already inherently forking it.
    - cocoa19 3y ago
      My thought process is: 1. You find a bug, open an issue. 2. If bug is a minor issue, wait. If I want a fix really bad, offer to fix it. 3. If I still want a fix really bad and author did not respond, or does not want a fix, then fork the project for my own consumption. Using alternative dependencies are also an option.
      - hughesjj 3y ago
        Yup, and forks don't have to be permanent. Imo be ready to fork all the things if need be. Get comfortable with it before you're forced to.
    - RussianCow 3y ago
      In my opinion, you should always ask the author first. Having your changes accessible from the primary repository is much easier and better for everyone, and there is a good chance the author is happy to accept your PRs. On the other hand, they might not agree with your proposal or might want to make significant changes to the design, at which point it becomes a personal tradeoff for you.
- jcgrillo 3y ago
  This is exactly what we all do every time we cargo update without reviewing the code changes in each dep (and their deps) wrt how they impact our compiled artifact. Ultimately we're each individually responsible for what we ship, and there's no amount of bureaucracy that will absolve us of the responsibility.
  - bee_rider 3y ago
    I think it is possible to build infrastructure that will allow entities to spread responsibility. They do it in other fields I think? For example I can’t imagine Ford checks literally every single individual component that gets put in their cars, right? I assume they have contracts and standards with manufacturers that make the parts, probably including random spot checks, but at some point to handle complexity it is necessary to trust/offload responsibility on some suppliers. I bet you could get those sort of guarantees from, like, Intel or IBM if people generally were willing to pay them enough (can you now? Not sure, probably depends on the specific package). But nobody would make that sort of promise for free. The transparency of the open source ecosystem is an alternative to that sort of thing, no promises but you can check everything. Just to be explicit (since I’m sure this looks like needless pedantry), I think it is important to remember that the open source and free software ecosystems are an active rejection of that sort of liability management framework, because that’s the only way to get contributions from a community of volunteers.
    - jcgrillo 3y ago
      There's something very different with software I think. For example, if a mechanism calls for a class 10.9 bolt with a certain diameter and thread pitch (let's say M6x1 for the sake of argument) then there are many different manufacturers who can supply equally suitable parts. They may each be dimensionally different, have different yield strengths, but they're all conforming to class 10.9 and dimensionally within tolerance. As the designer of the mechanical assembly I can trust the manufacturer of the bolts has a quality control procedure that'll make sure their bolts are good. As a software assembly designer, I don't really have any way to say "this subcomponent remains intact" when I integrate it into my system. The compiler will chew things up and rearrange them at link time. This is a good thing! It makes the resulting binary go brr real good. But it's ultimately my responsibility to make sure the thing I ship--the compiled artifact--is correct. It's not really a valid move to blame one of my dependencies. I was supposed to review those dependencies and make sure they work in the context of the thing I shipped. Wouldn't it be a bit like the mechanical assembly designer blaming the bolt manufacturer for a failure of a bolt they modified while building the assembly (EDIT: or blaming the supplier when the assembler used the bolt in a way it wasn't designed for)? What guarantees can the bolt manufacturer reasonably give in that case? EDIT: another thing is that source code isn't really analogous to a mechanical component. It's more like the design specification or blueprint for... something that converts electrical potential into heat and also does other stuff.
deleted 3y ago
[deleted]
bee_rider 3y ago
Thank you for describing the situation without the analogy.
- vsl 3y ago
  It's incorrect description, though.
  - bee_rider 3y ago
    Unfortunately the finance analogy is sufficiently outside my wheelhouse as to be indecipherable to me. (Analogies are helpful to add intuition but they shouldn’t replace the description of the actual thing). Maybe this description is wrong but I can’t tell.
    - WorldMaker 3y ago
      The problem is not just the analogy but the names were changed to protect the guilty parties adding further obfuscation. Though the links are correct and you can follow the bouncing ball a little easier replacing the fake names in the article with the real names from the links: stuff -> insta learned-rust-this-way -> yaml-rust to default/defaulted -> serde-yaml As I read it: insta was depending on yaml-rust. RUSTSEC issued an advisory against yaml-rust. Some shade is thrown on yaml-rust for being unmaintained and basically a student project of someone learning the language for the first time, adding to the irony/brittleness of its unmaintained (though still working) state and now the RUSTSEC callout. In insta trying to switch to some alternative to yaml-rust the next most common library serde-yaml preemptively announced its unmaintained status. So insta just vendored yaml-rust and called it a day, which currently works to remove the RUSTSEC complaints but doesn't fix the long term maintenance problem.
tayo42 3y ago
Isn't that author kind of prolific within the rust library world? He didn't do that to any other libraries. Is there something with more info?
- steveklabnik 3y ago
  Yes. The release notes for the last release explain: https://github.com/dtolnay/serde-yaml/releases/tag/0.9.34 https://github.com/dtolnay/serde-yaml/releases/tag/0.9.34
  - empath-nirvana 3y ago
    That is incredibly obnoxious and irresponsible and he should turn over the repo to someone else to manage.
    - ahahahahah 3y ago
      No, it's incredibly irresponsible to hand over the repo to someone else to manage, even someone who is an active contributor. See https://boehs.org/node/everything-i-know-about-the-xz-backdoor https://boehs.org/node/everything-i-know-about-the-xz-backdo... for example.
    - steveklabnik 3y ago
      I’m not sure how much open source work you’ve done or what packages you’ve maintained, but in my experience, the “someone else” rarely ever comes. Heck I have even been the someone else and then done a bad job of it! This is an unrealistic and frankly a bit entitled of a criticism. This is before we even get to stuff like “it’s not like yaml is a fast moving target, what continued maintenance is really even needed” parts of the equation, or the comment above that said he’s tried already.
      - curun1r 3y ago
        The “someone else” also needs to be vetted to ensure that their first update won’t include crypto mining malware. We should remember that an unmaintained dependency isn’t the worst thing that can happen to a supply chain. There are far worse nightmare scenarios that involve exfiltrated keys, ransomewared files and such. I’ll bet that if someone with a track record of contributing to the Rust community steps up, he’ll happily transfer control of the crate. But he’s not just going to assign it to some random internet user and put the whole ecosystem at risk.
devsda 3y ago
> Audit and auto-update tools will be complaining about use of an unmaintained crate. Not the case here but sometimes a library (with no dependencies) is really feature complete with no real need for changes other than security fixes. I wish audit tools and corporate policies are smart enough to recognize this instead of nagging about "unmaintained" versions of projects with no recent commits.
- ben_jones 3y ago
  > I wish audit tools and corporate policies are smart The ball is in the court of the enterprise purchaser mandating the auditing company and they don’t give a ** if your dev team has to hot swap a yaml parsing library
  - jimbokun 3y ago
    They should. They are paying devs a ton of money to futz around with code that works perfectly well and is just as secure today as it was yesterday. So you have security theater and TPS reports consuming the time of some of your most highly compensated human resources.
    - anotherhue 3y ago
      > just as secure today as it was yesterday. Agreed but the gotcha is when a new issue comes out and there is no security fix available. It would be enough to say 'we are comfortable patching this if a CVE is found', but at that point they might as well start to maintain it.
- appplication 3y ago
  Been doing this software thing for a bit now… while what you say is true in that it is possible in some rare cases, I think it’s a bit of a straw man exception. Code is, in the general sense, unfortunately never complete, regardless of how complete it seems in its context. The issue is that the context is always changing. It is almost like suggesting there is an evolutionarily perfect organism. At best, one suited perfectly well to the current context can exist, but that perfect suitability falls off the second the environment changes. And the environment always changes. I totally get the sentiment, but I think the idea that any code is ever complete is unfortunately just not a realistic one to apply outside of select and narrow contexts.
nindalf 3y ago
This isn't accurate. OP's article says they depend on yaml-rust (https://github.com/chyh1990/yaml-rust https://github.com/chyh1990/yaml-rust), which is now unmaintained. Simultaneously, serge-yaml was marked as unmaintained to prevent people from migrating there en masse.
- pornel 3y ago
  Sorry, indeed it's been about the other library. However, these issues are connected: dtolnay is a co-owner of yaml-rust too, so either way without him YAML in Rust has a https://xkcd.com/2347 https://xkcd.com/2347 problem.
  - masklinn 3y ago
    > so either way without him YAML in Rust has a https://xkcd.com/2347 https://xkcd.com/2347 You seem to be misunderstanding the comic you link. With him that was the situation. On crates.io, serde_yaml has 56m lifetime downloads and 8.5m recent downloads, and from his own announcement David hasn't used yaml in any of his project in a while, yet has kept plugging at it until now. And yaml-rust has 53.5m lifetime downloads and 5m recent downloads, and has been unmaintained for 4 years.
chipdart 3y ago
> (...) without warning or designating another maintainer. Why would he?

by pornel 3y ago

xeromal 3y ago

Sometimes you just say fuck it.

zozbot234 3y ago
> Sometimes you just say fork it. FTFY. And vendoring a dependency is really just a hidden fork.
- pengaru 3y ago
  > FTFY. And vendoring a dependency is really just a hidden fork. It's not a fork until you start diverging from upstream with local modifications.
- mwcampbell 3y ago
  That raises the question: what is the current etiquette around deciding whether to pester a dependency's author to include your fixes or just fork it, especially if it looks like you will be the biggest user of that dependency? I'm dealing with that question for a dependency that I want to introduce in one of my crates, but ideally not until at least one of my own fixes is merged.
  - foverzar 3y ago
    Obviously fork it, implement your feature, battle-test it, fix and make it more generic, battle-test it again, then ask the maintainer if they would be interested in a pull request. I mean, implementing your fixes in the first place means you are already inherently forking it.
  - cocoa19 3y ago
    My thought process is: 1. You find a bug, open an issue. 2. If bug is a minor issue, wait. If I want a fix really bad, offer to fix it. 3. If I still want a fix really bad and author did not respond, or does not want a fix, then fork the project for my own consumption. Using alternative dependencies are also an option.
    - hughesjj 3y ago
      Yup, and forks don't have to be permanent. Imo be ready to fork all the things if need be. Get comfortable with it before you're forced to.
  - RussianCow 3y ago
    In my opinion, you should always ask the author first. Having your changes accessible from the primary repository is much easier and better for everyone, and there is a good chance the author is happy to accept your PRs. On the other hand, they might not agree with your proposal or might want to make significant changes to the design, at which point it becomes a personal tradeoff for you.
jcgrillo 3y ago
This is exactly what we all do every time we cargo update without reviewing the code changes in each dep (and their deps) wrt how they impact our compiled artifact. Ultimately we're each individually responsible for what we ship, and there's no amount of bureaucracy that will absolve us of the responsibility.
- bee_rider 3y ago
  I think it is possible to build infrastructure that will allow entities to spread responsibility. They do it in other fields I think? For example I can’t imagine Ford checks literally every single individual component that gets put in their cars, right? I assume they have contracts and standards with manufacturers that make the parts, probably including random spot checks, but at some point to handle complexity it is necessary to trust/offload responsibility on some suppliers. I bet you could get those sort of guarantees from, like, Intel or IBM if people generally were willing to pay them enough (can you now? Not sure, probably depends on the specific package). But nobody would make that sort of promise for free. The transparency of the open source ecosystem is an alternative to that sort of thing, no promises but you can check everything. Just to be explicit (since I’m sure this looks like needless pedantry), I think it is important to remember that the open source and free software ecosystems are an active rejection of that sort of liability management framework, because that’s the only way to get contributions from a community of volunteers.
  - jcgrillo 3y ago
    There's something very different with software I think. For example, if a mechanism calls for a class 10.9 bolt with a certain diameter and thread pitch (let's say M6x1 for the sake of argument) then there are many different manufacturers who can supply equally suitable parts. They may each be dimensionally different, have different yield strengths, but they're all conforming to class 10.9 and dimensionally within tolerance. As the designer of the mechanical assembly I can trust the manufacturer of the bolts has a quality control procedure that'll make sure their bolts are good. As a software assembly designer, I don't really have any way to say "this subcomponent remains intact" when I integrate it into my system. The compiler will chew things up and rearrange them at link time. This is a good thing! It makes the resulting binary go brr real good. But it's ultimately my responsibility to make sure the thing I ship--the compiled artifact--is correct. It's not really a valid move to blame one of my dependencies. I was supposed to review those dependencies and make sure they work in the context of the thing I shipped. Wouldn't it be a bit like the mechanical assembly designer blaming the bolt manufacturer for a failure of a bolt they modified while building the assembly (EDIT: or blaming the supplier when the assembler used the bolt in a way it wasn't designed for)? What guarantees can the bolt manufacturer reasonably give in that case? EDIT: another thing is that source code isn't really analogous to a mechanical component. It's more like the design specification or blueprint for... something that converts electrical potential into heat and also does other stuff.

deleted 3y ago

[deleted]

bee_rider 3y ago

Thank you for describing the situation without the analogy.

vsl 3y ago
It's incorrect description, though.
- bee_rider 3y ago
  Unfortunately the finance analogy is sufficiently outside my wheelhouse as to be indecipherable to me. (Analogies are helpful to add intuition but they shouldn’t replace the description of the actual thing). Maybe this description is wrong but I can’t tell.
  - WorldMaker 3y ago
    The problem is not just the analogy but the names were changed to protect the guilty parties adding further obfuscation. Though the links are correct and you can follow the bouncing ball a little easier replacing the fake names in the article with the real names from the links: stuff -> insta learned-rust-this-way -> yaml-rust to default/defaulted -> serde-yaml As I read it: insta was depending on yaml-rust. RUSTSEC issued an advisory against yaml-rust. Some shade is thrown on yaml-rust for being unmaintained and basically a student project of someone learning the language for the first time, adding to the irony/brittleness of its unmaintained (though still working) state and now the RUSTSEC callout. In insta trying to switch to some alternative to yaml-rust the next most common library serde-yaml preemptively announced its unmaintained status. So insta just vendored yaml-rust and called it a day, which currently works to remove the RUSTSEC complaints but doesn't fix the long term maintenance problem.

tayo42 3y ago

Isn't that author kind of prolific within the rust library world? He didn't do that to any other libraries. Is there something with more info?

steveklabnik 3y ago
Yes. The release notes for the last release explain: https://github.com/dtolnay/serde-yaml/releases/tag/0.9.34 https://github.com/dtolnay/serde-yaml/releases/tag/0.9.34
- empath-nirvana 3y ago
  That is incredibly obnoxious and irresponsible and he should turn over the repo to someone else to manage.
  - ahahahahah 3y ago
    No, it's incredibly irresponsible to hand over the repo to someone else to manage, even someone who is an active contributor. See https://boehs.org/node/everything-i-know-about-the-xz-backdoor https://boehs.org/node/everything-i-know-about-the-xz-backdo... for example.
  - steveklabnik 3y ago
    I’m not sure how much open source work you’ve done or what packages you’ve maintained, but in my experience, the “someone else” rarely ever comes. Heck I have even been the someone else and then done a bad job of it! This is an unrealistic and frankly a bit entitled of a criticism. This is before we even get to stuff like “it’s not like yaml is a fast moving target, what continued maintenance is really even needed” parts of the equation, or the comment above that said he’s tried already.
    - curun1r 3y ago
      The “someone else” also needs to be vetted to ensure that their first update won’t include crypto mining malware. We should remember that an unmaintained dependency isn’t the worst thing that can happen to a supply chain. There are far worse nightmare scenarios that involve exfiltrated keys, ransomewared files and such. I’ll bet that if someone with a track record of contributing to the Rust community steps up, he’ll happily transfer control of the crate. But he’s not just going to assign it to some random internet user and put the whole ecosystem at risk.

devsda 3y ago

> Audit and auto-update tools will be complaining about use of an unmaintained crate. Not the case here but sometimes a library (with no dependencies) is really feature complete with no real need for changes other than security fixes. I wish audit tools and corporate policies are smart enough to recognize this instead of nagging about "unmaintained" versions of projects with no recent commits.

ben_jones 3y ago
> I wish audit tools and corporate policies are smart The ball is in the court of the enterprise purchaser mandating the auditing company and they don’t give a ** if your dev team has to hot swap a yaml parsing library
- jimbokun 3y ago
  They should. They are paying devs a ton of money to futz around with code that works perfectly well and is just as secure today as it was yesterday. So you have security theater and TPS reports consuming the time of some of your most highly compensated human resources.
  - anotherhue 3y ago
    > just as secure today as it was yesterday. Agreed but the gotcha is when a new issue comes out and there is no security fix available. It would be enough to say 'we are comfortable patching this if a CVE is found', but at that point they might as well start to maintain it.
appplication 3y ago
Been doing this software thing for a bit now… while what you say is true in that it is possible in some rare cases, I think it’s a bit of a straw man exception. Code is, in the general sense, unfortunately never complete, regardless of how complete it seems in its context. The issue is that the context is always changing. It is almost like suggesting there is an evolutionarily perfect organism. At best, one suited perfectly well to the current context can exist, but that perfect suitability falls off the second the environment changes. And the environment always changes. I totally get the sentiment, but I think the idea that any code is ever complete is unfortunately just not a realistic one to apply outside of select and narrow contexts.

nindalf 3y ago

This isn't accurate. OP's article says they depend on yaml-rust (https://github.com/chyh1990/yaml-rust https://github.com/chyh1990/yaml-rust), which is now unmaintained. Simultaneously, serge-yaml was marked as unmaintained to prevent people from migrating there en masse.

pornel 3y ago
Sorry, indeed it's been about the other library. However, these issues are connected: dtolnay is a co-owner of yaml-rust too, so either way without him YAML in Rust has a https://xkcd.com/2347 https://xkcd.com/2347 problem.
- masklinn 3y ago
  > so either way without him YAML in Rust has a https://xkcd.com/2347 https://xkcd.com/2347 You seem to be misunderstanding the comic you link. With him that was the situation. On crates.io, serde_yaml has 56m lifetime downloads and 8.5m recent downloads, and from his own announcement David hasn't used yaml in any of his project in a while, yet has kept plugging at it until now. And yaml-rust has 53.5m lifetime downloads and 5m recent downloads, and has been unmaintained for 4 years.

chipdart 3y ago

> (...) without warning or designating another maintainer. Why would he?