3 ms·

Could create-react-app have avoided this through regression suites?

by nemetroid 7y ago

0xff00ffee 7y ago
Bumping your comment because I would like to know. I'm following the github thread.
- jakear 7y ago
  Potentially. If cra had pinned all their deps, and used a bot to automatically bump deps contingent on passing a comprehensive regression matrix, this would have been avoided. GitHub's Dependabot is good for this. In my opinion everybody besides libraries should pin deps and use dependabot.
  - gombosg 7y ago
    Exactly. We use Renovatebot for the same purpose. It pins dependencies and creates PRs for updates. Amazing to see how often the builds break, even sometimes after minor updates. But at least we fix them before release, and not after... :)
    - jakear 7y ago
      Yep. One of the very nice things about npm/node versus python or go or some others is that package locks and dependency pinning is possible. But few people seem to use it. I’ve seen reports of people using a go library that gets a minor update and breaks their app, at which point they become SOL as go always installs the lad test version. I myself have been working in python projects where the dockerfile simply says “pip install blah” and I get different deps than the working version. No clue why anyone would be okay with working like that.
      - jen20 7y ago
        It's not true that Go always installs the latest version of a dependency. `go get github.com/x/y@v1.3.4` installs v1.3.4 of x/y, assuming there is a tag matching that.
        jakear 7y ago
        I’m not familiar with go, would this persist to other people attempting to install the package? The issue I’ve seen is: https://github.com/go-yaml/yaml/issues/558 https://github.com/go-yaml/yaml/issues/558 > Please do follow semver as it's a nightmare for us to manage particularly using go module (you can't stick to a particular version). And of course everybody’s idea of a breaking change is different, so this idea that you can’t install a particular version seems unworkable.
- detaro 7y ago
  If they'd pinned the dependency versions, and ran the tests before updating the pins, the tests should have catched it.
hn_throwaway_99 7y ago
Not really. NPM relies heavily on semver - https://semver.org/ https://semver.org/. In this case, the package that was updated updated a minor version, which means it should be backwards compatible, but it wasn't for later versions of Node. Of course, you can always lock your build to exact versions of your dependencies (lock files in NPM used to be a complete cluster, in my opinion they are less of a cluster now - you can pretty much do everything you want with them but there are some gotchas that make it easy to shoot yourself in the foot). The issue is that when you run 'npm install', it will pull the latest semver-compatible versions of your dependencies. So for everyone decrying how this is a bad example of NPM and the javascript ecosystem, I really think the opposite is true. Yes, it broke a lot of upstream dependencies, but importantly only for new builds of those items, and furthermore it was found almost immediately. Also, of course, you can specify exact versions of your dependencies - you don't have to rely on semver. That means, though, that you need to be more vigilant about pulling in bug fixed and security fixes, and most people take the tradeoff that they are comfortable pulling in patch or minor versions, but using lock files once they have a build they have verified.
salawat 7y ago
The regression suite never gets to run if it shares the dependency. And the system under test shouldn't even compile for the tests to run either. So it isn't so much the regression suite saving you so much as it is just acting as the client of first resort.
- jakear 7y ago
  CRA would be running the tests, not is-promise. CRA could have pinned every dep, and had a bot (dependabot) automatically run tests against every new version of every depended-upon package, and update only when those tests pass.

Could create-react-app have avoided this through regression suites?

0xff00ffee 7y ago

Bumping your comment because I would like to know. I'm following the github thread.

jakear 7y ago
Potentially. If cra had pinned all their deps, and used a bot to automatically bump deps contingent on passing a comprehensive regression matrix, this would have been avoided. GitHub's Dependabot is good for this. In my opinion everybody besides libraries should pin deps and use dependabot.
- gombosg 7y ago
  Exactly. We use Renovatebot for the same purpose. It pins dependencies and creates PRs for updates. Amazing to see how often the builds break, even sometimes after minor updates. But at least we fix them before release, and not after... :)
  - jakear 7y ago
    Yep. One of the very nice things about npm/node versus python or go or some others is that package locks and dependency pinning is possible. But few people seem to use it. I’ve seen reports of people using a go library that gets a minor update and breaks their app, at which point they become SOL as go always installs the lad test version. I myself have been working in python projects where the dockerfile simply says “pip install blah” and I get different deps than the working version. No clue why anyone would be okay with working like that.
    - jen20 7y ago
      It's not true that Go always installs the latest version of a dependency. `go get github.com/x/y@v1.3.4` installs v1.3.4 of x/y, assuming there is a tag matching that.
      - jakear 7y ago
        I’m not familiar with go, would this persist to other people attempting to install the package? The issue I’ve seen is: https://github.com/go-yaml/yaml/issues/558 https://github.com/go-yaml/yaml/issues/558 > Please do follow semver as it's a nightmare for us to manage particularly using go module (you can't stick to a particular version). And of course everybody’s idea of a breaking change is different, so this idea that you can’t install a particular version seems unworkable.
detaro 7y ago
If they'd pinned the dependency versions, and ran the tests before updating the pins, the tests should have catched it.

hn_throwaway_99 7y ago

Not really. NPM relies heavily on semver - https://semver.org/ https://semver.org/. In this case, the package that was updated updated a minor version, which means it should be backwards compatible, but it wasn't for later versions of Node. Of course, you can always lock your build to exact versions of your dependencies (lock files in NPM used to be a complete cluster, in my opinion they are less of a cluster now - you can pretty much do everything you want with them but there are some gotchas that make it easy to shoot yourself in the foot). The issue is that when you run 'npm install', it will pull the latest semver-compatible versions of your dependencies. So for everyone decrying how this is a bad example of NPM and the javascript ecosystem, I really think the opposite is true. Yes, it broke a lot of upstream dependencies, but importantly only for new builds of those items, and furthermore it was found almost immediately. Also, of course, you can specify exact versions of your dependencies - you don't have to rely on semver. That means, though, that you need to be more vigilant about pulling in bug fixed and security fixes, and most people take the tradeoff that they are comfortable pulling in patch or minor versions, but using lock files once they have a build they have verified.

salawat 7y ago

The regression suite never gets to run if it shares the dependency. And the system under test shouldn't even compile for the tests to run either. So it isn't so much the regression suite saving you so much as it is just acting as the client of first resort.

jakear 7y ago
CRA would be running the tests, not is-promise. CRA could have pinned every dep, and had a bot (dependabot) automatically run tests against every new version of every depended-upon package, and update only when those tests pass.