3 ms·

The licensing of a dependency could be easily determined programmatically (GitHub already built a decent scanner). However, I think that the quality of a depend

by 256dpi 7y ago

The licensing of a dependency could be easily determined programmatically (GitHub already built a decent scanner). However, I think that the quality of a dependency is most important and that requires a manual vetting process. A trivial solution would be to create a crowd-sourced dependency vetting platform.

kijin 7y ago
GitHub's license detection algorithm is crap. It tends to get GPLv2 right, but most other licenses are hit-and-miss. And I still can't find a setting where I can manually specify the license when GitHub can't autodetect one or gets it wrong.
- rurounijones 7y ago
  Ruby gems have the license as an explicit field in the gemspec (manifest file). Is this not the case elsewhere? It makes license scanning a doddle.
  - tsteenbe 7y ago
    > It makes license scanning a doddle. As someone working in the field of license compliance (leading an Open Source Program Office) and dealing with the licensing of tens of thousands of OSS dependencies on a daily basis for a large company I can tell you that license scanning / compliance is anything but a doddle. I do a lot of public speaking on this topic for a summary of the issues I recommend you to have a look at https://static.sched.com/hosted_files/ocs19/c7/OSS-Review-Toolkit-Using-FOSS-tools-for-FOSS-reviews-in-CI-CD-world.pdf https://static.sched.com/hosted_files/ocs19/c7/OSS-Review-To... Most modern package managers do offer project maintainers a way to declare the license for a project. However I can tell you that often the declared license does not match the licenses detected in the source code. What counts is the license stated in the source code files not what it's in the gemspec, package.json, pom.xml etc. This is quite common issue especially for older or larger OSS projects where various contributors have added new code over time that may be licensed under an OSS license that is compatible but not the same as the main license of the projects (Think adding BSD-2-Clause in Apache-2.0 project) What happens is that this contributions get accepted but the project maintainers do not to update the declared license. Not saying declaring a licensing in a .gemspec file is useless but just recommend you to take it as an indicator of the main license of the project - the project may include source code that licensed under a different license. Lack of clarity around licenses and security vulnerabilities is a big issue within the OSS community especially as lack of clarity reduces engagement — that means fewer users, fewer contributors and a smaller community. Several organizations are working on a open source solution for this community problems see for further details https://clearlydefined.io/about https://clearlydefined.io/about Full disclosure I am one of the maintainer of OSS Review Toolkit and contributor to ClearlyDefined.io
  - saagarjha 7y ago
    An even older tradition is a LICENSE file in the root of your project, which GitHub seems to read, but it's not great at matching up the license text to actual licenses.
    - kijin 7y ago
      An alternative tradition uses a COPYING file. I'm not sure which is older, but most GNU projects use COPYING instead of LICENSE. Meanwhile, some projects use COPYRIGHT or other variation, BSD-style licenses are often added directly to source files, and properly applying LGPLv3 to your project involves adding a separate COPYING.LESSER file. It's a mess.
  - ratww 7y ago
    I think most modern package managers have something like that: NPM has a license field in package.json, pypi in pkg-info, nuget in the nuspec file, etc. Github uses the LICENSE file to detect licenses by default IIRC.
  - jsjohnst 7y ago
    If you trust the license listed in gemspecs alone, you are almost guaranteed to be in violation of a license somewhere. ;)
- shakna 7y ago
  It also can get things problematically wrong. Like licenses based on BSD3 with extra clauses tend to show up as BSD3. But people will trust the license tag, and end up breaching the license.
m4rtink 7y ago
Well, Linux distros are already pretty much that - crow-sourced dependency vetting platforms. Also take care of the bits actually fitting together & quite a bit of QA.
- squiggleblaz 7y ago
  And so very old.
  - the8472 7y ago
    if you want the newest and shiniest there are rolling release distros.
squiggleblaz 7y ago
> A trivial solution would be to create a crowd-sourced dependency vetting platform. And then aren't you just recreating the old fashioned Linux distribution? No-one writes webapps to Linux distros any more; they're always based on language-specific, author-submitted, untrusted package managers because waiting for enough trust to build up to include the latest version of unnecessary-wrapper-for-document.getElementById-0.83.2 is considered stifling. Maybe nixpkgs comes a little close, somewhat trusted and requiring third-party involvement, multiple versions simultaneously, but including new packages quickly enough. (And indeed, nixpkgs acts as distribution that can run on top of your distro or standalone as NixOS.)
tsteenbe 7y ago
> A trivial solution would be to create a crowd-sourced dependency vetting platform. Such a platform is already being developed by various organizations, see https://clearlydefined.io/about https://clearlydefined.io/about and it already in use by GitHub. It's still under development and the initial focus is sharing data regarding copyrights, licenses and source code location for OSS packages. Note that ClearyDefined only scans the code repository of OSS project it does not resolve dependencies for scanned OSS project. This makes sense as not all dependencies of OSS project B are inherited by a project A which included B as a dependency (due to dependency version resolution by the package manager or dependencies only used for testing). The idea is that you will us an tool to resolve dependencies for the package manager your project uses which then queries the ClearyDefined APIs. We are building such a tool as an open source project to manage of licensing and security for dependencies named OSS Review Toolkit (https://github.com/heremaps/oss-review-toolkit https://github.com/heremaps/oss-review-toolkit). For a overview of the solution we are building see slide 9 in https://static.sched.com/hosted_files/ocs19/c7/OSS-Review-Toolkit-Using-FOSS-tools-for-FOSS-reviews-in-CI-CD-world.pdf https://static.sched.com/hosted_files/ocs19/c7/OSS-Review-To... Full disclosure: I am one of the maintainers of OSS Review Toolkit and also a contributor to ClearlyDefined.

by 256dpi 7y ago

kijin 7y ago

GitHub's license detection algorithm is crap. It tends to get GPLv2 right, but most other licenses are hit-and-miss. And I still can't find a setting where I can manually specify the license when GitHub can't autodetect one or gets it wrong.

rurounijones 7y ago
Ruby gems have the license as an explicit field in the gemspec (manifest file). Is this not the case elsewhere? It makes license scanning a doddle.
- tsteenbe 7y ago
  > It makes license scanning a doddle. As someone working in the field of license compliance (leading an Open Source Program Office) and dealing with the licensing of tens of thousands of OSS dependencies on a daily basis for a large company I can tell you that license scanning / compliance is anything but a doddle. I do a lot of public speaking on this topic for a summary of the issues I recommend you to have a look at https://static.sched.com/hosted_files/ocs19/c7/OSS-Review-Toolkit-Using-FOSS-tools-for-FOSS-reviews-in-CI-CD-world.pdf https://static.sched.com/hosted_files/ocs19/c7/OSS-Review-To... Most modern package managers do offer project maintainers a way to declare the license for a project. However I can tell you that often the declared license does not match the licenses detected in the source code. What counts is the license stated in the source code files not what it's in the gemspec, package.json, pom.xml etc. This is quite common issue especially for older or larger OSS projects where various contributors have added new code over time that may be licensed under an OSS license that is compatible but not the same as the main license of the projects (Think adding BSD-2-Clause in Apache-2.0 project) What happens is that this contributions get accepted but the project maintainers do not to update the declared license. Not saying declaring a licensing in a .gemspec file is useless but just recommend you to take it as an indicator of the main license of the project - the project may include source code that licensed under a different license. Lack of clarity around licenses and security vulnerabilities is a big issue within the OSS community especially as lack of clarity reduces engagement — that means fewer users, fewer contributors and a smaller community. Several organizations are working on a open source solution for this community problems see for further details https://clearlydefined.io/about https://clearlydefined.io/about Full disclosure I am one of the maintainer of OSS Review Toolkit and contributor to ClearlyDefined.io
- saagarjha 7y ago
  An even older tradition is a LICENSE file in the root of your project, which GitHub seems to read, but it's not great at matching up the license text to actual licenses.
  - kijin 7y ago
    An alternative tradition uses a COPYING file. I'm not sure which is older, but most GNU projects use COPYING instead of LICENSE. Meanwhile, some projects use COPYRIGHT or other variation, BSD-style licenses are often added directly to source files, and properly applying LGPLv3 to your project involves adding a separate COPYING.LESSER file. It's a mess.
- ratww 7y ago
  I think most modern package managers have something like that: NPM has a license field in package.json, pypi in pkg-info, nuget in the nuspec file, etc. Github uses the LICENSE file to detect licenses by default IIRC.
- jsjohnst 7y ago
  If you trust the license listed in gemspecs alone, you are almost guaranteed to be in violation of a license somewhere. ;)
shakna 7y ago
It also can get things problematically wrong. Like licenses based on BSD3 with extra clauses tend to show up as BSD3. But people will trust the license tag, and end up breaching the license.

m4rtink 7y ago

Well, Linux distros are already pretty much that - crow-sourced dependency vetting platforms. Also take care of the bits actually fitting together & quite a bit of QA.

squiggleblaz 7y ago
And so very old.
- the8472 7y ago
  if you want the newest and shiniest there are rolling release distros.

squiggleblaz 7y ago

> A trivial solution would be to create a crowd-sourced dependency vetting platform. And then aren't you just recreating the old fashioned Linux distribution? No-one writes webapps to Linux distros any more; they're always based on language-specific, author-submitted, untrusted package managers because waiting for enough trust to build up to include the latest version of unnecessary-wrapper-for-document.getElementById-0.83.2 is considered stifling. Maybe nixpkgs comes a little close, somewhat trusted and requiring third-party involvement, multiple versions simultaneously, but including new packages quickly enough. (And indeed, nixpkgs acts as distribution that can run on top of your distro or standalone as NixOS.)

tsteenbe 7y ago

> A trivial solution would be to create a crowd-sourced dependency vetting platform. Such a platform is already being developed by various organizations, see https://clearlydefined.io/about https://clearlydefined.io/about and it already in use by GitHub. It's still under development and the initial focus is sharing data regarding copyrights, licenses and source code location for OSS packages. Note that ClearyDefined only scans the code repository of OSS project it does not resolve dependencies for scanned OSS project. This makes sense as not all dependencies of OSS project B are inherited by a project A which included B as a dependency (due to dependency version resolution by the package manager or dependencies only used for testing). The idea is that you will us an tool to resolve dependencies for the package manager your project uses which then queries the ClearyDefined APIs. We are building such a tool as an open source project to manage of licensing and security for dependencies named OSS Review Toolkit (https://github.com/heremaps/oss-review-toolkit https://github.com/heremaps/oss-review-toolkit). For a overview of the solution we are building see slide 9 in https://static.sched.com/hosted_files/ocs19/c7/OSS-Review-Toolkit-Using-FOSS-tools-for-FOSS-reviews-in-CI-CD-world.pdf https://static.sched.com/hosted_files/ocs19/c7/OSS-Review-To... Full disclosure: I am one of the maintainers of OSS Review Toolkit and also a contributor to ClearlyDefined.