3 ms·

I would honestly completely ditch the Socket "ratings". React = 70 - what the hell does that mean? "quality = 82, license = 70" where did you get those numbers?

I would honestly completely ditch the Socket "ratings". React = 70 - what the hell does that mean? "quality = 82, license = 70" where did you get those numbers? They are basically made-up. There are certain ways you can objectively determine the security of a package: - If it or one of its dependencies has been compromised - If it and its dependencies are made by a trusted org (e.g. I would trust something from Microsoft more than something from a random developer) - If it and its dependencies have been audited and verified (this could even be a quality audit, or something like the Elm ecosystem) - If it and its dependencies are made by you, you can be pretty confident that nobody has tampered with them Similarly there are ways to determine the quality (e.g. code coverage, usage, whether it has tests). And of course you can see the license. But state these things directly, don't try to summarize them with numbers. Because ultimately those numbers are almost meaningless and make your tool look fake.

feross 5y ago
I think this is a valid criticism of using scores to summarize a complex subject, and I agree that we can definitely do a better job on this. What's interesting is that arbitrary, unit-less scores seem to work well in products like Lighthouse [1] and SSL Labs [2] even though they're also "basically made up" metrics. I'm curious if anyone has ideas about why this may be the case. To your point about just stating things directly, we also do that too. See https://socket.dev/npm/package/left-pad/issues https://socket.dev/npm/package/left-pad/issues, for example. [1]: https://developers.google.com/web/tools/lighthouse https://developers.google.com/web/tools/lighthouse [2]: https://www.ssllabs.com/ssltest/index.html https://www.ssllabs.com/ssltest/index.html
- notafraudster 5y ago
  SSL Labs works because letter grades, like most low granularity ordinal measures, clearly read as qualitative and approximate rather than having false precision. No one is saying things can't be in aggregate better or worse. Lighthouse is closer to nonsense, though the category and subcategory explanations at least seem transparent enough.

feross 5y ago

I think this is a valid criticism of using scores to summarize a complex subject, and I agree that we can definitely do a better job on this. What's interesting is that arbitrary, unit-less scores seem to work well in products like Lighthouse [1] and SSL Labs [2] even though they're also "basically made up" metrics. I'm curious if anyone has ideas about why this may be the case. To your point about just stating things directly, we also do that too. See https://socket.dev/npm/package/left-pad/issues https://socket.dev/npm/package/left-pad/issues, for example. [1]: https://developers.google.com/web/tools/lighthouse https://developers.google.com/web/tools/lighthouse [2]: https://www.ssllabs.com/ssltest/index.html https://www.ssllabs.com/ssltest/index.html

notafraudster 5y ago
SSL Labs works because letter grades, like most low granularity ordinal measures, clearly read as qualitative and approximate rather than having false precision. No one is saying things can't be in aggregate better or worse. Lighthouse is closer to nonsense, though the category and subcategory explanations at least seem transparent enough.