4 ms·

Ask HN: How do you measure risk with an open source project?

• kazinator 11y ago
  * I look at the code and determine, subjectively, whether this was written by first-rate developers or monkeys. I will consider this from various angles ranging from the overall program organization, to the details of how the programming language is used. If I spot bugs in this inspection, I will skip the project and look for something else. In particular anything that is a security flaw or could cause a crash is an instant deal-breaker. Not because everyone should be perfect and write error-free code the firs time, but because I was able to find it just by casually looking, whereas the maintainers have been working with that code for months and are blind to it---that erodes my confidence in the developers. * I will look for a regression test suite: how extensive is it? If you don't see any tests, that's a big warning sign. * If you don't see any documentation, another warning sign: the behavior is not specified and could change. * Who uses it? If the code is reasonably widely used, that de-risks it for you quite a bit, particularly if the existing uses resemble your intended use. They trod through the code paths before you and uncovered the bugs. * Lastly: if need be, can I just maintain this myself? How easily forkable is the thing? This point can overcome some other issues. If some code is 95% of the way there, and is organized in a good way that I can take it the last 5%, I might just do it.
  • bazMVP 11y ago
    Thank you for the insight, the last point (self maintenance) is something I've thought about too. I think the risk is lower if the project would not longer be supported and the skill to continue it could be hired or learned quickly. Great point on regression tests and associated coverage.
  • partisan 11y ago
    Keep in mind that the last 5% might mean a heck of a lot of time. There is usually a reason why some things go unimplemented in open source projects and the complexity and time required are usually high on the list.
    • kazinator 11y ago
      The last 5% is my last 5%, not (necessarily) the project's 5%. I.e. the difference between what is there and what I require, not between what is there, and the project's goals.
• twunde 11y ago
  When was the last time the code was updated? How many people have contributed to it?Really I'm looking to see whether the project is being actively maintained.
• Gratsby 11y ago
  If there's an IRC channel, mailing list, issue tracker, etc., I have a look at it. Active communities are bonus points regardless of the bug list. I also look at how friendly the project is to pull requests and outside development. If a bug is important to me, I will spend the time to code a fix, but if there's no hope for getting any changes made, that represents a large risk to me. It's not a bad thing if the team in charge pushes back for higher quality code, code style, or solutions practical for wider audiences. If there's continuous integration in place with automated testing and static code analysis that is fantastic. It's not a deal breaker if it's not, but having it in place is a good sign. Depending on the project, I may have a look at the source itself. I certainly don't look the source of every application I use. I have found that online recommendations in developer communities are not always good. More than once I've tried out projects based on people evangelizing them only to find out that they are pretty far from acceptable. There are differences in how I judge things that will be customer facing or that I'm going to have to support operationally. I'm a lot more critical at that point, but my top 3 points are: 1. Can I put it in a container. 2. Are the developers committed to long term support. 3. Will they give me a t-shirt.
• cweagans 11y ago
  I evaluate it the same way that I evaluate any other code that I'm bringing into our project: is it robust, extensible, free of obvious errors, covered by comprehensive unit/functional tests, etc. I would say that "project is open 6 months" is a pretty poor metric, because 6 months is a lot longer than the handful of days that your custom code will have existed when you add it instead.
  • bazMVP 11y ago
    That makes sense, any time period is probably less valuable than looking at it relative to commits or issue resolution rate.
    • cweagans 11y ago
      Even evaluating it on historical commits or issue resolution rate is not very productive, IMO. They really aren't an indication of future activity unless there are at least 10s of developers working on it constantly for a long period of time (say, > 6 months). I've been burned too many times by getting hopeful about what a project maintainer is going to do in the future, so my criteria is basically taking the code in it's current state and evaluating it like I would any other code. If it's a matter of opening a PR or two to get things cleaned up, and PRs seem to be merged relatively quickly, I might chance it, but I can't wait on an unresponsive maintainer to merge things. I have deadlines to meet. Ain't nobody got time for that.
• thealistra 11y ago
  0. Is README in good shape and the API is sane? 1. Last commit date in last month 2. Number of stars depending on the complexity of the lib