If some of society has benefitted and the organization has not made a profit from those benefits, then I'd say that the non-profit claim is valid and we are all

If some of society has benefitted and the organization has not made a profit from those benefits, then I'd say that the non-profit claim is valid and we are all better off for it. There are tons of people who use MongoDB without paying MongoDB(Tengen) a penny. Even though I totally understand the IRS concerns, eliminating this benefit creates a disincentive for the commons, and we are all poorer because of it. I think one of the metrics that could be used here to determine full or partial non-profit status is the community around a piece of software. The active community is a representative, albeit biased/skewed, sample of the entire community using the open source software in question. Polling these unrelated parties (i.e. there is no contractual obligation between them and the software producing entity beyond a commonly accepted open source license) should be sufficient to figure out if an organization should qualify for non-profit status.