3 ms·

I started the chemfp project in part to see if I could develop a self-funded free/open source product in my field, cheminformatics. (In short, storing and searc

by dalke 6y ago

I started the chemfp project in part to see if I could develop a self-funded free/open source product in my field, cheminformatics. (In short, storing and searching chemical information on a computer. Chemfp does very fast Jaccard-Tanimoto similarity search for "short"/O(1024 bit) bitstrings.) The answer: no. The section I linked to highlights some of the problems I had selling software under the principles of free software. For examples: How do I provide a demo if I always provide MIT licensed source code? Academics expect discounts, but they are also the ones most likely to redistribute the code. Which is not a wrong thing to do! But it affects the economics in a way I could never resolve, compared to proprietary/"software hoarding" licensing models. As an HN note, I contracted a couple people to help improve the popcount implementations. HN user nkurz developed and tweaked the AVX2 implementation, and proof-read the paper. Thanks nkurz! As a result, chemfp is, I believe, the fastest single-threaded Tanimoto search implementation for CPUs available, and most likely memory bandwidth limited, not CPU limited. (Note: the mods asked me to repost. My earlier post is at https://news.ycombinator.com/item?id=23598470 https://news.ycombinator.com/item?id=23598470 .)

deleted 6y ago
[deleted]
ssivark 6y ago
Very interesting experiment. Did you consider having academics cite your software as an academic work, and then monetizing those citations to get grants from funding agencies who were funding the research that used your software? Given this knowledge, can you think of other similar experiments worth performing? Alternatively, are there any likely changes that might lead to such approaches becoming more feasible? I’m quite interested in this question and appreciate your comments. In case you’ve already answered these in the article, I apologize — the article is long and the HN thread will likely expire before I have the chance to peruse it thoroughly.
- dalke 6y ago
  I tried talking with funding agencies. As a self-employed/sole-proprietership, it proved to be difficult. I conducted the "mmpdb crowdfunding project" at http://mmpdb.dalkescientific.com/ http://mmpdb.dalkescientific.com/ . Unfortunately, it's been on delay because of major changes to my work schedule once the coronovirus came. My conclusion from that was it's still easier for me to make money as a consultant than from selling a product. One thing I didn't mention in the paper was that being a consultant on different projects makes it easy to present new talks at conferences, which leads to new work. Continued development of a single project with one focus leads to variations on a theme, feeling sometimes like "have you ever looked at your hand. I mean, really looked at your hand" introspection. You can always email me as follow up. dalke at dalkescientific.
maximente 6y ago
in your future ideas section you have listed a few features or directions that the software could go. what about selling those? in my mind, in order to monetize a greenfields FLOSS project, it seems you need to basically create software so impactful that users are willing to pay for particular features or bug fixes that they need, and that don't already exist in the software. so basically, you have to not only get people to use this new thing, but also get them to be willing to pay for particular improvements to it! quite a task. this first occurred to me when i heard Stallman talk about how one can monetize FLOSS projects by selling bugfixes or improvements. contrast the typical developer-driven setup where a team implements the feature then sells it by selling a new version, upgrade fee, or what have you. so it's a tricky situation, in that one needs to offer a compelling product, but not something so good that individuals can take it from the shelf and never have to do anything to it. however, this is sort of helpful insofar as it acts as a way for you to ship something and then - hopefully - let users drive the product (we'd pay $X for a UI on top). for cheminformatics, i'm not really sure what some killer features are, but you have at least some ideas as potential things to sell. perhaps the community would be interested in pooling together funds for some of those ideas, or a UI/UX, or whatever - but yeah, definitely seems more customer-driven then traditional software.
- dalke 6y ago
  Yes, those future ideas are also possible salable products, thought the market is smaller. I had hoped that the surplus from the core bitstring search would provide the seed money for their development. Chemfp is (or rather was) at least an order of magnitude faster than its competitors. One of my early customers went from taking 5 days to cluster 1M fingerprints using their internal tool to several hours with a re-writes based on a chemfp core. I figured that was sufficiently impactful. But that was in the early stage when people did fund me to add new features as an open source project. I think that once it got "good enough", people didn't want to fund the incrementally smaller improvements. The commercial version is only about 2x faster than the no-cost version. I would talk with potential customers and they already had the no-cost version installed in-house. Now, there are a lot of other improvements in the commercial version, but it's a variation on the "offer a compelling product" - I was my own competition. That was the primary failure in the original business model, based on Ghostscript. (I didn't realize that hardware companies paid Aladdin to keep up with Adobe Postscript, which provided the reason to pay for the latest version rather a no-cost older version.) I tried crowdfunding a project, ie, "pooling together funds", for another project. It worked, in that it made enough money to fund the software development and the marketing, but it also required a lot of time for the marketing (which I'm not good at), and I would have made more money doing straight consulting work instead of product development. BTW, I focus on "classic" cheminformatics. Similarity search is from the 1980s, with roots in the 1970s. My thesis is that there people haven't looked into these topics in years and there might be orders of magnitude gains by developing better fits to modern architecture, that is, improve existing methods enough that UI/UX changes. Most other people's definition of "killer feature" is "an improveed method predict how a molecule will work in the body."
alicemaz 6y ago
have you considered offering it as a cloud platform? we're doing something along these lines, niche scientific software (biological modeling, bioinformatics) as a paid hosted service. still at the prototype stage! so I can't comment on how well the business model actually works yet lol but the idea is our mathematician will be able to publish whatever novel math she develops, and we may eventually open source the math core as a reference impl, but we'll keep all the cluster management and other supporting infrastructure code proprietary. sort of a "if you want to run it on your desktop, go ahead! if you want to actually scale this up for big jobs, we've done all the legwork already so it's really in your best interests to just pay us." I think open source ideals are good and worthy but from a business perspective, you capture value by providing value that can't be got without you. relying on customer goodwill is particularly difficult because any large org, the people who will feel goodwill toward you and the people who can authorize purchases are in two different departments also fwiw I think if you wanted to do the model you described in the paper unchanged, gpl is a much better choice than mit. copyleft actually serves as a wonderful poison pill: you can try us out for free, but if you want to ship us, you need to pay for a proprietary license or legal will nail you to the wall. whereas mit, there's no stick. I've seen affero used by several projects for this express purpose: you have to buy a proprietary license because agpl is so onerous you just can't use the code for commercial purposes at all interesting project btw, I love seeing stuff like this!
- dalke 6y ago
  Thanks! Yes, I've considered cloud platform. There are several big difficulties with that. First, data. It's easy to grab public data from PubChem, ChEMBL, and a few other projects, and make a service. But why would anyone pay for it given that PubChem, ChEMBL, ChemSpider, and others already provide free search services of that data? There's search-as-improved-sales, like how Sigma-Aldrich lets people do a substructure search to find chemicals available for sale. There's value-add data. eMolecules includes data from multiple vendors, to help those who want to purchase compounds more cheaply. Or there's ZINC, which already provides search for their data. So you can see there's plenty of competition for no-cost search. I don't have the ability to add significantly new abilities that people are willing to pay for. Note also there's a non-trivial maintenance cost to keep the data sets up-to-date. Second, the queries themselves may be proprietary. I talked with one of the eMolecules people. Pharmaceutical companies will block network access to a public services to reduce the temptation of internal users to do a query using a potential $1 billion molecular structure (or potential $0 structure). eMolecules instead has NDAs with many pharmas which legal bind them. Managing these negotiations takes experience I don't have, and neither do I have the right contacts at those pharmas. Sequences don't have quite the same connection between sequence and profit as molecules do. BTW, part of the conclusion of my work is that people don't need a cluster for search - they can handle nearly all data sets on their laptop, so there shouldn't be a need to scale up any more. And small molecule data has a much smaller growth curve than sequence data, so Moore's Law is keeping up. My first customer, who continues to be a customer, said outright that they would not buy if it were under GPL. Since my paying customers are pharmaceutical companies who, as a near-rule, don't redistribute software, it doesn't really matter if they don't redistribute under MIT or don't redistribute under GPL. I came into the project in part to see if FOSS could be self-supporting on it own. AGPL is often used as a stick to try to get people to use a commercial license - the implicit view of the two-license model is that FOSS is not sustainable. Which is now my conclusion, for this project and field.

by dalke 6y ago

deleted 6y ago

[deleted]

ssivark 6y ago

Very interesting experiment. Did you consider having academics cite your software as an academic work, and then monetizing those citations to get grants from funding agencies who were funding the research that used your software? Given this knowledge, can you think of other similar experiments worth performing? Alternatively, are there any likely changes that might lead to such approaches becoming more feasible? I’m quite interested in this question and appreciate your comments. In case you’ve already answered these in the article, I apologize — the article is long and the HN thread will likely expire before I have the chance to peruse it thoroughly.

dalke 6y ago
I tried talking with funding agencies. As a self-employed/sole-proprietership, it proved to be difficult. I conducted the "mmpdb crowdfunding project" at http://mmpdb.dalkescientific.com/ http://mmpdb.dalkescientific.com/ . Unfortunately, it's been on delay because of major changes to my work schedule once the coronovirus came. My conclusion from that was it's still easier for me to make money as a consultant than from selling a product. One thing I didn't mention in the paper was that being a consultant on different projects makes it easy to present new talks at conferences, which leads to new work. Continued development of a single project with one focus leads to variations on a theme, feeling sometimes like "have you ever looked at your hand. I mean, really looked at your hand" introspection. You can always email me as follow up. dalke at dalkescientific.

maximente 6y ago

in your future ideas section you have listed a few features or directions that the software could go. what about selling those? in my mind, in order to monetize a greenfields FLOSS project, it seems you need to basically create software so impactful that users are willing to pay for particular features or bug fixes that they need, and that don't already exist in the software. so basically, you have to not only get people to use this new thing, but also get them to be willing to pay for particular improvements to it! quite a task. this first occurred to me when i heard Stallman talk about how one can monetize FLOSS projects by selling bugfixes or improvements. contrast the typical developer-driven setup where a team implements the feature then sells it by selling a new version, upgrade fee, or what have you. so it's a tricky situation, in that one needs to offer a compelling product, but not something so good that individuals can take it from the shelf and never have to do anything to it. however, this is sort of helpful insofar as it acts as a way for you to ship something and then - hopefully - let users drive the product (we'd pay $X for a UI on top). for cheminformatics, i'm not really sure what some killer features are, but you have at least some ideas as potential things to sell. perhaps the community would be interested in pooling together funds for some of those ideas, or a UI/UX, or whatever - but yeah, definitely seems more customer-driven then traditional software.

dalke 6y ago
Yes, those future ideas are also possible salable products, thought the market is smaller. I had hoped that the surplus from the core bitstring search would provide the seed money for their development. Chemfp is (or rather was) at least an order of magnitude faster than its competitors. One of my early customers went from taking 5 days to cluster 1M fingerprints using their internal tool to several hours with a re-writes based on a chemfp core. I figured that was sufficiently impactful. But that was in the early stage when people did fund me to add new features as an open source project. I think that once it got "good enough", people didn't want to fund the incrementally smaller improvements. The commercial version is only about 2x faster than the no-cost version. I would talk with potential customers and they already had the no-cost version installed in-house. Now, there are a lot of other improvements in the commercial version, but it's a variation on the "offer a compelling product" - I was my own competition. That was the primary failure in the original business model, based on Ghostscript. (I didn't realize that hardware companies paid Aladdin to keep up with Adobe Postscript, which provided the reason to pay for the latest version rather a no-cost older version.) I tried crowdfunding a project, ie, "pooling together funds", for another project. It worked, in that it made enough money to fund the software development and the marketing, but it also required a lot of time for the marketing (which I'm not good at), and I would have made more money doing straight consulting work instead of product development. BTW, I focus on "classic" cheminformatics. Similarity search is from the 1980s, with roots in the 1970s. My thesis is that there people haven't looked into these topics in years and there might be orders of magnitude gains by developing better fits to modern architecture, that is, improve existing methods enough that UI/UX changes. Most other people's definition of "killer feature" is "an improveed method predict how a molecule will work in the body."

alicemaz 6y ago

have you considered offering it as a cloud platform? we're doing something along these lines, niche scientific software (biological modeling, bioinformatics) as a paid hosted service. still at the prototype stage! so I can't comment on how well the business model actually works yet lol but the idea is our mathematician will be able to publish whatever novel math she develops, and we may eventually open source the math core as a reference impl, but we'll keep all the cluster management and other supporting infrastructure code proprietary. sort of a "if you want to run it on your desktop, go ahead! if you want to actually scale this up for big jobs, we've done all the legwork already so it's really in your best interests to just pay us." I think open source ideals are good and worthy but from a business perspective, you capture value by providing value that can't be got without you. relying on customer goodwill is particularly difficult because any large org, the people who will feel goodwill toward you and the people who can authorize purchases are in two different departments also fwiw I think if you wanted to do the model you described in the paper unchanged, gpl is a much better choice than mit. copyleft actually serves as a wonderful poison pill: you can try us out for free, but if you want to ship us, you need to pay for a proprietary license or legal will nail you to the wall. whereas mit, there's no stick. I've seen affero used by several projects for this express purpose: you have to buy a proprietary license because agpl is so onerous you just can't use the code for commercial purposes at all interesting project btw, I love seeing stuff like this!

dalke 6y ago
Thanks! Yes, I've considered cloud platform. There are several big difficulties with that. First, data. It's easy to grab public data from PubChem, ChEMBL, and a few other projects, and make a service. But why would anyone pay for it given that PubChem, ChEMBL, ChemSpider, and others already provide free search services of that data? There's search-as-improved-sales, like how Sigma-Aldrich lets people do a substructure search to find chemicals available for sale. There's value-add data. eMolecules includes data from multiple vendors, to help those who want to purchase compounds more cheaply. Or there's ZINC, which already provides search for their data. So you can see there's plenty of competition for no-cost search. I don't have the ability to add significantly new abilities that people are willing to pay for. Note also there's a non-trivial maintenance cost to keep the data sets up-to-date. Second, the queries themselves may be proprietary. I talked with one of the eMolecules people. Pharmaceutical companies will block network access to a public services to reduce the temptation of internal users to do a query using a potential $1 billion molecular structure (or potential $0 structure). eMolecules instead has NDAs with many pharmas which legal bind them. Managing these negotiations takes experience I don't have, and neither do I have the right contacts at those pharmas. Sequences don't have quite the same connection between sequence and profit as molecules do. BTW, part of the conclusion of my work is that people don't need a cluster for search - they can handle nearly all data sets on their laptop, so there shouldn't be a need to scale up any more. And small molecule data has a much smaller growth curve than sequence data, so Moore's Law is keeping up. My first customer, who continues to be a customer, said outright that they would not buy if it were under GPL. Since my paying customers are pharmaceutical companies who, as a near-rule, don't redistribute software, it doesn't really matter if they don't redistribute under MIT or don't redistribute under GPL. I came into the project in part to see if FOSS could be self-supporting on it own. AGPL is often used as a stick to try to get people to use a commercial license - the implicit view of the two-license model is that FOSS is not sustainable. Which is now my conclusion, for this project and field.