7 ms·

The chemfp project: problems selling free software

66 points by dalke 6y ago

dalke 6y ago
I started the chemfp project in part to see if I could develop a self-funded free/open source product in my field, cheminformatics. (In short, storing and searching chemical information on a computer. Chemfp does very fast Jaccard-Tanimoto similarity search for "short"/O(1024 bit) bitstrings.) The answer: no. The section I linked to highlights some of the problems I had selling software under the principles of free software. For examples: How do I provide a demo if I always provide MIT licensed source code? Academics expect discounts, but they are also the ones most likely to redistribute the code. Which is not a wrong thing to do! But it affects the economics in a way I could never resolve, compared to proprietary/"software hoarding" licensing models. As an HN note, I contracted a couple people to help improve the popcount implementations. HN user nkurz developed and tweaked the AVX2 implementation, and proof-read the paper. Thanks nkurz! As a result, chemfp is, I believe, the fastest single-threaded Tanimoto search implementation for CPUs available, and most likely memory bandwidth limited, not CPU limited. (Note: the mods asked me to repost. My earlier post is at https://news.ycombinator.com/item?id=23598470 https://news.ycombinator.com/item?id=23598470 .)
- deleted 6y ago
  [deleted]
- ssivark 6y ago
  Very interesting experiment. Did you consider having academics cite your software as an academic work, and then monetizing those citations to get grants from funding agencies who were funding the research that used your software? Given this knowledge, can you think of other similar experiments worth performing? Alternatively, are there any likely changes that might lead to such approaches becoming more feasible? I’m quite interested in this question and appreciate your comments. In case you’ve already answered these in the article, I apologize — the article is long and the HN thread will likely expire before I have the chance to peruse it thoroughly.
  - dalke 6y ago
    I tried talking with funding agencies. As a self-employed/sole-proprietership, it proved to be difficult. I conducted the "mmpdb crowdfunding project" at http://mmpdb.dalkescientific.com/ http://mmpdb.dalkescientific.com/ . Unfortunately, it's been on delay because of major changes to my work schedule once the coronovirus came. My conclusion from that was it's still easier for me to make money as a consultant than from selling a product. One thing I didn't mention in the paper was that being a consultant on different projects makes it easy to present new talks at conferences, which leads to new work. Continued development of a single project with one focus leads to variations on a theme, feeling sometimes like "have you ever looked at your hand. I mean, really looked at your hand" introspection. You can always email me as follow up. dalke at dalkescientific.
- maximente 6y ago
  in your future ideas section you have listed a few features or directions that the software could go. what about selling those? in my mind, in order to monetize a greenfields FLOSS project, it seems you need to basically create software so impactful that users are willing to pay for particular features or bug fixes that they need, and that don't already exist in the software. so basically, you have to not only get people to use this new thing, but also get them to be willing to pay for particular improvements to it! quite a task. this first occurred to me when i heard Stallman talk about how one can monetize FLOSS projects by selling bugfixes or improvements. contrast the typical developer-driven setup where a team implements the feature then sells it by selling a new version, upgrade fee, or what have you. so it's a tricky situation, in that one needs to offer a compelling product, but not something so good that individuals can take it from the shelf and never have to do anything to it. however, this is sort of helpful insofar as it acts as a way for you to ship something and then - hopefully - let users drive the product (we'd pay $X for a UI on top). for cheminformatics, i'm not really sure what some killer features are, but you have at least some ideas as potential things to sell. perhaps the community would be interested in pooling together funds for some of those ideas, or a UI/UX, or whatever - but yeah, definitely seems more customer-driven then traditional software.
  - dalke 6y ago
    Yes, those future ideas are also possible salable products, thought the market is smaller. I had hoped that the surplus from the core bitstring search would provide the seed money for their development. Chemfp is (or rather was) at least an order of magnitude faster than its competitors. One of my early customers went from taking 5 days to cluster 1M fingerprints using their internal tool to several hours with a re-writes based on a chemfp core. I figured that was sufficiently impactful. But that was in the early stage when people did fund me to add new features as an open source project. I think that once it got "good enough", people didn't want to fund the incrementally smaller improvements. The commercial version is only about 2x faster than the no-cost version. I would talk with potential customers and they already had the no-cost version installed in-house. Now, there are a lot of other improvements in the commercial version, but it's a variation on the "offer a compelling product" - I was my own competition. That was the primary failure in the original business model, based on Ghostscript. (I didn't realize that hardware companies paid Aladdin to keep up with Adobe Postscript, which provided the reason to pay for the latest version rather a no-cost older version.) I tried crowdfunding a project, ie, "pooling together funds", for another project. It worked, in that it made enough money to fund the software development and the marketing, but it also required a lot of time for the marketing (which I'm not good at), and I would have made more money doing straight consulting work instead of product development. BTW, I focus on "classic" cheminformatics. Similarity search is from the 1980s, with roots in the 1970s. My thesis is that there people haven't looked into these topics in years and there might be orders of magnitude gains by developing better fits to modern architecture, that is, improve existing methods enough that UI/UX changes. Most other people's definition of "killer feature" is "an improveed method predict how a molecule will work in the body."
- alicemaz 6y ago
  have you considered offering it as a cloud platform? we're doing something along these lines, niche scientific software (biological modeling, bioinformatics) as a paid hosted service. still at the prototype stage! so I can't comment on how well the business model actually works yet lol but the idea is our mathematician will be able to publish whatever novel math she develops, and we may eventually open source the math core as a reference impl, but we'll keep all the cluster management and other supporting infrastructure code proprietary. sort of a "if you want to run it on your desktop, go ahead! if you want to actually scale this up for big jobs, we've done all the legwork already so it's really in your best interests to just pay us." I think open source ideals are good and worthy but from a business perspective, you capture value by providing value that can't be got without you. relying on customer goodwill is particularly difficult because any large org, the people who will feel goodwill toward you and the people who can authorize purchases are in two different departments also fwiw I think if you wanted to do the model you described in the paper unchanged, gpl is a much better choice than mit. copyleft actually serves as a wonderful poison pill: you can try us out for free, but if you want to ship us, you need to pay for a proprietary license or legal will nail you to the wall. whereas mit, there's no stick. I've seen affero used by several projects for this express purpose: you have to buy a proprietary license because agpl is so onerous you just can't use the code for commercial purposes at all interesting project btw, I love seeing stuff like this!
  - dalke 6y ago
    Thanks! Yes, I've considered cloud platform. There are several big difficulties with that. First, data. It's easy to grab public data from PubChem, ChEMBL, and a few other projects, and make a service. But why would anyone pay for it given that PubChem, ChEMBL, ChemSpider, and others already provide free search services of that data? There's search-as-improved-sales, like how Sigma-Aldrich lets people do a substructure search to find chemicals available for sale. There's value-add data. eMolecules includes data from multiple vendors, to help those who want to purchase compounds more cheaply. Or there's ZINC, which already provides search for their data. So you can see there's plenty of competition for no-cost search. I don't have the ability to add significantly new abilities that people are willing to pay for. Note also there's a non-trivial maintenance cost to keep the data sets up-to-date. Second, the queries themselves may be proprietary. I talked with one of the eMolecules people. Pharmaceutical companies will block network access to a public services to reduce the temptation of internal users to do a query using a potential $1 billion molecular structure (or potential $0 structure). eMolecules instead has NDAs with many pharmas which legal bind them. Managing these negotiations takes experience I don't have, and neither do I have the right contacts at those pharmas. Sequences don't have quite the same connection between sequence and profit as molecules do. BTW, part of the conclusion of my work is that people don't need a cluster for search - they can handle nearly all data sets on their laptop, so there shouldn't be a need to scale up any more. And small molecule data has a much smaller growth curve than sequence data, so Moore's Law is keeping up. My first customer, who continues to be a customer, said outright that they would not buy if it were under GPL. Since my paying customers are pharmaceutical companies who, as a near-rule, don't redistribute software, it doesn't really matter if they don't redistribute under MIT or don't redistribute under GPL. I came into the project in part to see if FOSS could be self-supporting on it own. AGPL is often used as a stick to try to get people to use a commercial license - the implicit view of the two-license model is that FOSS is not sustainable. Which is now my conclusion, for this project and field.
zokier 6y ago
I think to truly appreciate FOSS as a model, one needs to shift away from thinking software as an asset to be monetized to more of a liability that needs to be managed and maintained. Then the benefit of FOSS becomes clear: by publishing your software there is possibility of sharing that burden with others instead of carrying it alone yourself.
- taneq 6y ago
  I agree. I can't see how you can 'sell free software' as a standalone product. You build and evangelise free software while selling feature requests, services and support to the users.
  - dalke 6y ago
    You can sell a proprietary product, right? With a restrictive software license? That means customers are willing to pay $base + $yearly renewal for the product. Why aren't they willing to pay the same price for the same product but with an open source license? I really don't understand why they don't. I'll go one further - how much will people pay for an open source license over a source available license with a right to modify, no time limits/renewal requirement, but no distribution right? Answer: all but one of my customers jumped at the chance to reduce the cost by switching from the MIT license to a not-quite-open-source license. Which means they don't really value the redistribution right. And I saw this at one small conference about industry use of open source. The organizers - who use chemfp! - stated at the start that the biggest reason they love open source is because it's "free" (meaning no cost), not the principles of software freedom nor the improved development methodology of open source. I tried selling feature requests, services and support to the users. That was my original plan, and it worked so long as those feature requests were easy and there were enough of them. But consider that the upgrade to Python 3 took two months. Who pays for that? The first customer who wants Python 3 support 5 years ago, who pays $20K for a feature request which everyone else gets for free? Then there's inventive to wait for a feature request in hopes that someone else will pay it. While the sales model - even as free software - lets me split the cost among multiple customers who need that feature, and across a few years. I also pointed out that selling services is a disincentive to developing good document and good APIs. I feel like there's a sweet spot where if I were to skimp on the documentation some then there's an increased chance of getting consulting work.
dekhn 6y ago
It's funny just how much the implementations described in the paper map to how modern search engines implement retrieval. The same is true for BLAST and other search engines. (it's a very readable paper and I enjoy the frank expression of view, even if I have a vastly different perspective on how to accelerate problems like this)
- nkurz 6y ago
  > even if I have a vastly different perspective on how to accelerate problems like this Do tell more! After the initial optimization, I did hint at an approach to Andrew that I thought could get a further large speedup. Essentially, the idea was to "rotate" all the stored data 90 degrees, so that instead of counting the features present in each compound you read lists of compounds that contain a given feature, storing the hits in some very fast custom data structure. He wasn't particularly interested, likely correctly realizing that it would be a lot of work for an uncertain amount of gain. The question wasn't really whether I could achieve further speedup (although there was question as to how much), rather (as alluded to in the paper) whether he would be able to sufficiently increase sales to justify the additional development cost and added complexity of the codebase.
  - dalke 6y ago
    Towards the end of the writing the paper, another paper came out on "RISC: rapid inverted-index based search of chemical fingerprints", https://doi.org/10.1021/acs.jcim.9b00069 https://doi.org/10.1021/acs.jcim.9b00069 which does something along those lines. It was close enough that I published the pre-print "RISC and dense fingerprints" at https://doi.org/10.26434/chemrxiv.8218517.v1 https://doi.org/10.26434/chemrxiv.8218517.v1 to examine its claims. I found that their RISC implementation was faster than chemfp for low bit densities (<~5%), which includes the popular 2048-bit ECFP/Morgan fingerprints for smaller radii, and uncommonly high similarity thresholds. Otherwise, chemfp was faster. So while there's certainly something to investigate there, I think it's better to focus that effort on truly sparse fingerprints and count fingerprints, rather than nominally dense bit fingerprints. Just needs money and time. ;) Plus, part of the focus was on making chemfp a really good baseline for these sorts of timing tests.
rurban 6y ago
Im sceptical that a good single CPU search can compete with massive parallel HW, like this one: https://www.graphcore.ai/posts/introducing-second-generation-ipu-systems-for-ai-at-scale https://www.graphcore.ai/posts/introducing-second-generation...
- dalke 6y ago
  Sure. It can't. Even GPUs will beat a CPU. In my paper I commented: > GPU memory bandwidth is an order of magnitude higher than CPU bandwidth, so a GPU implementation of the Tanimoto search kernel should be about ten times faster. Chemfp has avoided GPU support so far because it’s not clear that the demand for similarity search justifies dedicated hardware, especially if the time to load the data into the GPU is slower than the time to search it on the CPU. GPUs are more likely to be appropriate for clustering mid-sized datasets where the fingerprints fit into GPU memory. Corporate compound sets have ~5 million records. That can be searched on a laptop in about 50ms. A large data set containing physically measured properties is ~100M records, which takes a bit over a second. The largest data sets people search, with synthetically generated compounds, is around 1G records. That requires distributed computing. But most people don't work with them. They say the best camera is the one you have with you. Most people have a CPU with them. Fewer have massive parallel HW with them.

66 points by dalke 6y ago

I started the chemfp project in part to see if I could develop a self-funded free/open source product in my field, cheminformatics. (In short, storing and searching chemical information on a computer. Chemfp does very fast Jaccard-Tanimoto similarity search for "short"/O(1024 bit) bitstrings.) The answer: no. The section I linked to highlights some of the problems I had selling software under the principles of free software. For examples: How do I provide a demo if I always provide MIT licensed source code? Academics expect discounts, but they are also the ones most likely to redistribute the code. Which is not a wrong thing to do! But it affects the economics in a way I could never resolve, compared to proprietary/"software hoarding" licensing models. As an HN note, I contracted a couple people to help improve the popcount implementations. HN user nkurz developed and tweaked the AVX2 implementation, and proof-read the paper. Thanks nkurz! As a result, chemfp is, I believe, the fastest single-threaded Tanimoto search implementation for CPUs available, and most likely memory bandwidth limited, not CPU limited. (Note: the mods asked me to repost. My earlier post is at https://news.ycombinator.com/item?id=23598470 https://news.ycombinator.com/item?id=23598470 .)

deleted 6y ago
[deleted]
ssivark 6y ago
Very interesting experiment. Did you consider having academics cite your software as an academic work, and then monetizing those citations to get grants from funding agencies who were funding the research that used your software? Given this knowledge, can you think of other similar experiments worth performing? Alternatively, are there any likely changes that might lead to such approaches becoming more feasible? I’m quite interested in this question and appreciate your comments. In case you’ve already answered these in the article, I apologize — the article is long and the HN thread will likely expire before I have the chance to peruse it thoroughly.
- dalke 6y ago
  I tried talking with funding agencies. As a self-employed/sole-proprietership, it proved to be difficult. I conducted the "mmpdb crowdfunding project" at http://mmpdb.dalkescientific.com/ http://mmpdb.dalkescientific.com/ . Unfortunately, it's been on delay because of major changes to my work schedule once the coronovirus came. My conclusion from that was it's still easier for me to make money as a consultant than from selling a product. One thing I didn't mention in the paper was that being a consultant on different projects makes it easy to present new talks at conferences, which leads to new work. Continued development of a single project with one focus leads to variations on a theme, feeling sometimes like "have you ever looked at your hand. I mean, really looked at your hand" introspection. You can always email me as follow up. dalke at dalkescientific.
maximente 6y ago
in your future ideas section you have listed a few features or directions that the software could go. what about selling those? in my mind, in order to monetize a greenfields FLOSS project, it seems you need to basically create software so impactful that users are willing to pay for particular features or bug fixes that they need, and that don't already exist in the software. so basically, you have to not only get people to use this new thing, but also get them to be willing to pay for particular improvements to it! quite a task. this first occurred to me when i heard Stallman talk about how one can monetize FLOSS projects by selling bugfixes or improvements. contrast the typical developer-driven setup where a team implements the feature then sells it by selling a new version, upgrade fee, or what have you. so it's a tricky situation, in that one needs to offer a compelling product, but not something so good that individuals can take it from the shelf and never have to do anything to it. however, this is sort of helpful insofar as it acts as a way for you to ship something and then - hopefully - let users drive the product (we'd pay $X for a UI on top). for cheminformatics, i'm not really sure what some killer features are, but you have at least some ideas as potential things to sell. perhaps the community would be interested in pooling together funds for some of those ideas, or a UI/UX, or whatever - but yeah, definitely seems more customer-driven then traditional software.
- dalke 6y ago
  Yes, those future ideas are also possible salable products, thought the market is smaller. I had hoped that the surplus from the core bitstring search would provide the seed money for their development. Chemfp is (or rather was) at least an order of magnitude faster than its competitors. One of my early customers went from taking 5 days to cluster 1M fingerprints using their internal tool to several hours with a re-writes based on a chemfp core. I figured that was sufficiently impactful. But that was in the early stage when people did fund me to add new features as an open source project. I think that once it got "good enough", people didn't want to fund the incrementally smaller improvements. The commercial version is only about 2x faster than the no-cost version. I would talk with potential customers and they already had the no-cost version installed in-house. Now, there are a lot of other improvements in the commercial version, but it's a variation on the "offer a compelling product" - I was my own competition. That was the primary failure in the original business model, based on Ghostscript. (I didn't realize that hardware companies paid Aladdin to keep up with Adobe Postscript, which provided the reason to pay for the latest version rather a no-cost older version.) I tried crowdfunding a project, ie, "pooling together funds", for another project. It worked, in that it made enough money to fund the software development and the marketing, but it also required a lot of time for the marketing (which I'm not good at), and I would have made more money doing straight consulting work instead of product development. BTW, I focus on "classic" cheminformatics. Similarity search is from the 1980s, with roots in the 1970s. My thesis is that there people haven't looked into these topics in years and there might be orders of magnitude gains by developing better fits to modern architecture, that is, improve existing methods enough that UI/UX changes. Most other people's definition of "killer feature" is "an improveed method predict how a molecule will work in the body."
alicemaz 6y ago
have you considered offering it as a cloud platform? we're doing something along these lines, niche scientific software (biological modeling, bioinformatics) as a paid hosted service. still at the prototype stage! so I can't comment on how well the business model actually works yet lol but the idea is our mathematician will be able to publish whatever novel math she develops, and we may eventually open source the math core as a reference impl, but we'll keep all the cluster management and other supporting infrastructure code proprietary. sort of a "if you want to run it on your desktop, go ahead! if you want to actually scale this up for big jobs, we've done all the legwork already so it's really in your best interests to just pay us." I think open source ideals are good and worthy but from a business perspective, you capture value by providing value that can't be got without you. relying on customer goodwill is particularly difficult because any large org, the people who will feel goodwill toward you and the people who can authorize purchases are in two different departments also fwiw I think if you wanted to do the model you described in the paper unchanged, gpl is a much better choice than mit. copyleft actually serves as a wonderful poison pill: you can try us out for free, but if you want to ship us, you need to pay for a proprietary license or legal will nail you to the wall. whereas mit, there's no stick. I've seen affero used by several projects for this express purpose: you have to buy a proprietary license because agpl is so onerous you just can't use the code for commercial purposes at all interesting project btw, I love seeing stuff like this!
- dalke 6y ago
  Thanks! Yes, I've considered cloud platform. There are several big difficulties with that. First, data. It's easy to grab public data from PubChem, ChEMBL, and a few other projects, and make a service. But why would anyone pay for it given that PubChem, ChEMBL, ChemSpider, and others already provide free search services of that data? There's search-as-improved-sales, like how Sigma-Aldrich lets people do a substructure search to find chemicals available for sale. There's value-add data. eMolecules includes data from multiple vendors, to help those who want to purchase compounds more cheaply. Or there's ZINC, which already provides search for their data. So you can see there's plenty of competition for no-cost search. I don't have the ability to add significantly new abilities that people are willing to pay for. Note also there's a non-trivial maintenance cost to keep the data sets up-to-date. Second, the queries themselves may be proprietary. I talked with one of the eMolecules people. Pharmaceutical companies will block network access to a public services to reduce the temptation of internal users to do a query using a potential $1 billion molecular structure (or potential $0 structure). eMolecules instead has NDAs with many pharmas which legal bind them. Managing these negotiations takes experience I don't have, and neither do I have the right contacts at those pharmas. Sequences don't have quite the same connection between sequence and profit as molecules do. BTW, part of the conclusion of my work is that people don't need a cluster for search - they can handle nearly all data sets on their laptop, so there shouldn't be a need to scale up any more. And small molecule data has a much smaller growth curve than sequence data, so Moore's Law is keeping up. My first customer, who continues to be a customer, said outright that they would not buy if it were under GPL. Since my paying customers are pharmaceutical companies who, as a near-rule, don't redistribute software, it doesn't really matter if they don't redistribute under MIT or don't redistribute under GPL. I came into the project in part to see if FOSS could be self-supporting on it own. AGPL is often used as a stick to try to get people to use a commercial license - the implicit view of the two-license model is that FOSS is not sustainable. Which is now my conclusion, for this project and field.

zokier 6y ago

I think to truly appreciate FOSS as a model, one needs to shift away from thinking software as an asset to be monetized to more of a liability that needs to be managed and maintained. Then the benefit of FOSS becomes clear: by publishing your software there is possibility of sharing that burden with others instead of carrying it alone yourself.

taneq 6y ago
I agree. I can't see how you can 'sell free software' as a standalone product. You build and evangelise free software while selling feature requests, services and support to the users.
- dalke 6y ago
  You can sell a proprietary product, right? With a restrictive software license? That means customers are willing to pay $base + $yearly renewal for the product. Why aren't they willing to pay the same price for the same product but with an open source license? I really don't understand why they don't. I'll go one further - how much will people pay for an open source license over a source available license with a right to modify, no time limits/renewal requirement, but no distribution right? Answer: all but one of my customers jumped at the chance to reduce the cost by switching from the MIT license to a not-quite-open-source license. Which means they don't really value the redistribution right. And I saw this at one small conference about industry use of open source. The organizers - who use chemfp! - stated at the start that the biggest reason they love open source is because it's "free" (meaning no cost), not the principles of software freedom nor the improved development methodology of open source. I tried selling feature requests, services and support to the users. That was my original plan, and it worked so long as those feature requests were easy and there were enough of them. But consider that the upgrade to Python 3 took two months. Who pays for that? The first customer who wants Python 3 support 5 years ago, who pays $20K for a feature request which everyone else gets for free? Then there's inventive to wait for a feature request in hopes that someone else will pay it. While the sales model - even as free software - lets me split the cost among multiple customers who need that feature, and across a few years. I also pointed out that selling services is a disincentive to developing good document and good APIs. I feel like there's a sweet spot where if I were to skimp on the documentation some then there's an increased chance of getting consulting work.

dekhn 6y ago

It's funny just how much the implementations described in the paper map to how modern search engines implement retrieval. The same is true for BLAST and other search engines. (it's a very readable paper and I enjoy the frank expression of view, even if I have a vastly different perspective on how to accelerate problems like this)

nkurz 6y ago
> even if I have a vastly different perspective on how to accelerate problems like this Do tell more! After the initial optimization, I did hint at an approach to Andrew that I thought could get a further large speedup. Essentially, the idea was to "rotate" all the stored data 90 degrees, so that instead of counting the features present in each compound you read lists of compounds that contain a given feature, storing the hits in some very fast custom data structure. He wasn't particularly interested, likely correctly realizing that it would be a lot of work for an uncertain amount of gain. The question wasn't really whether I could achieve further speedup (although there was question as to how much), rather (as alluded to in the paper) whether he would be able to sufficiently increase sales to justify the additional development cost and added complexity of the codebase.
- dalke 6y ago
  Towards the end of the writing the paper, another paper came out on "RISC: rapid inverted-index based search of chemical fingerprints", https://doi.org/10.1021/acs.jcim.9b00069 https://doi.org/10.1021/acs.jcim.9b00069 which does something along those lines. It was close enough that I published the pre-print "RISC and dense fingerprints" at https://doi.org/10.26434/chemrxiv.8218517.v1 https://doi.org/10.26434/chemrxiv.8218517.v1 to examine its claims. I found that their RISC implementation was faster than chemfp for low bit densities (<~5%), which includes the popular 2048-bit ECFP/Morgan fingerprints for smaller radii, and uncommonly high similarity thresholds. Otherwise, chemfp was faster. So while there's certainly something to investigate there, I think it's better to focus that effort on truly sparse fingerprints and count fingerprints, rather than nominally dense bit fingerprints. Just needs money and time. ;) Plus, part of the focus was on making chemfp a really good baseline for these sorts of timing tests.

rurban 6y ago

Im sceptical that a good single CPU search can compete with massive parallel HW, like this one: https://www.graphcore.ai/posts/introducing-second-generation-ipu-systems-for-ai-at-scale https://www.graphcore.ai/posts/introducing-second-generation...

dalke 6y ago
Sure. It can't. Even GPUs will beat a CPU. In my paper I commented: > GPU memory bandwidth is an order of magnitude higher than CPU bandwidth, so a GPU implementation of the Tanimoto search kernel should be about ten times faster. Chemfp has avoided GPU support so far because it’s not clear that the demand for similarity search justifies dedicated hardware, especially if the time to load the data into the GPU is slower than the time to search it on the CPU. GPUs are more likely to be appropriate for clustering mid-sized datasets where the fingerprints fit into GPU memory. Corporate compound sets have ~5 million records. That can be searched on a laptop in about 50ms. A large data set containing physically measured properties is ~100M records, which takes a bit over a second. The largest data sets people search, with synthetically generated compounds, is around 1G records. That requires distributed computing. But most people don't work with them. They say the best camera is the one you have with you. Most people have a CPU with them. Fewer have massive parallel HW with them.