4 ms·

You have to do a lot of advertising. Not buying ads on Facebook, but talking about the project and trying to convince people to join. Putting a project up on Gi

You have to do a lot of advertising. Not buying ads on Facebook, but talking about the project and trying to convince people to join. Putting a project up on GitHub and then sitting back and waiting for people to show up will result in exactly 0 people showing up. You'll need to identify where your target developers are, currently, and go to them. It helps to get interviewed in tech blogs, too.

marginalia_nu 4y ago
I don't think finding developers is my first obstacle, I have a bunch of people who have contacted me and shown interest in participating. The problem is the logistics of an open search project that is a web service with serious hardware requirements. I think the minimum hardware requirements is about 14 Gb of RAM, that's without any real data loaded into the system. Testing is awkward and cumbersome, and the data logistics are a real headache even on the same network as the production instance. To even run the search engine, you need a few hundred megabytes of language models, as well as a probably few gigabytes of website data to conduct meaningful testing. The production instance has a disk footprint of about half a terabyte.
- closedloop129 4y ago
  Have you asked that bunch of people how they would like to cooperate? My ignorant suggestion towards sharing those gigabytes of data: have you considered bittorrents?
- 42jd 4y ago
  I’m not sure how one could enable others to contribute easily but just making the code open source as is would be interesting. Those with a passing interest in search engines can browse the code of a capable engine and pick up some knowledge. While those who really want to help out will find a way to run it and contribute (in the process helping document and maybe making it easier). My philosophy to open source is just do my personal projects in the open even if it’s not “accessible” and if someone comes along and likes it they can contribute or fork it and modify to their liking. Or they might just browse the code and pick up something in passing.
- nijave 4y ago
  Are those hard requirements and, if so, why? It might be worthwhile to add a "local" or "dev" mode which loads a very minimal amount of data (maybe only a handful of items are indexed and only a handful of search terms that work with them).

marginalia_nu 4y ago

I don't think finding developers is my first obstacle, I have a bunch of people who have contacted me and shown interest in participating. The problem is the logistics of an open search project that is a web service with serious hardware requirements. I think the minimum hardware requirements is about 14 Gb of RAM, that's without any real data loaded into the system. Testing is awkward and cumbersome, and the data logistics are a real headache even on the same network as the production instance. To even run the search engine, you need a few hundred megabytes of language models, as well as a probably few gigabytes of website data to conduct meaningful testing. The production instance has a disk footprint of about half a terabyte.

closedloop129 4y ago
Have you asked that bunch of people how they would like to cooperate? My ignorant suggestion towards sharing those gigabytes of data: have you considered bittorrents?
42jd 4y ago
I’m not sure how one could enable others to contribute easily but just making the code open source as is would be interesting. Those with a passing interest in search engines can browse the code of a capable engine and pick up some knowledge. While those who really want to help out will find a way to run it and contribute (in the process helping document and maybe making it easier). My philosophy to open source is just do my personal projects in the open even if it’s not “accessible” and if someone comes along and likes it they can contribute or fork it and modify to their liking. Or they might just browse the code and pick up something in passing.
nijave 4y ago
Are those hard requirements and, if so, why? It might be worthwhile to add a "local" or "dev" mode which loads a very minimal amount of data (maybe only a handful of items are indexed and only a handful of search terms that work with them).