3 ms·
To give some context, for this demo we crawled the entirety of public GitHub to show off our idea of what we mean by novel. We chose GitHub because we figured H
by Grimm1 7y ago
To give some context, for this demo we crawled the entirety of public GitHub to show off our idea of what we mean by novel. We chose GitHub because we figured HN might like to see some cooler repos for things they are interested in.
We have the capability to crawl about a hundred million sites a day and are figuring out where to go from here.
If you want to learn more about what we're doing.
https://medium.com/@iantbutler01/existing-search-engines-fail-independent-and-small-businesses-enter-whize-404958949534 https://medium.com/@iantbutler01/existing-search-engines-fai...
- kburman 7y ago> entirety of public GitHub to show off our idea Well, I tried searching for my public repo and end up with 0 results. Same with my username.
- Grimm1 7y agoIf it's newer than a few weeks ago it's not in there, we also dropped about 150k-300k results on indexing and figured it really wasn't a concern considering it's an alpha. Sorry about that, the index has about 1.4m records in it though.
- Grimm1 7y agoWorth noting too you may just not meet our idea of novel if your score is sufficiently low you are not ranked.