4 ms·
I started research on making of an alternative search engine. It would not index sites serving ads and possibly e-commerce. I would like to also penalize JavaSc
by hawski 9y ago
I started research on making of an alternative search engine. It would not index sites serving ads and possibly e-commerce. I would like to also penalize JavaScript use at least as an option. At the beginning I would use Adblock rulesets like the Easy List - if there is a match I do not index the site. I named it Abracabra.
I hope that this would remove most crap out there with some minor collateral damage. Also that the index would be small enough that a little fish like me could do it without massive cost or infrastructure.
Regarding JavaScript use penalization I have in mind at least lower ranking. Probably for the first version not including them at all would be the simplest thing to do. Some later version could attempt to classify used JavaScript.
I would like it to index information first and not care much about web apps. Sometimes within the information only site there could be a link to a webapp. I’m wondering if it would make sense to distribute whole index via torrent. Then search could be done locally. But for this too make sense it would have to be in an order of, at most, tens of gigabytes. The problem would be to make updates as small as possible and also to not use prohibitive amount of CPU time.
I don't have any monetization in mind as you probably should have guessed at this point. Probably if it would be frugal enough it could run from my pocket and hopefully some donations.
However I’m almost totally green in this area. I started a bit with learning how to index and search with SQLite's FTS5. I don't like dependencies too much and would like to keep the local version option available. So probably typical ElasticSearch and other Java apps are probably too heavy. You can safely ignore technical side of my comment if you know better. If someone is more capable to do this than me, please make it instead of me ;)