60 ms·
Interesting. I had some interests in building a search engine myself (for playing around ofcourse). I had read a blog post by Michael Nielson [1] which had spar
by DavidCole1 5y ago
Interesting. I had some interests in building a search engine myself (for playing around ofcourse). I had read a blog post by Michael Nielson [1] which had sparked my interest. Do you have any written material about your architecture and stuff like that? Would love to read up.
[1]: https://michaelnielsen.org/ddi/how-to-crawl-a-quarter-billion-webpages-in-40-hours/ https://michaelnielsen.org/ddi/how-to-crawl-a-quarter-billio...
- gbmatt 5y agothere's some stuff here : https://github.com/gigablast/open-source-search-engine https://github.com/gigablast/open-source-search-engine
- DavidCole1 5y agoThank you.
- entropie 5y agoHoly, thats a huge codebase. Github even shows no code/syntax hl for many cpp files because they are so big. I fiddled around and searched for some not so well known sites in germany and the results were surprisingly good. But it looks really... aged.
- kingcharles 5y agoHoly shit. Click on random .cpp file. Browser hangs. O_O