I've been working on a web crawler for a while. Once it's done,I'll start building a web search engine based on the crawler. It's not been in public yet. I use

I've been working on a web crawler for a while. Once it's done,I'll start building a web search engine based on the crawler. It's not been in public yet. I use C,OpenMPI,Pthread,Valgrind on Linux.