3 ms·
I will suggest that you read the article again and revise your judgement. This is why. I founded a search engine (web, images, video and business search) now
by okeumeni 19y ago
I will suggest that you read the article again and revise your judgement. This is why.
I founded a search engine (web, images, video and business search) now in Alpha, with limited resources as well; trust me I know how hard it is.
Simple questions, how good is your crawler? (You shouldn’t implement a scheme for each web site, though I understand for your business it kinda make sense)
How much information do you have in your repository? (You should consider data in TB if not you are still far from any trouble)
Ferret is a great indexing tool but how much data can it index? How scalable is it?
Looking at the tech behind Ferret, how much resource does it use?
How good is your relevancy model? (This question is tightly link to your indexing)
I loved your idea,
Just one thing: work on the relevancy again; I searched for ‘ruby on rails’ and got ‘ruby’ only related results first the <relevant ones> after. Also I will suggest you cache images to enhance user experience. Please don’t take my review personal.
- Readmore 19y agoThat's good advice. I'll admit the scaling aspects of Ferret, and Rails at this point, scare me quite a bit. My goal right now is to continue to expand my index and learn what's working and what's not. As was mentioned in another comment I'm looking into other, more long term, indexing solutions (possibly SOLR), so hopefully that will help in the relevancy area. I'm also working on an in-house ranking algo to better sort results. I appreciate your feedback.