3 ms·
How does your performance scale? If it scales linearly in records, you're going to have trouble if you're only below 200ms on 1k records. If it scales logarithm
by tomsmeding 7y ago
How does your performance scale? If it scales linearly in records, you're going to have trouble if you're only below 200ms on 1k records. If it scales logarithmically, there are still a couple orders of magnitude between a comprehensive index like Google's and your 1k record index, which is still going to give you trouble.
In short, 200ms is slow, not fast.
EDIT: Just to clarify, I'm not saying it's necessarily easy to be faster than 200ms (I'd need to look closer into fuzzy text searching algorithms to be able to make an educated guess here); I'm just saying it's not fast enough.
- arch-ninja 7y agoThanks for the insight; indexing the entire web is not a goal of dindex. Instead, it's designed to work like DNS where you can host your own server + have it federate queries to other servers if it doesn't have records which match a query. Basically it moves control to the client; if the client wants to query servers across the globe there's nothing stopping them, but by default you only query servers in your config file. Organizations like the new york times, universities, and governments would host servers for the same reason they host their own email infrastructure. It also moves a ton of control to publishers; dindex doesn't crawl the web unless a client explicitly wants to.
- arch-ninja 7y agoBut yea while performance is a huge goal of mine, at the moment I don't even cache compiled regular expressions. It's definitely an alpha-stage project.