3 ms·

I think relevance is the thing to work on. I did a search in a space I know a lot about and the majority of first page results were domains for sale. The compan

I think relevance is the thing to work on. I did a search in a space I know a lot about and the majority of first page results were domains for sale. The company I have an interest in, which is the first result for certain keywords on Google and Bing, wasn't to be found anywhere in the first 4 pages on here. It would be interesting to hear a bit more about your plans for moving this forward. I have a few questions it would be great to hear about either here or in a follow-up blog post: * Is your reimplementation in JS a translation of your current from Pascal or a redevelopment? * Why have you decided to move from Pascal to JS rather than spending the effort improving the current implementation? * What lessons have you learned from your current implementation that you are attempting to overcome with your new version in JS?

deusu 12y ago
The rewrite in JavaScript will be mixture of porting and redesign. Some parts of the Pascal-sources are relatively new and can almost be translated as is. But the older the Pascal-source is, the more a redesign is a better idea. The old parts of the sources are in VERY bad shape. This project started more than 15 years ago on a much, MUCH smaller scale (about 2 million pages compared to 320 million now). Some design-choices simply aren't valid anymore. Also the use of JS has the advantage that it will make the project attractive to a lot more programmers than staying with Pascal could achieve. The current implementation keeps the search-index as one big thing. I will split that up into many smaller pieces. That way it can run multi-threaded or even on multiple servers. Currently queries are running single-threaded. Another shortcoming is that building an index involves several manual steps. I want to automate that.

3 ms·

I think relevance is the thing to work on. I did a search in a space I know a lot about and the majority of first page results were domains for sale. The compan

by concerto 12y ago

deusu 12y ago
The rewrite in JavaScript will be mixture of porting and redesign. Some parts of the Pascal-sources are relatively new and can almost be translated as is. But the older the Pascal-source is, the more a redesign is a better idea. The old parts of the sources are in VERY bad shape. This project started more than 15 years ago on a much, MUCH smaller scale (about 2 million pages compared to 320 million now). Some design-choices simply aren't valid anymore. Also the use of JS has the advantage that it will make the project attractive to a lot more programmers than staying with Pascal could achieve. The current implementation keeps the search-index as one big thing. I will split that up into many smaller pieces. That way it can run multi-threaded or even on multiple servers. Currently queries are running single-threaded. Another shortcoming is that building an index involves several manual steps. I want to automate that.