5 ms·

Have some amusement courtesy of https://www.mojeek.com/preferences https://www.mojeek.com/preferences: Crawl date: [x] no (default) [ ] yes [ ] no (It was

by i336_ 10y ago

Have some amusement courtesy of https://www.mojeek.com/preferences https://www.mojeek.com/preferences: Crawl date: [x] no (default) [ ] yes [ ] no (It was in amongst a bunch of other settings that also had those same 3 options) As can be seen by simply mentally timing the delay, and also by the "in 0.47s", queries seem to be being cached. That sort of admits that the cache isn't that up-to-date. Also, I tried my standard "bamboozle google" query, "ketmax", while looking for results suffixed with "35", "." and "zip". No dice. (ixquick eventually found it that one time.)

nerdponx 10y ago
What's that query supposed to return?
- i336_ 10y ago
  A ZIP file for an old DOS disassembler, which Google will readily find if passed ".zip", "35" and "ketmax" in reverse order (not strrev, reverse order) as one string with no spaces. If you leave out the "35" part of the string (and just supply the other two parts), Google has no idea what I'm talking about no matter what I try - it was ixquick that helped me find the file without that middle tidbit. Whenever I mention this bug I'm challenged with how to refer to it in a way that humans will be able to understand (without resorting to base64 or similar...) but which Google's indexer won't "get", so that the bug stays there and I'm able to point it out to people. My point is that Google's correlation system is nowhere near perfect. (I would intuitively categorize this with the much-vaunted systems that determine context, "did you mean" etc.)
i336_ 10y ago
Woops, completely forgot to complete what I was talking about the timing thing. When you first do a query, it takes 0.47s, when you redo it, it takes like 0.00 or 0.01 seconds. I have no idea how long for.
mojeek 10y ago
Hi, thanks for the feedback. I'll most likely be changing the preferences back to a more traditional yes/no soon! It was meant as a clear setting and use whatever the default is for future searches (which may change). Yes the results will be cached for up to 1 hour which is why subsequent searches didn't take as long. I'm limited to how much I can index, so ZIP files have been skipped for now.
- i336_ 10y ago
  Wow, cool to see the author checking in! :) Thanks for clarifying the purpose of the "clear" preference. There's no description underneath that option (and only that option, making it seem like an outlier that was supposed to be removed or something) so it was far too easy to assume there was a glitch somewhere, eg a template fault (that made that option get the same set of radio buttons as the three elements underneath it). Ah, the cache lasts an hour. That's only four times longer than Google's turnaround time (they aim to have data from most sites into search results pages within 15 minutes), so all things considered that's not shabby at all. (I'm not saying "only" here sarcastically) I'm curious as to the indexing limitation, which I presume directly translates to current storage capacity. This may (may) be interesting: GitLab recently asked Hacker News for advice about a systems upgrade https://news.ycombinator.com/item?id=13153031 https://news.ycombinator.com/item?id=13153031 and received many, many comments. This forum is no replacement for the existing partnerships you've forged over the last 13 years but it's possible that networking on here may be interesting. Not presuming; just putting a datapoint out there. (The main caveat is that GitLab is within the collective on here. I can't see why Mojeek couldn't be too though.) Now for a bunch of other things. I'm very curious how you're doing text analysis for the search entries, How are you figuring out the (full-resolution) result counts (like "18,342,669" - not "16.8 million") for the "from $resultcount in $time" bit? This is something Google continually gets wrong! Most my interest stem from impressedness at the fact that you're using C for all of this (which absolutely makes sense; it's cheaper to run!); my curiosity is specifically about how you're doing those things from C, how you're managing inevitable crashes, etc etc. I'm also curious what database you're using (and also the query parsing architecture) - because searching is actually pretty fast (even from Australia!). Feel free to answer whatever you like here (highly appreciated!), but what would be really cool would be a multi-page writeup explaining everything in-depth (basically everything that's non-competitive) and then posting the URL as a new article here (it might take a couple goes till it "sticks" and people see and upvote it). That would be very interesting to a lot of people on here, I think. Just a thought.

by i336_ 10y ago

nerdponx 10y ago

What's that query supposed to return?

i336_ 10y ago
A ZIP file for an old DOS disassembler, which Google will readily find if passed ".zip", "35" and "ketmax" in reverse order (not strrev, reverse order) as one string with no spaces. If you leave out the "35" part of the string (and just supply the other two parts), Google has no idea what I'm talking about no matter what I try - it was ixquick that helped me find the file without that middle tidbit. Whenever I mention this bug I'm challenged with how to refer to it in a way that humans will be able to understand (without resorting to base64 or similar...) but which Google's indexer won't "get", so that the bug stays there and I'm able to point it out to people. My point is that Google's correlation system is nowhere near perfect. (I would intuitively categorize this with the much-vaunted systems that determine context, "did you mean" etc.)

i336_ 10y ago

Woops, completely forgot to complete what I was talking about the timing thing. When you first do a query, it takes 0.47s, when you redo it, it takes like 0.00 or 0.01 seconds. I have no idea how long for.

mojeek 10y ago

Hi, thanks for the feedback. I'll most likely be changing the preferences back to a more traditional yes/no soon! It was meant as a clear setting and use whatever the default is for future searches (which may change). Yes the results will be cached for up to 1 hour which is why subsequent searches didn't take as long. I'm limited to how much I can index, so ZIP files have been skipped for now.

i336_ 10y ago
Wow, cool to see the author checking in! :) Thanks for clarifying the purpose of the "clear" preference. There's no description underneath that option (and only that option, making it seem like an outlier that was supposed to be removed or something) so it was far too easy to assume there was a glitch somewhere, eg a template fault (that made that option get the same set of radio buttons as the three elements underneath it). Ah, the cache lasts an hour. That's only four times longer than Google's turnaround time (they aim to have data from most sites into search results pages within 15 minutes), so all things considered that's not shabby at all. (I'm not saying "only" here sarcastically) I'm curious as to the indexing limitation, which I presume directly translates to current storage capacity. This may (may) be interesting: GitLab recently asked Hacker News for advice about a systems upgrade https://news.ycombinator.com/item?id=13153031 https://news.ycombinator.com/item?id=13153031 and received many, many comments. This forum is no replacement for the existing partnerships you've forged over the last 13 years but it's possible that networking on here may be interesting. Not presuming; just putting a datapoint out there. (The main caveat is that GitLab is within the collective on here. I can't see why Mojeek couldn't be too though.) Now for a bunch of other things. I'm very curious how you're doing text analysis for the search entries, How are you figuring out the (full-resolution) result counts (like "18,342,669" - not "16.8 million") for the "from $resultcount in $time" bit? This is something Google continually gets wrong! Most my interest stem from impressedness at the fact that you're using C for all of this (which absolutely makes sense; it's cheaper to run!); my curiosity is specifically about how you're doing those things from C, how you're managing inevitable crashes, etc etc. I'm also curious what database you're using (and also the query parsing architecture) - because searching is actually pretty fast (even from Australia!). Feel free to answer whatever you like here (highly appreciated!), but what would be really cool would be a multi-page writeup explaining everything in-depth (basically everything that's non-competitive) and then posting the URL as a new article here (it might take a couple goes till it "sticks" and people see and upvote it). That would be very interesting to a lot of people on here, I think. Just a thought.