4 ms·

You always need to balance Precision and Recall. For something like this, you want to be exhaustive. I've worked on search engines, and depending on who is usi

You always need to balance Precision and Recall. For something like this, you want to be exhaustive. I've worked on search engines, and depending on who is using them, that balance gets struck differently on the ROC curve. For legal matters, for example, they want every record that might match. For ad-hoc (google style) queries, nobody reads the second page so you care more about Precision @ 20 (or really, at 3)

BlueTemplar 4y ago
Somewhat offtopic, I'm curious about this "nobody reads the 2nd page" meme... I find myself reading pages 2-5 quite often, because page 1 just didn't give enough results, and I doubt I'm that much in a minority ? (I'm talking about actual generalist searches, not people that use a global search engine as a replacement to bookmarks or directly searching, for instance, Wikipedia.)
- mjevans 4y ago
  I imagine that each additional page presentation is exponentially less likely to be reached and considered a 'reputable' result by someone normal. If I'm looking for a specific issue I'll sometimes try out things as deep as 10 or more pages of search results if nothing on the first 100 ish hits selects the issue, but then only if I can't think of any keyword variations to use that might get me a better result match. I don't expect the average user to go even remotely that far.
- daanlo 4y ago
  Source: https://www.sistrix.com/blog/why-almost-everything-you-knew-about-google-ctr-is-no-longer-valid/ https://www.sistrix.com/blog/why-almost-everything-you-knew-... TLDR: on the second results page, each result gets <1% click through rate. So it‘s not nobody, but statistically speaking not very many.
  - kqr 4y ago
    That's not how statistics work. I consider myself a frequent visitor of the second results page, but even for me the CTR of the results on the second page are < 1 %, because I almost always find the thing I want on the first page. The question ought to be "conditional on not finding the result on the first page, how likely is the user to go to the second page versus balk, or re-try a different query?" I'm fairly confident that number is higher than 1 %, but I don't have the data.
    - MonkeyMalarky 4y ago
      Or alternatively, a number "significantly less than 1%" is still massive when multiplied by the number of searches google gets.
  - BlueTemplar 4y ago
    But how much is this affected by my caveat inside the parentheses ?
- dboreham 4y ago
  I always read the 2nd page because the 1st is all SEO crap.
- smeagull 4y ago
  There are types of queries that people discount. Navigational queries for instance, that essentially use Google as a spellcheck + DNS. And many queries that are solved with 0 visits, thanks to the information being pulled out from the wiki, or giving up and rephrasing the query. After those, then yes, most queries will be answered by the first page. It's what retrieval is optimized for.
c7b 4y ago
> You always need to balance Precision and Recall. For something like this, you want to be exhaustive. Respectfully disagree. You're right in principle if you build a tool like that for yourself. But since this is Open Source, you have to take into account that people who don't understand that will use the tool as well and then use that as "evidence" in whatever arguments they're having with someone.
- jhugo 4y ago
  Hopefully they read one sentence about what the tool does before using it. If they don't, their arguments will be weak due to their own sloppiness in research, which doesn't seem like a new problem.
  - c7b 4y ago
    the thing is, people who want to use this for defamation don't have an incentive to sanitize the data - the message might stick around regardless of whether it has any merit (we've seen this happening even at the most publicly scrutinized national level, imagine how much fewer defenses smaller players have)
    - voakbasda 4y ago
      Defamation is definitely one problem with this tool. Once there is “evidence”, it can be (ab)used without consequence by someone, because they can simply point at the tool output. If the output is usually “good enough”, it will provide an alibi for their actions: “I didn’t make a mistake, the computer did.”
      - jhugo 4y ago
        Nobody who bothers to check what the tool does will be swayed by this. If someone wants to make shit up, and their audience will not bother to spend 30 seconds looking at their "sources", they could just write a script that outputs whatever they wanted and then point at that script.

BlueTemplar 4y ago

Somewhat offtopic, I'm curious about this "nobody reads the 2nd page" meme... I find myself reading pages 2-5 quite often, because page 1 just didn't give enough results, and I doubt I'm that much in a minority ? (I'm talking about actual generalist searches, not people that use a global search engine as a replacement to bookmarks or directly searching, for instance, Wikipedia.)

mjevans 4y ago
I imagine that each additional page presentation is exponentially less likely to be reached and considered a 'reputable' result by someone normal. If I'm looking for a specific issue I'll sometimes try out things as deep as 10 or more pages of search results if nothing on the first 100 ish hits selects the issue, but then only if I can't think of any keyword variations to use that might get me a better result match. I don't expect the average user to go even remotely that far.
daanlo 4y ago
Source: https://www.sistrix.com/blog/why-almost-everything-you-knew-about-google-ctr-is-no-longer-valid/ https://www.sistrix.com/blog/why-almost-everything-you-knew-... TLDR: on the second results page, each result gets <1% click through rate. So it‘s not nobody, but statistically speaking not very many.
- kqr 4y ago
  That's not how statistics work. I consider myself a frequent visitor of the second results page, but even for me the CTR of the results on the second page are < 1 %, because I almost always find the thing I want on the first page. The question ought to be "conditional on not finding the result on the first page, how likely is the user to go to the second page versus balk, or re-try a different query?" I'm fairly confident that number is higher than 1 %, but I don't have the data.
  - MonkeyMalarky 4y ago
    Or alternatively, a number "significantly less than 1%" is still massive when multiplied by the number of searches google gets.
- BlueTemplar 4y ago
  But how much is this affected by my caveat inside the parentheses ?
dboreham 4y ago
I always read the 2nd page because the 1st is all SEO crap.
smeagull 4y ago
There are types of queries that people discount. Navigational queries for instance, that essentially use Google as a spellcheck + DNS. And many queries that are solved with 0 visits, thanks to the information being pulled out from the wiki, or giving up and rephrasing the query. After those, then yes, most queries will be answered by the first page. It's what retrieval is optimized for.

c7b 4y ago

> You always need to balance Precision and Recall. For something like this, you want to be exhaustive. Respectfully disagree. You're right in principle if you build a tool like that for yourself. But since this is Open Source, you have to take into account that people who don't understand that will use the tool as well and then use that as "evidence" in whatever arguments they're having with someone.

jhugo 4y ago
Hopefully they read one sentence about what the tool does before using it. If they don't, their arguments will be weak due to their own sloppiness in research, which doesn't seem like a new problem.
- c7b 4y ago
  the thing is, people who want to use this for defamation don't have an incentive to sanitize the data - the message might stick around regardless of whether it has any merit (we've seen this happening even at the most publicly scrutinized national level, imagine how much fewer defenses smaller players have)
  - voakbasda 4y ago
    Defamation is definitely one problem with this tool. Once there is “evidence”, it can be (ab)used without consequence by someone, because they can simply point at the tool output. If the output is usually “good enough”, it will provide an alibi for their actions: “I didn’t make a mistake, the computer did.”
    - jhugo 4y ago
      Nobody who bothers to check what the tool does will be swayed by this. If someone wants to make shit up, and their audience will not bother to spend 30 seconds looking at their "sources", they could just write a script that outputs whatever they wanted and then point at that script.