5 ms·

Now I want to mark this site (in Google, say) with a "high quality" measure. Then I want my search results to prefer this page, and other pages marked high for

by abraae 5y ago

Now I want to mark this site (in Google, say) with a "high quality" measure. Then I want my search results to prefer this page, and other pages marked high for quality by people who [some algorithm] to this page. Then Google would be far more useful to me than it is now, when screeds of blogspam always seem to occupy the top organic results for any "how to" tech questions. In turn I would be a happier and more loyal Google customer (not a high bar).

kabouseng 5y ago
And then less reputable website will hire consultants and influencers to mark their website as high quality. And then click farms will appear in India and China to mark website's as high quality at a price. And that is why we cannot have nice stuff / nice search results...
- hansvm 5y ago
  I think what they're going for is more personalized results -- i.e., don't aggregate my preferences with those of a bunch of click farms.
  - gpm 5y ago
    Let everyone mark sites as high or low quality, and then count other peoples ranking in proportion to how well the correlate with your ranking. (A vague idea I've wanted to implement for awhile now for both search engines, and for HN/reddit like sites... but the amount of effort to do it well would be really high)
    - abraae 5y ago
      I would guess that, apart from the immense effort of building it, delivering personalised search results like this would be enormously expensive in storage for the search engine. Much more than sorting people into a few cohorts/buckets. But FFS, it's 2021, we deserve some decent search engine results. I doubt Google would do it unless they absolutely had to, so I hope you or someone else forces their hand and shows them that it's time.
      - hansvm 5y ago
        > I would guess that, apart from the immense effort of building it, delivering personalised search results like this would be enormously expensive in storage for the search engine. How expensive would it really be? You have O(the_internet) in pages and metadata, and you have O(world_population) in user preferences. So long as your index structure allows those to be mostly decoupled (if I had to take a first crack at it I'd probably try to embed preferences and pages into a vector space and build a projection index -- exact matches are hard in that system, but decent personalized results are easy), I don't think it'd be all that much more space than a non-personalized search engine, especially given that the world population is kind of small compared to the size of the internet. For that matter, the web isn't thaaat big (ignoring images and video). The entire common crawl can fit on a single $3k-$5k disk uncompressed.
- abraae 5y ago
  It's not about who else marked this as high quality. I want my signal to count (disproportionally).
  - hamstergene 5y ago
    You're contradicting your first comment: > I want my search results to prefer [.....] other pages marked high for quality by people who [some algorithm] to this page. Facebook/YouTube already do this and it is a disaster, which maybe even has indirectly contributed to rise of antivaxx, Q, Trump election, Capitol attack etc. Before personalized recommendation systems, when an antivaxxer made a video, hardly anyone saw it, when they liked it, the like resulted in nothing. When their likes started affecting their personal search results, liking one antivaxx video made search produce hundreds more of them. It finds so much of the same stuff they no longer have free time to check an opposing opinion. You're operating under an assumption that your judgement is more correct or better for you, and the entire problem is about finding gems in an ocean of trash. The reality is more complicated, for you antivaxx content is trash, for them it is your content that is trash.
    - true_religion 5y ago
      I see what you mean, but like the OP, I would rather have google search preference resources that I like over resources that everyone in the internet. Like if I think Wikipedia is a better resource than crunch base, then let me make that decision. I don’t care how good their SEO is.
    - abraae 5y ago
      This is not really whataboutism, with two sides. And in any case, I don't care. I know what I feel is good content and it's stuff like this, and I want more of it, and less blogspam. So yes, my assumption is that my judgement is more correct for me - because it is.
      - zikzak 5y ago
        I will subscribe to a site I like in an rss reader. Then you can search your feeds. If that fails, you can broaden your search (e.g. Google).
- MaxBarraclough 5y ago
  A web-of-trust approach might help with that. Select your own trusted reviewers, and unsubscribe from any that start recommending garbage.
  - abraae 5y ago
    Something like this would feel like a small first step, maybe the other way around though. I'd like to put a timeout on any blogspam web sites I come across, and not have them show up in my search results again for one year. By that time they might have switched to having decent content, or maybe just gone down the toilet anyway or been pushed down by Google's algorithms. Or if not I could ban hammer them again.
jaredsohn 5y ago
Maybe you want to set up a programmable search engine? https://support.google.com/programmable-search/answer/4513882?hl=en https://support.google.com/programmable-search/answer/451388... Haven't tried it myself; but remember using a predecessor to this maybe 15 years ago. "You may want to augment your results with general Web Search results. This includes results from anywhere on the web, but places emphasis on your personalized results"
- abraae 5y ago
  That's interesting, but it lost me at " all you have to do is choose which sites to search".
  - jaredsohn 5y ago
    I was hoping you would just need to add urls to it. I'm guessing the right way to handle this problem is to search your browser history or bookmarks but those are additional searches.
nl 5y ago
You know about https://www.google.com/save https://www.google.com/save aka https://www.google.com/collections https://www.google.com/collections right? Some details: https://support.google.com/websearch/answer/9217379?co=GENIE.Platform%3DDesktop&hl=en&oco=0 https://support.google.com/websearch/answer/9217379?co=GENIE...
ultramundane828 5y ago
This instinct is so right, but the multitude of comments show how hard of a problem this is. I'd be so elated to work on something that solves this but I think the answer may lie in self-regulation. We software people are generally mini-philosophers and it can be easy to lose sight of ourselves. But can you or I solve Search? I guess what I've learned before replying is that it is often a high bar whether we recognize it as such.
mwest 5y ago
I'd love something like this. I've managed to get partway there by using ArchiveBox and feeding it my bookmarks, and then using the sonic to search through them. Would be great to have this more streamlined into a normal Google/DuckDuckGo/whatever search workflow, with an option to also search your friends' bookmarks.

by abraae 5y ago

kabouseng 5y ago

And then less reputable website will hire consultants and influencers to mark their website as high quality. And then click farms will appear in India and China to mark website's as high quality at a price. And that is why we cannot have nice stuff / nice search results...

hansvm 5y ago
I think what they're going for is more personalized results -- i.e., don't aggregate my preferences with those of a bunch of click farms.
- gpm 5y ago
  Let everyone mark sites as high or low quality, and then count other peoples ranking in proportion to how well the correlate with your ranking. (A vague idea I've wanted to implement for awhile now for both search engines, and for HN/reddit like sites... but the amount of effort to do it well would be really high)
  - abraae 5y ago
    I would guess that, apart from the immense effort of building it, delivering personalised search results like this would be enormously expensive in storage for the search engine. Much more than sorting people into a few cohorts/buckets. But FFS, it's 2021, we deserve some decent search engine results. I doubt Google would do it unless they absolutely had to, so I hope you or someone else forces their hand and shows them that it's time.
    - hansvm 5y ago
      > I would guess that, apart from the immense effort of building it, delivering personalised search results like this would be enormously expensive in storage for the search engine. How expensive would it really be? You have O(the_internet) in pages and metadata, and you have O(world_population) in user preferences. So long as your index structure allows those to be mostly decoupled (if I had to take a first crack at it I'd probably try to embed preferences and pages into a vector space and build a projection index -- exact matches are hard in that system, but decent personalized results are easy), I don't think it'd be all that much more space than a non-personalized search engine, especially given that the world population is kind of small compared to the size of the internet. For that matter, the web isn't thaaat big (ignoring images and video). The entire common crawl can fit on a single $3k-$5k disk uncompressed.
abraae 5y ago
It's not about who else marked this as high quality. I want my signal to count (disproportionally).
- hamstergene 5y ago
  You're contradicting your first comment: > I want my search results to prefer [.....] other pages marked high for quality by people who [some algorithm] to this page. Facebook/YouTube already do this and it is a disaster, which maybe even has indirectly contributed to rise of antivaxx, Q, Trump election, Capitol attack etc. Before personalized recommendation systems, when an antivaxxer made a video, hardly anyone saw it, when they liked it, the like resulted in nothing. When their likes started affecting their personal search results, liking one antivaxx video made search produce hundreds more of them. It finds so much of the same stuff they no longer have free time to check an opposing opinion. You're operating under an assumption that your judgement is more correct or better for you, and the entire problem is about finding gems in an ocean of trash. The reality is more complicated, for you antivaxx content is trash, for them it is your content that is trash.
  - true_religion 5y ago
    I see what you mean, but like the OP, I would rather have google search preference resources that I like over resources that everyone in the internet. Like if I think Wikipedia is a better resource than crunch base, then let me make that decision. I don’t care how good their SEO is.
  - abraae 5y ago
    This is not really whataboutism, with two sides. And in any case, I don't care. I know what I feel is good content and it's stuff like this, and I want more of it, and less blogspam. So yes, my assumption is that my judgement is more correct for me - because it is.
    - zikzak 5y ago
      I will subscribe to a site I like in an rss reader. Then you can search your feeds. If that fails, you can broaden your search (e.g. Google).
MaxBarraclough 5y ago
A web-of-trust approach might help with that. Select your own trusted reviewers, and unsubscribe from any that start recommending garbage.
- abraae 5y ago
  Something like this would feel like a small first step, maybe the other way around though. I'd like to put a timeout on any blogspam web sites I come across, and not have them show up in my search results again for one year. By that time they might have switched to having decent content, or maybe just gone down the toilet anyway or been pushed down by Google's algorithms. Or if not I could ban hammer them again.

jaredsohn 5y ago

Maybe you want to set up a programmable search engine? https://support.google.com/programmable-search/answer/4513882?hl=en https://support.google.com/programmable-search/answer/451388... Haven't tried it myself; but remember using a predecessor to this maybe 15 years ago. "You may want to augment your results with general Web Search results. This includes results from anywhere on the web, but places emphasis on your personalized results"

abraae 5y ago
That's interesting, but it lost me at " all you have to do is choose which sites to search".
- jaredsohn 5y ago
  I was hoping you would just need to add urls to it. I'm guessing the right way to handle this problem is to search your browser history or bookmarks but those are additional searches.

nl 5y ago

You know about https://www.google.com/save https://www.google.com/save aka https://www.google.com/collections https://www.google.com/collections right? Some details: https://support.google.com/websearch/answer/9217379?co=GENIE.Platform%3DDesktop&hl=en&oco=0 https://support.google.com/websearch/answer/9217379?co=GENIE...

ultramundane828 5y ago

This instinct is so right, but the multitude of comments show how hard of a problem this is. I'd be so elated to work on something that solves this but I think the answer may lie in self-regulation. We software people are generally mini-philosophers and it can be easy to lose sight of ourselves. But can you or I solve Search? I guess what I've learned before replying is that it is often a high bar whether we recognize it as such.

mwest 5y ago

I'd love something like this. I've managed to get partway there by using ArchiveBox and feeding it my bookmarks, and then using the sonic to search through them. Would be great to have this more streamlined into a normal Google/DuckDuckGo/whatever search workflow, with an option to also search your friends' bookmarks.