5 ms·

I wonder what the next search engine algorithm will be... I've been thinking about up-votes/down-votes. And I could swear Google experimented with Random rankin

by z3t4 7y ago

I wonder what the next search engine algorithm will be... I've been thinking about up-votes/down-votes. And I could swear Google experimented with Random ranking a few years back, which gave very good results, but it was a bit annoying when you forget to bookmark a site, and not finding it when searching for it again the next day. But I'm thinking random plus voting, so if you find a good site, you upvote it, if you find spam you downvote it. If you want to find that good site again, you just check your upvote history. Random sorting give small interesting web sites a chance to get found. When doing a search you can use sliders to fine-tune the order using popularity (upvote/downvote ratio), last updated, first found, region, and a checkbox to only show links you haven't already clicked on, or haven't already voted on. And there could be an option to discover sites you would probably like, based on which links you have upvoted, and what people that upvote sites simlar to you also up-voted - but you have not yet clicked on. The problem with upvote bots have already been solved by social networks like Facebook, where using like-farms, follower-farms etc are not very effective. You should not be too baised on the votes though, or you would end up with echo-chamber loops like with Spotify and Youtube. The key is randomness! A random brute force-like algorithm is often just as good and sometimes better then a sophisticated system. It would be hard to game/cheat a random algorithm.

lethologica 7y ago
All of this almost has me wishing for a human curated list of high quality sites for each topic, though that would obviously have its problems, too.
- SiVal 7y ago
  Yeah, I remember talking to a student over at our local college a few years back who had started doing that with a classmate. They called it Yet Another Hierarchical... something or other. Jerry, I think his name was. I probably should have joined them. It turned out to be quite an experience.
- jakemal 7y ago
  That's what search engines used to be in the early days of the internet.
- deleted 7y ago
  [deleted]
- mathnmusic 7y ago
  I'm building https://learnawesome.org https://learnawesome.org for this. Started by collecting various awesome lists and then adding search, reviews, related items etc. For eg, when looking up a book, you can easily discover the TED talk or podcast given by the author which has the same ideas. It's early days though. I imported around 10,000 MOOCs from various platforms and organized them by topics. The webapp is open-source so fixes/features are always welcome. :)
- squiggleblaz 7y ago
  I remember dmoz! Nowadays any curated list would need to have the properties of (a) being publicly editable and (b) being prose, because for the time being it's sufficiently difficult for machines to write prose. Or at least, I assume so, since everything popular except wikipedia has been gamed.
ghostbrainalpha 7y ago
Google already has the equivalent of upvoting without requiring anything from the users. If you click into a site and you don't return to your search, they consider that an upvote. If you continue clicking different search results they consider that a downvote for the site you visited.
- jessaustin 7y ago
  This is a really noisy signal.
- dual_basis 7y ago
  Then that's not great, based on the way I (and, from observation, many others) use search results. I often middle-click (or control+click) several links which look promising, loading them in new tabs in the background. When I have a decent number, I then go visit the sites themselves. In addition, the best Google searches result in finding the answer directly in the preview of the page from the search results. Unfortunately there's not a clear signal to Google which site preview provided the answer in this case, or even if none of them did and I just gave up searching. In reality, I'm sure Google's determination of which site was relevant is much more sophisticated, likely involving some machine learning.
  - squiggleblaz 7y ago
    Since google's spying on you, you'd think they'd be able to link the page you're reading to the search that triggered it, and filter the analytics back to the search. But google won't use their spying to our benefit...
dual_basis 7y ago
How do you stop the bots from upvoting? If you make people fill out a recaptcha every time to upvote they just won't bother.
- soulofmischief 7y ago
  An anonymous, decentralized reputation system which any service could hook into.
  - 1_player 7y ago
    Days later, the HN post "a virus ruined my virtual reputation and I'm now banned from the internet".
    - soulofmischief 7y ago
      That's the secret sauce behind decentralization. Damage from any one actor, including yourself, is limited both temporally and in scope. Each service would get a unique hash relevant only to them and giving them access to only their scope, and you would keep your private keys safe just like you would SSH keys, crypto wallet keys, or any other sensitive data. If you can't be bothered to do that, then a minor hit to your reputation is deserved from a philosophical standpoint, even if we work to make the system robust and resilient.
  - ori_b 7y ago
    Including the spammers?
    - soulofmischief 7y ago
      The idea is that it would take a long time and lots of effort to build a credible reputation. Someone just beginning to use the internet would either need to slowly build reputation in the most open spaces, or get one or more people to vouch for them through an API. I want to make it fiscally infeasible to run a mass persona farm.
- 7y ago
judge2020 7y ago
> I've been thinking about up-votes/down-votes. I don't have proof but I'm sure the amount of votes on eg. reddit (or perhaps how long it was on the front page) has an effect on search results; see the Comcast swastika[0] and Trump showing up when searching 'Idiot'[1]. 0: https://old.reddit.com/r/funny/comments/403brr/10_months_later_and_the_swastika_is_still_on_the/ https://old.reddit.com/r/funny/comments/403brr/10_months_lat... 1: https://thehill.com/policy/technology/397920-activists-manipulating-google-so-trump-photos-come-up-when-you-search-for https://thehill.com/policy/technology/397920-activists-manip...
- intarga 7y ago
  I don't think it actually has anything to do with upvotes, but is just an indirect effect since highly upvoted pages are generally more trafficked, and it's the high traffic that really matters.

by z3t4 7y ago

lethologica 7y ago

All of this almost has me wishing for a human curated list of high quality sites for each topic, though that would obviously have its problems, too.

SiVal 7y ago
Yeah, I remember talking to a student over at our local college a few years back who had started doing that with a classmate. They called it Yet Another Hierarchical... something or other. Jerry, I think his name was. I probably should have joined them. It turned out to be quite an experience.
jakemal 7y ago
That's what search engines used to be in the early days of the internet.
deleted 7y ago
[deleted]
mathnmusic 7y ago
I'm building https://learnawesome.org https://learnawesome.org for this. Started by collecting various awesome lists and then adding search, reviews, related items etc. For eg, when looking up a book, you can easily discover the TED talk or podcast given by the author which has the same ideas. It's early days though. I imported around 10,000 MOOCs from various platforms and organized them by topics. The webapp is open-source so fixes/features are always welcome. :)
squiggleblaz 7y ago
I remember dmoz! Nowadays any curated list would need to have the properties of (a) being publicly editable and (b) being prose, because for the time being it's sufficiently difficult for machines to write prose. Or at least, I assume so, since everything popular except wikipedia has been gamed.

ghostbrainalpha 7y ago

Google already has the equivalent of upvoting without requiring anything from the users. If you click into a site and you don't return to your search, they consider that an upvote. If you continue clicking different search results they consider that a downvote for the site you visited.

jessaustin 7y ago
This is a really noisy signal.
dual_basis 7y ago
Then that's not great, based on the way I (and, from observation, many others) use search results. I often middle-click (or control+click) several links which look promising, loading them in new tabs in the background. When I have a decent number, I then go visit the sites themselves. In addition, the best Google searches result in finding the answer directly in the preview of the page from the search results. Unfortunately there's not a clear signal to Google which site preview provided the answer in this case, or even if none of them did and I just gave up searching. In reality, I'm sure Google's determination of which site was relevant is much more sophisticated, likely involving some machine learning.
- squiggleblaz 7y ago
  Since google's spying on you, you'd think they'd be able to link the page you're reading to the search that triggered it, and filter the analytics back to the search. But google won't use their spying to our benefit...

dual_basis 7y ago

How do you stop the bots from upvoting? If you make people fill out a recaptcha every time to upvote they just won't bother.

soulofmischief 7y ago
An anonymous, decentralized reputation system which any service could hook into.
- 1_player 7y ago
  Days later, the HN post "a virus ruined my virtual reputation and I'm now banned from the internet".
  - soulofmischief 7y ago
    That's the secret sauce behind decentralization. Damage from any one actor, including yourself, is limited both temporally and in scope. Each service would get a unique hash relevant only to them and giving them access to only their scope, and you would keep your private keys safe just like you would SSH keys, crypto wallet keys, or any other sensitive data. If you can't be bothered to do that, then a minor hit to your reputation is deserved from a philosophical standpoint, even if we work to make the system robust and resilient.
- ori_b 7y ago
  Including the spammers?
  - soulofmischief 7y ago
    The idea is that it would take a long time and lots of effort to build a credible reputation. Someone just beginning to use the internet would either need to slowly build reputation in the most open spaces, or get one or more people to vouch for them through an API. I want to make it fiscally infeasible to run a mass persona farm.
7y ago

judge2020 7y ago

> I've been thinking about up-votes/down-votes. I don't have proof but I'm sure the amount of votes on eg. reddit (or perhaps how long it was on the front page) has an effect on search results; see the Comcast swastika[0] and Trump showing up when searching 'Idiot'[1]. 0: https://old.reddit.com/r/funny/comments/403brr/10_months_later_and_the_swastika_is_still_on_the/ https://old.reddit.com/r/funny/comments/403brr/10_months_lat... 1: https://thehill.com/policy/technology/397920-activists-manipulating-google-so-trump-photos-come-up-when-you-search-for https://thehill.com/policy/technology/397920-activists-manip...

intarga 7y ago
I don't think it actually has anything to do with upvotes, but is just an indirect effect since highly upvoted pages are generally more trafficked, and it's the high traffic that really matters.