3 ms·

It's your ranking system, so it's correct by definition (for you), but I've got to point out you're not considering the quality of the text. In your model, an e

by flir 2y ago

It's your ranking system, so it's correct by definition (for you), but I've got to point out you're not considering the quality of the text. In your model, an empty page is perfect. Cynically, maybe you're on to something...

lelanthran 2y ago
> It's your ranking system, so it's correct by definition (for you), but I've got to point out you're not considering the quality of the text. Yes. This is by design![1] > In your model, an empty page is perfect. Yes. What's the problem with that? An empty page won't match any search terms, will it? [1] It's impossible to quickly and cheaply determine the quality of the text, in an age where it's cheaper for a blog-spammer to use AI to write 3k articles than for the crawler to use AI to check 3m articles.
- Folcon 2y ago
  You might be onto an interesting new view of the web here, if you've ever seen https://www.builtwith.com https://www.builtwith.com, something similar that also processes script tags and downweights based on what the script is would also add to this effect. At the very least I'd be interested in looking at such a site and seeing how it differs from https://www.millionshort.com https://www.millionshort.com =)...
- verisimi 2y ago
  I think this is a great idea. I really like the principle. But it does miss the point that you need a huge amount of resources. Plus 20 billion or so to get Apple to use your search.
  - Aachen 2y ago
    If it is actually as much better as it sounds, even apple users will find a way to set their search engine to what works once word of mouth gets around. (Until Apple turns that option off and waits for for the EU to pass another gatekeeper law anyway.) If it is actually as much better as it sounds, one wonders why Google or Duckduckgo doesn't do this already. For Google, killing the advertisement incentive is probably counterproductive in the long run, but why doesn't DDG?
- nkoren 2y ago
  Nah, you need some way to consider the quality of your text. No way around that. Let's say I publish a page -- nothing fancy to it, no dark patterns at all -- just the word "architecture". For anybody searching for the term "architecture", this is now the perfect page -- a 100% match to their search, with no dark patterns at all. Completely useless, but perfect. What would my motivation be for doing something like this? Well, imagine that I now deploy 100,000 identical or very similar pages. Your search engine is now fully _nuked_ for those terms. Does your competitor have a product that you don't want being found via the search engine? I'll happily nuke its search terms for you, for a fee of course. That's my motivation. Remember that advertising is not just a competition for you to be seen, but also for your competitors to not be seen. Your search engine, as-is, would be great at enabling the latter. (Maybe you could fix this by allowing down-votes from logged-in, IP-logged users... but with 100k pages to downvote, that's not gonna help. So maybe you take pages which have been downvoted and do similarity comparisons to pages which haven't. A simple Levenshtein distance is too crude and easily-gamed, but maybe you could feed the downvotes into a neural-network classification system... and down the rabbit hole you go.)
  - animuchan 2y ago
    Doing this only makes sense if the search engine gained some nontrivial traction, at which point it becomes feasible to hire a small number of students to manually delist obvious garbage. I guess this will hold up as long as you're not the №1 default search engine.

by flir 2y ago

lelanthran 2y ago

> It's your ranking system, so it's correct by definition (for you), but I've got to point out you're not considering the quality of the text. Yes. This is by design![1] > In your model, an empty page is perfect. Yes. What's the problem with that? An empty page won't match any search terms, will it? [1] It's impossible to quickly and cheaply determine the quality of the text, in an age where it's cheaper for a blog-spammer to use AI to write 3k articles than for the crawler to use AI to check 3m articles.

Folcon 2y ago
You might be onto an interesting new view of the web here, if you've ever seen https://www.builtwith.com https://www.builtwith.com, something similar that also processes script tags and downweights based on what the script is would also add to this effect. At the very least I'd be interested in looking at such a site and seeing how it differs from https://www.millionshort.com https://www.millionshort.com =)...
verisimi 2y ago
I think this is a great idea. I really like the principle. But it does miss the point that you need a huge amount of resources. Plus 20 billion or so to get Apple to use your search.
- Aachen 2y ago
  If it is actually as much better as it sounds, even apple users will find a way to set their search engine to what works once word of mouth gets around. (Until Apple turns that option off and waits for for the EU to pass another gatekeeper law anyway.) If it is actually as much better as it sounds, one wonders why Google or Duckduckgo doesn't do this already. For Google, killing the advertisement incentive is probably counterproductive in the long run, but why doesn't DDG?
nkoren 2y ago
Nah, you need some way to consider the quality of your text. No way around that. Let's say I publish a page -- nothing fancy to it, no dark patterns at all -- just the word "architecture". For anybody searching for the term "architecture", this is now the perfect page -- a 100% match to their search, with no dark patterns at all. Completely useless, but perfect. What would my motivation be for doing something like this? Well, imagine that I now deploy 100,000 identical or very similar pages. Your search engine is now fully _nuked_ for those terms. Does your competitor have a product that you don't want being found via the search engine? I'll happily nuke its search terms for you, for a fee of course. That's my motivation. Remember that advertising is not just a competition for you to be seen, but also for your competitors to not be seen. Your search engine, as-is, would be great at enabling the latter. (Maybe you could fix this by allowing down-votes from logged-in, IP-logged users... but with 100k pages to downvote, that's not gonna help. So maybe you take pages which have been downvoted and do similarity comparisons to pages which haven't. A simple Levenshtein distance is too crude and easily-gamed, but maybe you could feed the downvotes into a neural-network classification system... and down the rabbit hole you go.)
- animuchan 2y ago
  Doing this only makes sense if the search engine gained some nontrivial traction, at which point it becomes feasible to hire a small number of students to manually delist obvious garbage. I guess this will hold up as long as you're not the №1 default search engine.