6 ms·

The problem isn't that Google doesn't employ these people or invest in their activities. It's that Google has destroyed their own search results in order to co

The problem isn't that Google doesn't employ these people or invest in their activities. It's that Google has destroyed their own search results in order to continue to expand their revenue opportunities. If Google: - Enabled downvoting on results, like YT videos. (Has its own spam problems, just like YT) - Allowed you to block certain domains from your search results, like YT videos. (If they added some kind of "coordinated network detection" and down-ranked domains coordinating with ones you've blocked, that'd be pretty cool). - Allowed you to create your own custom search engines, like "Programmable Search Engine". That would be incredibly valuable. They already have most of the tech. They could even create a subscription service around custom search engines if they really wanted. Plenty of people would find something like that incredibly valuable. Anyhow, buried in there is your startup idea. Remember: your startup doesn't have to generate the same revenue or profit as the incumbent on day one to be successful.

ErikVandeWater 5y ago
> Enabled downvoting on results, like YT videos. (Has its own spam problems, just like YT) Are there any search engines that do this? It's a great, simple idea.
- mad182 5y ago
  Not really that simple, I see a lot of potential for abuse - using bots and brigading to mass downvote your competitors or political opponents. Couple of positions up or down in google results for somewhat popular and valuable keywords can mean the difference in thousands of dollars per day of ad or affiliate revenue. I suspect it would get pretty wild if google launched something like this. There already are black-hat seo methods and services, but something so simple and direct would turn it up to 11.
  - numbsafari 5y ago
    > I see a lot of potential for abuse They already have the tech to fight this on YT. They, in theory, are supposed to be doing the same thing to detect inauthentic behavior on ad placement and click abuse.
  - ErikVandeWater 5y ago
    Downvotes could apply just to future recommendations for search results you see, and not apply to advertisements.
endisneigh 5y ago
> That would be incredibly valuable. They already have most of the tech. They could even create a subscription service around custom search engines if they really wanted. Plenty of people would find something like that incredibly valuable. Why would they do this? Google's customers are the advertisers, not the end-users. And no one is going to pay for a search engine, it's been tried and has failed.
- wakiza33 5y ago
  there is a smaller niche market. SEMrush is a tool used in the digital marketing industry that is now public and has a multi-billion dollar market cap. It originally started as a search engine. When they didnt gain traction they used the tech to monitor Google and interface it for customers who are tracking their performance in search results (and much more).
  - yawaworht1978 5y ago
    Can I ask what you mean by public? It's not open source as far I know and there's only the free trial way to try it.
    - 555watch 5y ago
      Probably that they completed an IPO (initial _public_ offering)
- lumost 5y ago
  You only need ~1/10000th of Google's revenue to be a financially successful startup. 1/1000th and you'll have a great business, and at 1/100th you'll be somewhere between a unicorn and a decacorn.
  - endisneigh 5y ago
    sure, but you'd need a better search engine if people are going to pay for it. A company with an objectively superior search engine could make even more money with ads so now you’re back to the beginning
edrxty 5y ago
The problem with all of this is it would help us greatly, but it would be useless to the 99% that the internet is increasingly being designed for. Modern UI trends are becoming obsessed with removing as many options and features as possible so the dumbest humans bordering on smartest vegetables can still use the service.
- saalweachter 5y ago
  And customization breaks caching.
  - mrkramer 5y ago
    It does not if there are common interests and characteristics among users. Let's say for example I'm a young African-American girl who wants to learn how to code and I query "how can African-American girl learn coding?" and Google shows me Black Girls Code a non-profit organization that focuses on providing technology education for African-American girls. Considering that Google knows that I'm African-American girl and that I want to learn coding, how many other African-American girls want to learn coding? Probably many so caching doesn't break customization and personalization as long as Google knows my characteristics and interests and characteristics and interests of other people that are similar to mine.
  - oceanplexian 5y ago
    It doesn’t have to actually, there are some pretty advanced caching mechanisms that allow you to combine cached elements together. For the web at least, you could do this back in the day with Server Side Include, and a place I worked at used it to cache logged in content.
fragmede 5y ago
How do you fight brigading, the organization of groups elsewhere to collectively vote on something? Eg white supremacist groups get together and vote down everything by people of color, and vote up their pages about how great they are?
- kleer001 5y ago
  easy, voting blocs, you assign yourself to the results of people who vote similarly to you. additionally there'd be local and regional blocs too. I can't think of a reason that the naive everyone sees everything everyone else is doing would work in the long run. That's Twitter, and it's garbage.
- metabagel 5y ago
  This is a great point. I would think Google could rank users from low quality to high quality in terms of the quality of the websites which they recommend or downvote. Tricky business and could be difficult to control, but basically the same thing they currently do for websites, but extended to humans.
- giantrobot 5y ago
  Randomly select votes that are actually recorded. Then add in metavoting that votes on the votes with random sampling. At Google's scale with a sufficiently random sampling you'd be extremely hard pressed to successfully brigade or spam the voting. Google could easily use its current fingerprinting to constrain (to an extent) multiple votes. Even knowing only a portion of the population will participate in the voting they can use a Wilson confidence interval[0] or similar to properly weight votes. Random sampling works here since you're not guaranteed one vote per user per page and the outcome in binomial, seen and downvoted or seen and not downvoted. [0] https://www.mikulskibartosz.name/wilson-score-in-python-example/ https://www.mikulskibartosz.name/wilson-score-in-python-exam...
- numbsafari 5y ago
  How does Google already handle this exact problem on YT?
  - 5y ago
bogwog 5y ago
Those shitty SEO spam sites exist only to serve ads, and Google has a monopoly on internet ads. So there is no real incentive for them to solve the problem.
- deleted 5y ago
  [deleted]
- kevinmchugh 5y ago
  Google had the same incentives in 2011, 2012 when they built and released Panda and Penguin.
- mminer237 5y ago
  Real review sites serve ads too. I don't think Google has any incentive to make things worse, and they still want people to google reviews instead of just asking friends or people on reddit for reviews.
- freyr 5y ago
  Google has 28.9% share, Facebook 25.2%, and Amazon has 10% and growing fast. Not a monopoly, and the incentive is there: if search results are consistently bad, people will stop searching as much, and revenue and market share decline.
Rastonbury 5y ago
The biggest perverse incentive for Google is that making better search results can mean less clicks to ads (clicking an ad because results are crap, going thru more pages of results means more ads). Clicks are revenue which is much easier to optimise for. Internal Search owners can push for better algos, but what if the algo causes revenue to fall? Are there strong forces strong enough within the organisation to ensure that search quality prevails? If this is the case, the problem is existential. It can only be arrested at the very top https://en.wikipedia.org/wiki/Perverse_incentive https://en.wikipedia.org/wiki/Perverse_incentive
- yabones 5y ago
  If I had to pay $10/month for good search results, I absolutely would. I think most people would. Get rid of the ads and spam, and you have a service worth a premium. The solution is to make it user-centric instead of advertiser(spammer)-centric.
  - visarga 5y ago
    Some kind of browser or extension that re-ranks and filters search results on the web.
- nitrogen 5y ago
  The biggest perverse incentive for Google is that making better search results can mean less clicks to ads This gets close to the real root of the issue -- attention is monetizable independently of the quality of content. There would be much less incentive to create SEO spam if search engines negatively weighted pages with ads and affiliate links, and if manufacturers were barred (e.g. by the FTC) from owning or imitating reviewers.
- Closi 5y ago
  > The biggest perverse incentive for Google is that making better search results can mean less clicks to ads This is also something that Google can control if competitors come along. i.e. If a reasonable competitor comes along that is willing to sacrifice ad revenue for better search result quality than Google, google can just adjust their search quality upwards to knock them out (and then adjust it back once the competitive threat is gone). Perverse incentives from Google are all over the place - Searching for the delivery business "Just Eat" in the UK for instance returns an ad for their competitor Deliveroo above the legitimate organic search result for me - and I can also see that JustEat are trying to pay for their own brand name just to compete - and IMO this sort of behaviour is anti-competitive, borderline extortion considering Google is the de-facto way of searching for a business, and shocking from a search-quality perspective (where the wrong result is intentionally shown at the top because they paid more money).
PaulHoule 5y ago
Some kinds of "spam" can improve search results. Things have changed in the past few years, now that Google has developed advanced transformer models, but for a long time Google's question answering facility has been: "let spammers make 10^8 pages where the title is the question and the answer is in the page". The trouble is that there's a fine line between "answer is in the page" and "word salad!"
willhinsa 5y ago
> - Enabled downvoting on results, like YT videos. (Has its own spam problems, just like YT) They're going the OPPOSITE DIRECTION from this!! They recently removed all downvote visibility of YouTube videos from the user, so now downvotes only feed into their algorithm. So in the last line of defense of me ending up watching a shitty video, one of the most valuable tools has been removed by my betters. It's preposterous that people think that Google is doing a good job. They're actively getting worse, and ignoring everyone saying so.
- asiachick 5y ago
  They're doing a great job. I'm so happy dislike visibility have been removed. It removes the effectiveness of pile on coordinated harassment which many youtubers have fallen victim too.
  - slig 5y ago
    Except the ratio is visible for the video creator, so they know nonetheless. And there's the possibility of disabling the the like/dislike for each video. All the big channels I follow are mad at this change, and there's a coordinated effort to bring it back.
PaulHoule 5y ago
The custom search engine is harder than you'd think. Google's search algorithm is tuned up for searching the whole web. It turns out the heuristics you need are very different depending on the size of the collection. When Gerard Salton was doing IR experiments with punched cards he was working with collections of as little as 70 documents and in that case you are going to be very concerned about recall and not precision. Maybe there is 1 relevant document and if you miss it you failed. If you had 70 billion documents you might have 10,000 relevant documents and if you lost 60% of them you still have 4,000 documents. The end user gets more results than they can sift through. Thus I always groan when I see a site is using "Google Site Search" because the relevance is usually worse than you'd get with the alternatives. Connected with that is the tuning work: Google has sufficient data to tune up a big model for everybody but true personalized search eludes them because they don't have enough data from you to tune up a model for you.
- mrkramer 5y ago
  I agree with you that "true personalized search eludes them because they don't have enough data from you to tune up a model for you". That's what Larry Page said as well "Google doesn't know what you know". His ultimate goal is Answer Machine powered by AI but that's not happening anytime soon. I think internet search engines that we are using today are primitive compared to what we will have in the future.
propogandist 5y ago
>Enabled downvoting on results, like YT videos. you mean the dislike counter they just disabled to force people to sit through more low quality content and pre-roll ads to claim increase in platform engagement and viewership? The only thing matters is revenue and Google had increases in acquisition costs in prior revenue reports. Expect to see the data points for the latter metrics to be highlighted on the earnings announcement, and a record quarter for YT coming out of the change.
asiachick 5y ago
> - Enabled downvoting on results, like YT videos. (Has its own spam problems, just like YT) Not convinced this would help. The spammers would just hire people to dislike competitors > - Allowed you to block certain domains from your search results This I would use. Never show me results form collider, watchmojo, ranker, > - Allowed you to create your own custom search engines, like "Programmable Search Engine". I think this would lead to people writing highly polarized engines. The Red Pill engine for example and we'd have a new problem, the proliferation of popular highly biased results. Of course that's not to say Google's results aren't already biased but they certainly are trying to cover everyone.
ancarda 5y ago
> Allowed you to block certain domains from your search results I would love for Google to build this in. Until they do, there is a WebExtension that does this: https://addons.mozilla.org/en-US/firefox/addon/hohser/ https://addons.mozilla.org/en-US/firefox/addon/hohser/ ("Block or Highlight Search Engine Results"). I use it to block stuff like W3Schools so when I search for something, MDN is always #1. Saves me a lot of time having to add "MDN" to the end of every query.
us0r 5y ago
> Allowed you to block certain domains from your search results Blocking pintrest would be a dream come true.

ErikVandeWater 5y ago

> Enabled downvoting on results, like YT videos. (Has its own spam problems, just like YT) Are there any search engines that do this? It's a great, simple idea.

mad182 5y ago
Not really that simple, I see a lot of potential for abuse - using bots and brigading to mass downvote your competitors or political opponents. Couple of positions up or down in google results for somewhat popular and valuable keywords can mean the difference in thousands of dollars per day of ad or affiliate revenue. I suspect it would get pretty wild if google launched something like this. There already are black-hat seo methods and services, but something so simple and direct would turn it up to 11.
- numbsafari 5y ago
  > I see a lot of potential for abuse They already have the tech to fight this on YT. They, in theory, are supposed to be doing the same thing to detect inauthentic behavior on ad placement and click abuse.
- ErikVandeWater 5y ago
  Downvotes could apply just to future recommendations for search results you see, and not apply to advertisements.

endisneigh 5y ago

> That would be incredibly valuable. They already have most of the tech. They could even create a subscription service around custom search engines if they really wanted. Plenty of people would find something like that incredibly valuable. Why would they do this? Google's customers are the advertisers, not the end-users. And no one is going to pay for a search engine, it's been tried and has failed.

wakiza33 5y ago
there is a smaller niche market. SEMrush is a tool used in the digital marketing industry that is now public and has a multi-billion dollar market cap. It originally started as a search engine. When they didnt gain traction they used the tech to monitor Google and interface it for customers who are tracking their performance in search results (and much more).
- yawaworht1978 5y ago
  Can I ask what you mean by public? It's not open source as far I know and there's only the free trial way to try it.
  - 555watch 5y ago
    Probably that they completed an IPO (initial _public_ offering)
lumost 5y ago
You only need ~1/10000th of Google's revenue to be a financially successful startup. 1/1000th and you'll have a great business, and at 1/100th you'll be somewhere between a unicorn and a decacorn.
- endisneigh 5y ago
  sure, but you'd need a better search engine if people are going to pay for it. A company with an objectively superior search engine could make even more money with ads so now you’re back to the beginning

edrxty 5y ago

The problem with all of this is it would help us greatly, but it would be useless to the 99% that the internet is increasingly being designed for. Modern UI trends are becoming obsessed with removing as many options and features as possible so the dumbest humans bordering on smartest vegetables can still use the service.

saalweachter 5y ago
And customization breaks caching.
- mrkramer 5y ago
  It does not if there are common interests and characteristics among users. Let's say for example I'm a young African-American girl who wants to learn how to code and I query "how can African-American girl learn coding?" and Google shows me Black Girls Code a non-profit organization that focuses on providing technology education for African-American girls. Considering that Google knows that I'm African-American girl and that I want to learn coding, how many other African-American girls want to learn coding? Probably many so caching doesn't break customization and personalization as long as Google knows my characteristics and interests and characteristics and interests of other people that are similar to mine.
- oceanplexian 5y ago
  It doesn’t have to actually, there are some pretty advanced caching mechanisms that allow you to combine cached elements together. For the web at least, you could do this back in the day with Server Side Include, and a place I worked at used it to cache logged in content.

fragmede 5y ago

How do you fight brigading, the organization of groups elsewhere to collectively vote on something? Eg white supremacist groups get together and vote down everything by people of color, and vote up their pages about how great they are?

kleer001 5y ago
easy, voting blocs, you assign yourself to the results of people who vote similarly to you. additionally there'd be local and regional blocs too. I can't think of a reason that the naive everyone sees everything everyone else is doing would work in the long run. That's Twitter, and it's garbage.
metabagel 5y ago
This is a great point. I would think Google could rank users from low quality to high quality in terms of the quality of the websites which they recommend or downvote. Tricky business and could be difficult to control, but basically the same thing they currently do for websites, but extended to humans.
giantrobot 5y ago
Randomly select votes that are actually recorded. Then add in metavoting that votes on the votes with random sampling. At Google's scale with a sufficiently random sampling you'd be extremely hard pressed to successfully brigade or spam the voting. Google could easily use its current fingerprinting to constrain (to an extent) multiple votes. Even knowing only a portion of the population will participate in the voting they can use a Wilson confidence interval[0] or similar to properly weight votes. Random sampling works here since you're not guaranteed one vote per user per page and the outcome in binomial, seen and downvoted or seen and not downvoted. [0] https://www.mikulskibartosz.name/wilson-score-in-python-example/ https://www.mikulskibartosz.name/wilson-score-in-python-exam...
numbsafari 5y ago
How does Google already handle this exact problem on YT?
- 5y ago

bogwog 5y ago

Those shitty SEO spam sites exist only to serve ads, and Google has a monopoly on internet ads. So there is no real incentive for them to solve the problem.

deleted 5y ago
[deleted]
kevinmchugh 5y ago
Google had the same incentives in 2011, 2012 when they built and released Panda and Penguin.
mminer237 5y ago
Real review sites serve ads too. I don't think Google has any incentive to make things worse, and they still want people to google reviews instead of just asking friends or people on reddit for reviews.
freyr 5y ago
Google has 28.9% share, Facebook 25.2%, and Amazon has 10% and growing fast. Not a monopoly, and the incentive is there: if search results are consistently bad, people will stop searching as much, and revenue and market share decline.

Rastonbury 5y ago

The biggest perverse incentive for Google is that making better search results can mean less clicks to ads (clicking an ad because results are crap, going thru more pages of results means more ads). Clicks are revenue which is much easier to optimise for. Internal Search owners can push for better algos, but what if the algo causes revenue to fall? Are there strong forces strong enough within the organisation to ensure that search quality prevails? If this is the case, the problem is existential. It can only be arrested at the very top https://en.wikipedia.org/wiki/Perverse_incentive https://en.wikipedia.org/wiki/Perverse_incentive

yabones 5y ago
If I had to pay $10/month for good search results, I absolutely would. I think most people would. Get rid of the ads and spam, and you have a service worth a premium. The solution is to make it user-centric instead of advertiser(spammer)-centric.
- visarga 5y ago
  Some kind of browser or extension that re-ranks and filters search results on the web.
nitrogen 5y ago
The biggest perverse incentive for Google is that making better search results can mean less clicks to ads This gets close to the real root of the issue -- attention is monetizable independently of the quality of content. There would be much less incentive to create SEO spam if search engines negatively weighted pages with ads and affiliate links, and if manufacturers were barred (e.g. by the FTC) from owning or imitating reviewers.
Closi 5y ago
> The biggest perverse incentive for Google is that making better search results can mean less clicks to ads This is also something that Google can control if competitors come along. i.e. If a reasonable competitor comes along that is willing to sacrifice ad revenue for better search result quality than Google, google can just adjust their search quality upwards to knock them out (and then adjust it back once the competitive threat is gone). Perverse incentives from Google are all over the place - Searching for the delivery business "Just Eat" in the UK for instance returns an ad for their competitor Deliveroo above the legitimate organic search result for me - and I can also see that JustEat are trying to pay for their own brand name just to compete - and IMO this sort of behaviour is anti-competitive, borderline extortion considering Google is the de-facto way of searching for a business, and shocking from a search-quality perspective (where the wrong result is intentionally shown at the top because they paid more money).

PaulHoule 5y ago

Some kinds of "spam" can improve search results. Things have changed in the past few years, now that Google has developed advanced transformer models, but for a long time Google's question answering facility has been: "let spammers make 10^8 pages where the title is the question and the answer is in the page". The trouble is that there's a fine line between "answer is in the page" and "word salad!"

willhinsa 5y ago

> - Enabled downvoting on results, like YT videos. (Has its own spam problems, just like YT) They're going the OPPOSITE DIRECTION from this!! They recently removed all downvote visibility of YouTube videos from the user, so now downvotes only feed into their algorithm. So in the last line of defense of me ending up watching a shitty video, one of the most valuable tools has been removed by my betters. It's preposterous that people think that Google is doing a good job. They're actively getting worse, and ignoring everyone saying so.

asiachick 5y ago
They're doing a great job. I'm so happy dislike visibility have been removed. It removes the effectiveness of pile on coordinated harassment which many youtubers have fallen victim too.
- slig 5y ago
  Except the ratio is visible for the video creator, so they know nonetheless. And there's the possibility of disabling the the like/dislike for each video. All the big channels I follow are mad at this change, and there's a coordinated effort to bring it back.

PaulHoule 5y ago

The custom search engine is harder than you'd think. Google's search algorithm is tuned up for searching the whole web. It turns out the heuristics you need are very different depending on the size of the collection. When Gerard Salton was doing IR experiments with punched cards he was working with collections of as little as 70 documents and in that case you are going to be very concerned about recall and not precision. Maybe there is 1 relevant document and if you miss it you failed. If you had 70 billion documents you might have 10,000 relevant documents and if you lost 60% of them you still have 4,000 documents. The end user gets more results than they can sift through. Thus I always groan when I see a site is using "Google Site Search" because the relevance is usually worse than you'd get with the alternatives. Connected with that is the tuning work: Google has sufficient data to tune up a big model for everybody but true personalized search eludes them because they don't have enough data from you to tune up a model for you.

mrkramer 5y ago
I agree with you that "true personalized search eludes them because they don't have enough data from you to tune up a model for you". That's what Larry Page said as well "Google doesn't know what you know". His ultimate goal is Answer Machine powered by AI but that's not happening anytime soon. I think internet search engines that we are using today are primitive compared to what we will have in the future.

propogandist 5y ago

>Enabled downvoting on results, like YT videos. you mean the dislike counter they just disabled to force people to sit through more low quality content and pre-roll ads to claim increase in platform engagement and viewership? The only thing matters is revenue and Google had increases in acquisition costs in prior revenue reports. Expect to see the data points for the latter metrics to be highlighted on the earnings announcement, and a record quarter for YT coming out of the change.

asiachick 5y ago

> - Enabled downvoting on results, like YT videos. (Has its own spam problems, just like YT) Not convinced this would help. The spammers would just hire people to dislike competitors > - Allowed you to block certain domains from your search results This I would use. Never show me results form collider, watchmojo, ranker, > - Allowed you to create your own custom search engines, like "Programmable Search Engine". I think this would lead to people writing highly polarized engines. The Red Pill engine for example and we'd have a new problem, the proliferation of popular highly biased results. Of course that's not to say Google's results aren't already biased but they certainly are trying to cover everyone.

ancarda 5y ago

> Allowed you to block certain domains from your search results I would love for Google to build this in. Until they do, there is a WebExtension that does this: https://addons.mozilla.org/en-US/firefox/addon/hohser/ https://addons.mozilla.org/en-US/firefox/addon/hohser/ ("Block or Highlight Search Engine Results"). I use it to block stuff like W3Schools so when I search for something, MDN is always #1. Saves me a lot of time having to add "MDN" to the end of every query.

us0r 5y ago

> Allowed you to block certain domains from your search results Blocking pintrest would be a dream come true.