9 ms·

Disclaimer: I work for Google, though far away from Search. Regardless of search engine design, there's HUGE money in SEO. Any successful search engine will be

by larsrc 4y ago

Disclaimer: I work for Google, though far away from Search. Regardless of search engine design, there's HUGE money in SEO. Any successful search engine will be gamed. Do you have the developer power to go red-queen against all the large companies in the world?

DisjointedHunt 4y ago
And that money goes to support many industries beyond search itself. The author really needs to get off a computer for a minute and understand the economics of the web as it stands today and how the free ad supported model supports millions of peoples livelihoods before jumping to "This is all bullshit"
- vineyardmike 4y ago
  While I agree that they may need to better consider the economics - both of the engine and the websites that may SEO it- that doesn’t mean we should just assume as supported is the way to go or the only way to support people. The economy of today looks different than 20 years ago or 20 years before that. Doesn’t mean we shouldn’t grow and change.
  - DisjointedHunt 4y ago
    I respect that. I guess my point is "Grow and change with a decent understanding of what the present state enables" A search engine like Google isn't just a search engine as the author describes it. It is a very integral part of the economy of the internet and just labelling a simplistic interpretation of the present state as "Evil" with an academically poor write up of what a viable alternative is does little good.
    - 6510 4y ago
      I want to write articles and read articles written for me by others. Ideally as few as possible should profit from this process. As google is now a turd, not just no longer capable of delivering this service but actively destroying the good part of the web by refusing to index it. It is my attention, it doesn't belong to anyone else. My access to information and educated opinion is a far more integral part of the greater economy. Google is like a screaming man at a town meeting making sure no one else can get a word in. The meeting is now pointless.
      - DisjointedHunt 4y ago
        Google is a catalogue. There is no physical analogy that comes close. Your point about wanting to read articles written for you by others is certainly possible. The very fact that such a desirable outcome drives you to Google and nowhere else should suggest the complexity of the problem they’re solving and how there isn’t really anything else out there doing so well. >Ideally as few as possible should profit from this process. Why? When you accepted a job offer in the software industry, did you stipulate that your mission is to write code for your employer and you will be charging as little as possible for that privilege? Minimum wage should get you by just fine, right? I hate fully grown adults behaving as though anyone except them making a profit is somehow evil.
        6510 4y ago
        > The very fact that such a desirable outcome drives you to Google and nowhere else should suggest the complexity of the problem they’re solving and how there isn’t really anything else out there doing so well. Yes and I'm not impressed. > > Ideally as few as possible should profit from this process. > Why? > When you accepted a job offer in the software industry, did you stipulate that your mission is to write code for your employer and you will be charging as little as possible for that privilege? Minimum wage should get you by just fine, right? I'm not a good example as I indeed live wonderfully on minimum wage and write software for free. > I hate fully grown adults behaving as though anyone except them making a profit is somehow evil. Don't worry, my philosophy is not that superficial. We have people who make things, people who organize the making of things and people who organize the things made. It can be true that the meta data is more valuable than the data it self and organizing an effort can be much more intense than any of the tasks involved. But lets not pretend that is always the case. Before money and before the written word we had the exchange of thoughts, observations and ideas. I believe this to be somewhat like the foundation on which everything else we did is build. I want to see this process benefit from technology. You wrote your comment perhaps a bit limited by the ropes of the platform but sincerely, free from any agenda, you wrote pretty much what you think. Now if we [beyond HN] add additional layers of agendas between our exchange, each interested in maximizing their profit from it perhaps not you but many others will resort to self-moderation. You wont be able to state it simply like: "I hate fully grown adults behaving as though anyone except them making a profit is somehow evil." It could become something like "I don't understand why some people don't like others making money" stripped from how strong you feel about the subject. You could also chose not to say anything. At that point we are messing with the very fabric of our collective reality. If I had to chose between freely communicating and the economy it wouldn't be a hard choice.
- Nextgrid 4y ago
  > the free ad supported model supports millions of peoples livelihoods That doesn't necessarily make it right. Online scamming also supports a lot of people's livelihoods.
  - DisjointedHunt 4y ago
    One is a business model, one is a crime and fraud. I’d expect you to know the difference.
    - Nextgrid 4y ago
      Lying in a privacy policy and breaching privacy-related regulations can also be fraud and illegal. Think about why there's so much pushback against the GDPR despite it only primarily mandating transparency with regards to data usage (if they were doing things above-board why would they be afraid?).
mattpavelle 4y ago
This is a great point and a really hard problem to solve. Do you think a Wikipedia style team of human reviewers with upvote/downvote capabilities can help reduce the SEO spam? Obviously it’s hard to review the reviewers as well but Wikipedia seems to have done that.
amelius 4y ago
I think Google search is using the wrong approach. When I'm looking for e.g. a new camera, I want to use my network which I trust. E.g. I want to ask "what camera would HN recommend?" We should think more about how we can use trust as a basis for how we explore the internet.
- sevazhidkov 4y ago
  Given that enough people use this heuristic, there will be companies focusing on earning karma on HN, writing comments and voting for products they are getting paid for.
  - kreeben 4y ago
    Wouldn't the world become a little bit better if they did? Earning karma on HN is not the easiest thing. I would hate to speak for all of us but collectively, don't you think we have a pretty good marketspeak alert system here?
    - blowski 4y ago
      I’ve got 10k karma by making pithy criticisms of Node.js and Kubernetes. Do you therefore trust my opinions on US healthcare?
      - DoctorDabadedoo 4y ago
        Show us the way, reasonable man.
      - kreeben 4y ago
        I would, if what you say were to make a whole lot of sense, otherwise, no. You know how it works here. We would strip you of your internet points if you start being nonsensical.
        jstummbillig 4y ago
        I think you are kind of ignoring the issue that parent brought up. The hn machine is not all that smart and can easily be gamed, and would, if the stakes were to become high enough. Farming hn karma by making pointed statements on crowd favorites (parent named two, oss licensing or privacy also spring to mind) gets you your 10k, no originality or honesty required, in no time. What's protecting hn is a lot of moderation + relative irrelevance. If those 10k were to systematically bring you enough eyes (by driving search results), you are in effect printing money. There is no reason to assume the number of people doing it would not scale with the return attached to doing it.
bsenftner 4y ago
Actually, it is entire possible to create search criteria that cannot be gamed. That is what Google Research's various arms ought to be working on. But we all know if is far more profitable to have a manipulatable system.
- jonathanstrange 4y ago
  Could you expand on this a bit more? I don't see any obvious way to develop non-manipulable search criteria, I was suspecting that there might even be an impossibility theorem about this (which would depend a lot on the exact formulation, though), and I'd like to know what you have in mind.
nonameking2026 4y ago
Allow users to blacklist sites. Share blacklisted sites. Have the option of instead of hiding blacklisted sites entirely - show them in a different column. Have 4 different types of list. Whitelist - highest scoring Not listed - these websites are not ranked. Yellowlist - show but keep in a separate column Blacklist - don't show https://search.brave.com/help/goggles https://search.brave.com/help/goggles is interesting.
- allendoerfer 4y ago
  I found the thought of adding a "yellowlist" funny, as some people are starting to find the use of "whitelist" and "blacklist" racist.
  - fauigerzigerk 4y ago
    It would be more consistent to call it greenlist, yellowlist, redlist then.
    - recuter 4y ago
      How dare you, some of my best friends identify as traffic lights.
      - swayvil 4y ago
        That term has been judged problematic. The appropriate term is "person of vehicular illuminativeness".
    - salawat 4y ago
      How dare you. Some of my best friends are color blind! You insensitive clod!
  - Parzival99 4y ago
    lmao, feeling left out with no brown list then.
  - 4y ago
O__________O 4y ago
For clarification, “red queen” means a conflict between two or more entities where the cost of engagement grows, but the relative advantage does not change. Simply put, search engines have been at war with SEO for over 30-years, which has significantly raised the bar not only being a search engine, but producing content; not to mention knowing how to search for information. With the introduction of machine generated content, information wars between countries, global dependence of online commerce & information, etc — the speed of change shows no signs of letting up. In my opinion, for the average person, knowing how to search for information is the real issue, not that the quality of information available has become worse or that Google has become a worse search engine. If anything, Google has reduced its advantage search capabilities not for financial gain, but because average user is just too lazy to learn how to search and keep up with changes required to continue to be an advanced searcher.
- IgorPartola 4y ago
  To your last point, I don’t quite agree. Google’s incentives are misaligned such that keeping you on Google.com just a little longer is better than not because you are more likely to click on an ad. But also yes the users and the UI both fail. When I used to search for something I would type in something like “gutter clog clean” but slowly started noticing that Google likes longer sentences like “how do I clean a clog in my gutters?”. In pursuit of making Knowsmore (from Ralph Breaks the Internet), Google lost the power user features. Search would be infinitely better if they actually fucking respected literal mode and stopped trying to treat me like an idiot with no attention span. Having search results that contain one out of like 8 words in my query and asking me if I want to include others and then when I say I do still showing me results without them is broken UI and not a user problem.
  - Gareth321 4y ago
    >To your last point, I don’t quite agree. Google’s incentives are misaligned such that keeping you on Google.com just a little longer is better than not because you are more likely to click on an ad. I agree, which leads me to the conclusion that subscription is the best way to avoid this conflict of interest. Unfortunately, most of the world won't subscribe to a search engine, and doesn't seem to mind ads - to a degree. With Google looking more and more like AltaVista before its demise (to Google), my conclusion is that Google will strangle itself out of existence and give way for the next "new, streamlined, not-full-of-ads" competitor.
marginalia_nu 4y ago
I think this is mostly a problem if you are in Google's position, near total market dominance. This not only means you exert massive selection pressure on the shape of websites. The SEO spammers don't need to be good or know what they are doing, they just need to be lucky once. If they get it right, they float to the top, and can iterate on that design. This effectively is saying that no matter how secret or smart your algorithms are, it doesn't matter if you're in Google's position. The numbers are stacked against you. To make matters worse, any company with that sort of a market share has serious handcuffs in how heavy handed and "unfair" you can be without risking litigation for anti-competitive practices. I think the best thing that could happen for Google is ironically serious competition in the search market. This would help both problems at once.
philjr 4y ago
That’s a first world problem. The big problem is becoming successful.
fartcannon 4y ago
Authorship attribution AI's
groffee 4y ago
> Any successful search engine will be gamed. Only because it's profitable for them to allow it to be gamed, like all the spam sites now when you search for SO, Google allows them to be ranked because they're filled with Google ads. But it'd be trivial to just delist them all, that'd be beneficial for the user but not the search engine. It's not a matter of 'developer power' just flip a boolean somewhere and delist the site.
Terry_Roll 4y ago
Because of the filter bubble highlighted by Eli Pariser, there is massive opportunities for SEO companies to trick their customers into thinking they have got high up in the SE results, when all they are seeing is their filter bubble! https://www.ted.com/talks/eli_pariser_beware_online_filter_bubbles https://www.ted.com/talks/eli_pariser_beware_online_filter_b... The last link someone clicks on, is usually the right link to associate with the search term for future users. Its not hard to make a search engine! I'm not a Robot!
nisa 4y ago
> Do you have the developer power to go red-queen against all the large companies in the world? Let's try? It's also an interesting research topic in itself and might be a topic for academic research. At the moment Google is a black box and their incentives are not really aligned to stop SEO. It's good for them to show more ads, it's good for them to show you copycat pages of github/stackoverflow with ads. Not saying that Google is doing this on purpose - I doubt it - but we don't know. It's surely possible to create an index and ranking that prefers different things than Google. Let's try something at least. It will be gamed, will be worse probably but it's open and can be a playground for academic research. Best that can happen is that there are ways to for a better ranking and Google was dishonest to maximize profit. If it's still gamed at least the mechanics can be studied and analysed and maybe someone can figure out a switch like 'be unfair in ways Google can't' - would love a 'no ads on the page' switch that would probably solve quite a few problems. Trust our black-box and you won't have enough devs for this is just a bad answer for such an important problem. The amount of stupid simple redirection spam in my results in the last few years also looks like that Google just doesn't care alot about this anymore.

by larsrc 4y ago

DisjointedHunt 4y ago

And that money goes to support many industries beyond search itself. The author really needs to get off a computer for a minute and understand the economics of the web as it stands today and how the free ad supported model supports millions of peoples livelihoods before jumping to "This is all bullshit"

vineyardmike 4y ago
While I agree that they may need to better consider the economics - both of the engine and the websites that may SEO it- that doesn’t mean we should just assume as supported is the way to go or the only way to support people. The economy of today looks different than 20 years ago or 20 years before that. Doesn’t mean we shouldn’t grow and change.
- DisjointedHunt 4y ago
  I respect that. I guess my point is "Grow and change with a decent understanding of what the present state enables" A search engine like Google isn't just a search engine as the author describes it. It is a very integral part of the economy of the internet and just labelling a simplistic interpretation of the present state as "Evil" with an academically poor write up of what a viable alternative is does little good.
  - 6510 4y ago
    I want to write articles and read articles written for me by others. Ideally as few as possible should profit from this process. As google is now a turd, not just no longer capable of delivering this service but actively destroying the good part of the web by refusing to index it. It is my attention, it doesn't belong to anyone else. My access to information and educated opinion is a far more integral part of the greater economy. Google is like a screaming man at a town meeting making sure no one else can get a word in. The meeting is now pointless.
    - DisjointedHunt 4y ago
      Google is a catalogue. There is no physical analogy that comes close. Your point about wanting to read articles written for you by others is certainly possible. The very fact that such a desirable outcome drives you to Google and nowhere else should suggest the complexity of the problem they’re solving and how there isn’t really anything else out there doing so well. >Ideally as few as possible should profit from this process. Why? When you accepted a job offer in the software industry, did you stipulate that your mission is to write code for your employer and you will be charging as little as possible for that privilege? Minimum wage should get you by just fine, right? I hate fully grown adults behaving as though anyone except them making a profit is somehow evil.
      - 6510 4y ago
        > The very fact that such a desirable outcome drives you to Google and nowhere else should suggest the complexity of the problem they’re solving and how there isn’t really anything else out there doing so well. Yes and I'm not impressed. > > Ideally as few as possible should profit from this process. > Why? > When you accepted a job offer in the software industry, did you stipulate that your mission is to write code for your employer and you will be charging as little as possible for that privilege? Minimum wage should get you by just fine, right? I'm not a good example as I indeed live wonderfully on minimum wage and write software for free. > I hate fully grown adults behaving as though anyone except them making a profit is somehow evil. Don't worry, my philosophy is not that superficial. We have people who make things, people who organize the making of things and people who organize the things made. It can be true that the meta data is more valuable than the data it self and organizing an effort can be much more intense than any of the tasks involved. But lets not pretend that is always the case. Before money and before the written word we had the exchange of thoughts, observations and ideas. I believe this to be somewhat like the foundation on which everything else we did is build. I want to see this process benefit from technology. You wrote your comment perhaps a bit limited by the ropes of the platform but sincerely, free from any agenda, you wrote pretty much what you think. Now if we [beyond HN] add additional layers of agendas between our exchange, each interested in maximizing their profit from it perhaps not you but many others will resort to self-moderation. You wont be able to state it simply like: "I hate fully grown adults behaving as though anyone except them making a profit is somehow evil." It could become something like "I don't understand why some people don't like others making money" stripped from how strong you feel about the subject. You could also chose not to say anything. At that point we are messing with the very fabric of our collective reality. If I had to chose between freely communicating and the economy it wouldn't be a hard choice.
Nextgrid 4y ago
> the free ad supported model supports millions of peoples livelihoods That doesn't necessarily make it right. Online scamming also supports a lot of people's livelihoods.
- DisjointedHunt 4y ago
  One is a business model, one is a crime and fraud. I’d expect you to know the difference.
  - Nextgrid 4y ago
    Lying in a privacy policy and breaching privacy-related regulations can also be fraud and illegal. Think about why there's so much pushback against the GDPR despite it only primarily mandating transparency with regards to data usage (if they were doing things above-board why would they be afraid?).

mattpavelle 4y ago

This is a great point and a really hard problem to solve. Do you think a Wikipedia style team of human reviewers with upvote/downvote capabilities can help reduce the SEO spam? Obviously it’s hard to review the reviewers as well but Wikipedia seems to have done that.

amelius 4y ago

I think Google search is using the wrong approach. When I'm looking for e.g. a new camera, I want to use my network which I trust. E.g. I want to ask "what camera would HN recommend?" We should think more about how we can use trust as a basis for how we explore the internet.

sevazhidkov 4y ago
Given that enough people use this heuristic, there will be companies focusing on earning karma on HN, writing comments and voting for products they are getting paid for.
- kreeben 4y ago
  Wouldn't the world become a little bit better if they did? Earning karma on HN is not the easiest thing. I would hate to speak for all of us but collectively, don't you think we have a pretty good marketspeak alert system here?
  - blowski 4y ago
    I’ve got 10k karma by making pithy criticisms of Node.js and Kubernetes. Do you therefore trust my opinions on US healthcare?
    - DoctorDabadedoo 4y ago
      Show us the way, reasonable man.
    - kreeben 4y ago
      I would, if what you say were to make a whole lot of sense, otherwise, no. You know how it works here. We would strip you of your internet points if you start being nonsensical.
      - jstummbillig 4y ago
        I think you are kind of ignoring the issue that parent brought up. The hn machine is not all that smart and can easily be gamed, and would, if the stakes were to become high enough. Farming hn karma by making pointed statements on crowd favorites (parent named two, oss licensing or privacy also spring to mind) gets you your 10k, no originality or honesty required, in no time. What's protecting hn is a lot of moderation + relative irrelevance. If those 10k were to systematically bring you enough eyes (by driving search results), you are in effect printing money. There is no reason to assume the number of people doing it would not scale with the return attached to doing it.

bsenftner 4y ago

Actually, it is entire possible to create search criteria that cannot be gamed. That is what Google Research's various arms ought to be working on. But we all know if is far more profitable to have a manipulatable system.

jonathanstrange 4y ago
Could you expand on this a bit more? I don't see any obvious way to develop non-manipulable search criteria, I was suspecting that there might even be an impossibility theorem about this (which would depend a lot on the exact formulation, though), and I'd like to know what you have in mind.

nonameking2026 4y ago

Allow users to blacklist sites. Share blacklisted sites. Have the option of instead of hiding blacklisted sites entirely - show them in a different column. Have 4 different types of list. Whitelist - highest scoring Not listed - these websites are not ranked. Yellowlist - show but keep in a separate column Blacklist - don't show https://search.brave.com/help/goggles https://search.brave.com/help/goggles is interesting.

allendoerfer 4y ago
I found the thought of adding a "yellowlist" funny, as some people are starting to find the use of "whitelist" and "blacklist" racist.
- fauigerzigerk 4y ago
  It would be more consistent to call it greenlist, yellowlist, redlist then.
  - recuter 4y ago
    How dare you, some of my best friends identify as traffic lights.
    - swayvil 4y ago
      That term has been judged problematic. The appropriate term is "person of vehicular illuminativeness".
  - salawat 4y ago
    How dare you. Some of my best friends are color blind! You insensitive clod!
- Parzival99 4y ago
  lmao, feeling left out with no brown list then.
- 4y ago

O__________O 4y ago

For clarification, “red queen” means a conflict between two or more entities where the cost of engagement grows, but the relative advantage does not change. Simply put, search engines have been at war with SEO for over 30-years, which has significantly raised the bar not only being a search engine, but producing content; not to mention knowing how to search for information. With the introduction of machine generated content, information wars between countries, global dependence of online commerce & information, etc — the speed of change shows no signs of letting up. In my opinion, for the average person, knowing how to search for information is the real issue, not that the quality of information available has become worse or that Google has become a worse search engine. If anything, Google has reduced its advantage search capabilities not for financial gain, but because average user is just too lazy to learn how to search and keep up with changes required to continue to be an advanced searcher.

IgorPartola 4y ago
To your last point, I don’t quite agree. Google’s incentives are misaligned such that keeping you on Google.com just a little longer is better than not because you are more likely to click on an ad. But also yes the users and the UI both fail. When I used to search for something I would type in something like “gutter clog clean” but slowly started noticing that Google likes longer sentences like “how do I clean a clog in my gutters?”. In pursuit of making Knowsmore (from Ralph Breaks the Internet), Google lost the power user features. Search would be infinitely better if they actually fucking respected literal mode and stopped trying to treat me like an idiot with no attention span. Having search results that contain one out of like 8 words in my query and asking me if I want to include others and then when I say I do still showing me results without them is broken UI and not a user problem.
- Gareth321 4y ago
  >To your last point, I don’t quite agree. Google’s incentives are misaligned such that keeping you on Google.com just a little longer is better than not because you are more likely to click on an ad. I agree, which leads me to the conclusion that subscription is the best way to avoid this conflict of interest. Unfortunately, most of the world won't subscribe to a search engine, and doesn't seem to mind ads - to a degree. With Google looking more and more like AltaVista before its demise (to Google), my conclusion is that Google will strangle itself out of existence and give way for the next "new, streamlined, not-full-of-ads" competitor.

marginalia_nu 4y ago

I think this is mostly a problem if you are in Google's position, near total market dominance. This not only means you exert massive selection pressure on the shape of websites. The SEO spammers don't need to be good or know what they are doing, they just need to be lucky once. If they get it right, they float to the top, and can iterate on that design. This effectively is saying that no matter how secret or smart your algorithms are, it doesn't matter if you're in Google's position. The numbers are stacked against you. To make matters worse, any company with that sort of a market share has serious handcuffs in how heavy handed and "unfair" you can be without risking litigation for anti-competitive practices. I think the best thing that could happen for Google is ironically serious competition in the search market. This would help both problems at once.

philjr 4y ago

That’s a first world problem. The big problem is becoming successful.

fartcannon 4y ago

Authorship attribution AI's

groffee 4y ago

> Any successful search engine will be gamed. Only because it's profitable for them to allow it to be gamed, like all the spam sites now when you search for SO, Google allows them to be ranked because they're filled with Google ads. But it'd be trivial to just delist them all, that'd be beneficial for the user but not the search engine. It's not a matter of 'developer power' just flip a boolean somewhere and delist the site.

Terry_Roll 4y ago

Because of the filter bubble highlighted by Eli Pariser, there is massive opportunities for SEO companies to trick their customers into thinking they have got high up in the SE results, when all they are seeing is their filter bubble! https://www.ted.com/talks/eli_pariser_beware_online_filter_bubbles https://www.ted.com/talks/eli_pariser_beware_online_filter_b... The last link someone clicks on, is usually the right link to associate with the search term for future users. Its not hard to make a search engine! I'm not a Robot!

nisa 4y ago

> Do you have the developer power to go red-queen against all the large companies in the world? Let's try? It's also an interesting research topic in itself and might be a topic for academic research. At the moment Google is a black box and their incentives are not really aligned to stop SEO. It's good for them to show more ads, it's good for them to show you copycat pages of github/stackoverflow with ads. Not saying that Google is doing this on purpose - I doubt it - but we don't know. It's surely possible to create an index and ranking that prefers different things than Google. Let's try something at least. It will be gamed, will be worse probably but it's open and can be a playground for academic research. Best that can happen is that there are ways to for a better ranking and Google was dishonest to maximize profit. If it's still gamed at least the mechanics can be studied and analysed and maybe someone can figure out a switch like 'be unfair in ways Google can't' - would love a 'no ads on the page' switch that would probably solve quite a few problems. Trust our black-box and you won't have enough devs for this is just a bad answer for such an important problem. The amount of stupid simple redirection spam in my results in the last few years also looks like that Google just doesn't care alot about this anymore.