5 ms·

Apparently this leak is a huge boon to SEO. Expect more SERPs to become SEO-overtaken garbage dumps in the near future. Truly depressing.

by oefrha 4y ago

ad404b8a372f2b9 4y ago
I'm doubtful this will have much of an effect for three reasons: - These are factors, not the underlying algorithms which use them, which means it's not possible to deduce your own ranking from that data. - Many of those are so vague as to be useless. - The most important factors for SEO are those we already know about.
- zexodus 4y ago
  It's possible to use an algorithmic approach (i.e. linear regression) to derive the weights for each of these factors.
  - toto444 4y ago
    All you need to do is scrape the whole internet beforehand.
  - thrwyndx 4y ago
    A lot of these factors depend on internal yandex data, serp clicks especially - good luck getting those outside of yandex prod, and those are the strongest signals.
  - abm53 4y ago
    Given some samples of search results we still don't know: - The X matrix (e.g. the page rank score) for each result. - The y vector, i.e the score for each result. Although we can observe the relative ranking in each sample (would be interested to hear about techniques to cope with this).
  - michaelmior 4y ago
    This is assuming the weights are combined via some linear function. That's not necessarily the case. It's likely that the ranking algorithm is more complex than: calculate features, multiply by weights, add to get ranking. Sure you'll probably still be able to learn something more by playing with the different features, but I doubt you'll get any real meaningful "weights." Especially when how many features are calculated is a black box.
    - h0l0cube 4y ago
      Why would it be more complicated? A simple dot product would have excellent scaling characteristics
      - toto444 4y ago
        When it comes to ML scalability is a constraint not a goal. The goal is to minimize some loss function and it turns out simple dot product can be outperformed by more complex algorithms. I remember reading a few years ago that most search engines use some tree based model. If that's the case, that means the idea of monotonic linear weights is not relevant.
        h0l0cube 4y ago
        Can you be more specific? Dot product is about as performant as it gets with linear memory access and SIMD multiply accumulate. Throw random memory access and flow control in there and it’s a struggle to do it faster. Unless the factors are sparse, in which case just elide the zero values.
        ethbr0 4y ago
        > scalability is a constraint not a goal. The goal is to minimize some loss function
        h0l0cube 4y ago
        My bad. I was under the impression that most search engines are compute bound, but if anything there’s probably a glut of compute for such applications and a market appetite for better results.
luckylion 4y ago
It's not. In addition to what ad404b8a372f2b9 said in a sibling comment: - nobody in the West cares about Yandex SERPs and Yandex isn't Google, so you can't easily transfer the factors - Your favorite SERPs aren't being overtaken by SEO garbage because they're not big/profitable enough to catch attention. It's really more of a dark forest thing in SEO, and search volume is how you detect prey. SEOs don't care about things they cannot make a lot of money from. - SEOs aren't that technical. They employ some technical people (me for instance), but they don't listen to technical people, they care about links links links links, headlines, title, description, microdata (for fancier Google SERP display because having a fancy display can make rank 2 work just as well as rank 1) and keywords. Some do keyword-related stuff like WDF*IDF, but it's more of a ritual where they throw text into different tools and wait for all lamps to turn green. They're really not that sophisticated. Source: am working for large affiliates with millions of visits per month.
- asah 4y ago
  I'm curious: how much of modern SEO is low-level trickery, and how much is deeper marketing strategy ?
  - luckylion 4y ago
    From my understanding, they're somewhat distinct. There are blackhats that try to exploit some quirk and shoot for quick & good rankings that get killed when Google notices or patches a bug etc. The people I work for are mostly light-grey-hats and don't usually do this because they run large sites they don't want to put at risk and they're very conservative about changing anything and doing anything that could be remotely viewed as malicious by Google (even when it's obviously not malicious and would be good for users, there's a lot of "yeah, I agree, but I did/hear about/saw something somewhat similar 10 years ago and Google killed the site and it never recovered" in SEO). Everyone in SEO treats Google like a god. If you have a somewhat stable + successful project, you're sacrificing things to Google (let's all adopt AMP, yes, it's so great! let's all do CWV and say we believe in user experience!) and praise Google each day and avoid anything that you've heard might displease Google for fear of Google sending down lightning and turning your projects to ashes. If you don't have one of those golden geese, you must be more nimble and make sure that the lightning only strikes where you've been yesterday and doesn't catch up with you. The blackhat part is much more stressful (I've had clients get super depressed when their old tricks no longer worked and they burned site after site after site and nothing worked for months), and everyone I know that did that has transitioned to whitehat as soon as an opportunity presented itself and gotten rid of all the blackhat stuff to not have their main sites get caught in a penalty.
ren_engineer 4y ago
not really, 90% of SEO is backlinks/site authority. You can post the same article to a new site and not rank at all, or post it to a stronger old domain and have it rank #1. Google got a lot of flak over the fake news stuff so they've gotten even more conservative with what they rank in recent years. I think that's a big reason people have noticed results getting worse, there is a lot of really great niche stuff on small blogs but Google would rather rank more generic content from an established website. A few major sites like CNET have already gotten caught churning out AI produced stuff and Google still ranks it due to site authority
Pet_Ant 4y ago
SERP = search engine result page Apparently. Had heard of SEO, but not SERP
swyx 4y ago
i always wonder why google doesnt offer a downvote button for obvious bad links. the only reason i can think of is vote bombing, but if you contain that for personalization and do basic bomb detection you could greatly improve the xperience for people
- anaganisk 4y ago
  Google already wants a working phone number for majority of users, making it logged only and leave it to the community to decide. But it will conflict with its "paid" programs so I guess it will never happen.
- ogurechny 4y ago
  A link that is bad for you might serve a lot of ads for a lot of people, which is good for company that has most of its profits from commissions on that process (Google). Here's the shortest explanation of why you'll never get anything better.

Apparently this leak is a huge boon to SEO. Expect more SERPs to become SEO-overtaken garbage dumps in the near future. Truly depressing.

by oefrha 4y ago

Apparently this leak is a huge boon to SEO. Expect more SERPs to become SEO-overtaken garbage dumps in the near future. Truly depressing.

ad404b8a372f2b9 4y ago

I'm doubtful this will have much of an effect for three reasons: - These are factors, not the underlying algorithms which use them, which means it's not possible to deduce your own ranking from that data. - Many of those are so vague as to be useless. - The most important factors for SEO are those we already know about.

zexodus 4y ago
It's possible to use an algorithmic approach (i.e. linear regression) to derive the weights for each of these factors.
- toto444 4y ago
  All you need to do is scrape the whole internet beforehand.
- thrwyndx 4y ago
  A lot of these factors depend on internal yandex data, serp clicks especially - good luck getting those outside of yandex prod, and those are the strongest signals.
- abm53 4y ago
  Given some samples of search results we still don't know: - The X matrix (e.g. the page rank score) for each result. - The y vector, i.e the score for each result. Although we can observe the relative ranking in each sample (would be interested to hear about techniques to cope with this).
- michaelmior 4y ago
  This is assuming the weights are combined via some linear function. That's not necessarily the case. It's likely that the ranking algorithm is more complex than: calculate features, multiply by weights, add to get ranking. Sure you'll probably still be able to learn something more by playing with the different features, but I doubt you'll get any real meaningful "weights." Especially when how many features are calculated is a black box.
  - h0l0cube 4y ago
    Why would it be more complicated? A simple dot product would have excellent scaling characteristics
    - toto444 4y ago
      When it comes to ML scalability is a constraint not a goal. The goal is to minimize some loss function and it turns out simple dot product can be outperformed by more complex algorithms. I remember reading a few years ago that most search engines use some tree based model. If that's the case, that means the idea of monotonic linear weights is not relevant.
      - h0l0cube 4y ago
        Can you be more specific? Dot product is about as performant as it gets with linear memory access and SIMD multiply accumulate. Throw random memory access and flow control in there and it’s a struggle to do it faster. Unless the factors are sparse, in which case just elide the zero values.
        ethbr0 4y ago
        > scalability is a constraint not a goal. The goal is to minimize some loss function
        h0l0cube 4y ago
        My bad. I was under the impression that most search engines are compute bound, but if anything there’s probably a glut of compute for such applications and a market appetite for better results.

luckylion 4y ago

It's not. In addition to what ad404b8a372f2b9 said in a sibling comment: - nobody in the West cares about Yandex SERPs and Yandex isn't Google, so you can't easily transfer the factors - Your favorite SERPs aren't being overtaken by SEO garbage because they're not big/profitable enough to catch attention. It's really more of a dark forest thing in SEO, and search volume is how you detect prey. SEOs don't care about things they cannot make a lot of money from. - SEOs aren't that technical. They employ some technical people (me for instance), but they don't listen to technical people, they care about links links links links, headlines, title, description, microdata (for fancier Google SERP display because having a fancy display can make rank 2 work just as well as rank 1) and keywords. Some do keyword-related stuff like WDF*IDF, but it's more of a ritual where they throw text into different tools and wait for all lamps to turn green. They're really not that sophisticated. Source: am working for large affiliates with millions of visits per month.

asah 4y ago
I'm curious: how much of modern SEO is low-level trickery, and how much is deeper marketing strategy ?
- luckylion 4y ago
  From my understanding, they're somewhat distinct. There are blackhats that try to exploit some quirk and shoot for quick & good rankings that get killed when Google notices or patches a bug etc. The people I work for are mostly light-grey-hats and don't usually do this because they run large sites they don't want to put at risk and they're very conservative about changing anything and doing anything that could be remotely viewed as malicious by Google (even when it's obviously not malicious and would be good for users, there's a lot of "yeah, I agree, but I did/hear about/saw something somewhat similar 10 years ago and Google killed the site and it never recovered" in SEO). Everyone in SEO treats Google like a god. If you have a somewhat stable + successful project, you're sacrificing things to Google (let's all adopt AMP, yes, it's so great! let's all do CWV and say we believe in user experience!) and praise Google each day and avoid anything that you've heard might displease Google for fear of Google sending down lightning and turning your projects to ashes. If you don't have one of those golden geese, you must be more nimble and make sure that the lightning only strikes where you've been yesterday and doesn't catch up with you. The blackhat part is much more stressful (I've had clients get super depressed when their old tricks no longer worked and they burned site after site after site and nothing worked for months), and everyone I know that did that has transitioned to whitehat as soon as an opportunity presented itself and gotten rid of all the blackhat stuff to not have their main sites get caught in a penalty.

ren_engineer 4y ago

not really, 90% of SEO is backlinks/site authority. You can post the same article to a new site and not rank at all, or post it to a stronger old domain and have it rank #1. Google got a lot of flak over the fake news stuff so they've gotten even more conservative with what they rank in recent years. I think that's a big reason people have noticed results getting worse, there is a lot of really great niche stuff on small blogs but Google would rather rank more generic content from an established website. A few major sites like CNET have already gotten caught churning out AI produced stuff and Google still ranks it due to site authority

Pet_Ant 4y ago

SERP = search engine result page Apparently. Had heard of SEO, but not SERP

swyx 4y ago

i always wonder why google doesnt offer a downvote button for obvious bad links. the only reason i can think of is vote bombing, but if you contain that for personalization and do basic bomb detection you could greatly improve the xperience for people

anaganisk 4y ago
Google already wants a working phone number for majority of users, making it logged only and leave it to the community to decide. But it will conflict with its "paid" programs so I guess it will never happen.
ogurechny 4y ago
A link that is bad for you might serve a lot of ads for a lot of people, which is good for company that has most of its profits from commissions on that process (Google). Here's the shortest explanation of why you'll never get anything better.