7 ms·

This needs to be higher up. If you go to the about section of Yep.com, you can see this is powered by the "Ahrefs" bot (not just for sourcing crawler data but

by nacs 4y ago

This needs to be higher up. If you go to the about section of Yep.com, you can see this is powered by the "Ahrefs" bot (not just for sourcing crawler data but also run by the same people). Ahrefs is a sleazy "SEO tool" ( https://ahrefs.com/pricing https://ahrefs.com/pricing ) that basically exists to exploit weaknesses in search engine algorithms and other shady things (and make lots of money doing it -- $99/mo to $999/mo).

fooey 4y ago
This whole yep.com thing feels like an attempt to get everyone to please stop blocking their bot
- blinding-streak 4y ago
  100% this right here. Ahrefs and Yep is not some altruistic endeavor.
- moneywoes 4y ago
  Dumb question but would they not rotate bots in a situation like this? Or do they have a specific crawler
  - archerx 4y ago
    They have am aggressive crawler that I block. Why would I let a company use my resources to gather my data just so they can sell it to others for a premium at my expense?
  - ricardo81 4y ago
    They use a specific UA which is in the spirit of robots.txt, you're able to identify and allow/disallow access. Trying to masquerade as another agent would be considered bad form, but obviously happens a lot. There's a similar bot, MJ12Bot that powers Majestic's index which is similar to ahrefs. IIRC they have a user agent but their crawling is distributed, it's impossible to verify whether someone with that UA is them or someone else masquerading. Good practice by bot owners is having a UA and known IPs they crawl from which can be verified by DNS and reverse DNS lookups.
swinnipeg 4y ago
Been using ahrefs for years, not sure what is sleazy about it. It audits your site for issues, shows Google rankings for keywords, estimated volume for keywords, backlinks to specific sites, backlinks to specific pages...etc. It doesn't do anything for SEO (no content, no links...etc), it simply reports on the state of affairs.
- colordrops 4y ago
  SEO itself is sleazy. Any gaming of search is sleazy, whether active or passive. The NSA is sleazy, even though they simply report on "the state of affairs".
  - richardpetersen 4y ago
    SEO is a skill set that helps algorithms understand content. Google even lists it as a required skill for some of their roles: https://careers.google.com/jobs/results/83882884131103430-product-marketing-manager-seo-growth-lab/ https://careers.google.com/jobs/results/83882884131103430-pr...
    - Goronmon 4y ago
      Google even lists it as a required skill for some of their roles: I don't know if I'd use Google as a great example of sleazy vs non-sleazy...
      - deleted 4y ago
        [deleted]
  - wonderbore 4y ago
    I get this sentiment. "SEO" is basically an euphemism for spamming at this point. It's no longer just "Let's repeat this keyword 3 times" but "let's add 120 footer links and spam 35 blog comment sections per day"
    - 4y ago
deleted 4y ago
[deleted]
encoderer 4y ago
ah yes the sleazy underbelly of “researching what users are actually searching for”
- Spivak 4y ago
  - puranjay 4y ago
    Yeah, a bad article ranking higher is obviously graverobbing /s
    - drdeca 4y ago
      what part of the analogy do you object to? Is it just that it isn't literally desecrating a corpse? Do you have an alternative example of "something which a shove is especially useful for and which which could be lucrative, but which is widely regarded as bad" ?
patrickstox 4y ago
Folks here have a really strange view of SEO. These days it's really following guidelines, creating content that people want to see and is useful to searchers, linking to sources, etc.
- account42 4y ago
  Content that people want to see ... plastered with affiliate links and/or biased to sell those people what you want them to buy. This is about extracting value and that value has to come from somewhere. Searchers are not benefiting here, they are the ones ultimately paying for it.
- wonderbore 4y ago
  It really depends on who you ask. The most "successful" companies don't whitehat their way there. Just open any modern social media website and tell me if they are respectful of the users — and you can see that part. SEO is mostly invisible so they really don't have to care about you.
- endless1234 4y ago
  The massive amount of github/stackoverflow content clones sitting at the top of the results for programming related queries seem to disagree with this sentiment. Ok I guess they are "creating content people want to see", but if just copypasting everyting from another source is enough for that...
  - rapnie 4y ago
    Also the large amount of bots creating fake accounts on forums (my experience is with Discourse) with profiles and spam posts to SEO boost some site indicate a whole shady background business going here.
    - popcorncowboy 4y ago
      Yeah, reminds me of anecdotes from the Dan Lyons book [0] about Hubspot. There was the shiny fun stuff happening "front of house", but back in the boiler-room it was ugly. And everyone gets shown only what they need to see to make them feel happy/sign/pay, and nothing else. No-one actually wants to know how the sausage is made. [0] - https://www.goodreads.com/book/show/26030703-disrupted https://www.goodreads.com/book/show/26030703-disrupted
puranjay 4y ago
What exactly is “sleazy” about Ahrefs? Its literally just data.
dewey 4y ago
It sounds like you never actually used ahrefs, or even know what it does. You just read “SEO” and sent straight to “these are the people spamming Google with content marketing fluff posts when I search for a product review!” What it does is showing you errors on your page 404, slow urls, pages that link to your site, missing tags, images that are too big and load slowly, keyword research and other things. I use it even for my small projects (the free version) because it helps me fix broken links or improve the page speed which is all good for users.

by nacs 4y ago

fooey 4y ago

This whole yep.com thing feels like an attempt to get everyone to please stop blocking their bot

blinding-streak 4y ago
100% this right here. Ahrefs and Yep is not some altruistic endeavor.
moneywoes 4y ago
Dumb question but would they not rotate bots in a situation like this? Or do they have a specific crawler
- archerx 4y ago
  They have am aggressive crawler that I block. Why would I let a company use my resources to gather my data just so they can sell it to others for a premium at my expense?
- ricardo81 4y ago
  They use a specific UA which is in the spirit of robots.txt, you're able to identify and allow/disallow access. Trying to masquerade as another agent would be considered bad form, but obviously happens a lot. There's a similar bot, MJ12Bot that powers Majestic's index which is similar to ahrefs. IIRC they have a user agent but their crawling is distributed, it's impossible to verify whether someone with that UA is them or someone else masquerading. Good practice by bot owners is having a UA and known IPs they crawl from which can be verified by DNS and reverse DNS lookups.

swinnipeg 4y ago

Been using ahrefs for years, not sure what is sleazy about it. It audits your site for issues, shows Google rankings for keywords, estimated volume for keywords, backlinks to specific sites, backlinks to specific pages...etc. It doesn't do anything for SEO (no content, no links...etc), it simply reports on the state of affairs.

colordrops 4y ago
SEO itself is sleazy. Any gaming of search is sleazy, whether active or passive. The NSA is sleazy, even though they simply report on "the state of affairs".
- richardpetersen 4y ago
  SEO is a skill set that helps algorithms understand content. Google even lists it as a required skill for some of their roles: https://careers.google.com/jobs/results/83882884131103430-product-marketing-manager-seo-growth-lab/ https://careers.google.com/jobs/results/83882884131103430-pr...
  - Goronmon 4y ago
    Google even lists it as a required skill for some of their roles: I don't know if I'd use Google as a great example of sleazy vs non-sleazy...
    - deleted 4y ago
      [deleted]
- wonderbore 4y ago
  I get this sentiment. "SEO" is basically an euphemism for spamming at this point. It's no longer just "Let's repeat this keyword 3 times" but "let's add 120 footer links and spam 35 blog comment sections per day"
  - 4y ago

deleted 4y ago

[deleted]

encoderer 4y ago

ah yes the sleazy underbelly of “researching what users are actually searching for”

Spivak 4y ago
- puranjay 4y ago
  Yeah, a bad article ranking higher is obviously graverobbing /s
  - drdeca 4y ago
    what part of the analogy do you object to? Is it just that it isn't literally desecrating a corpse? Do you have an alternative example of "something which a shove is especially useful for and which which could be lucrative, but which is widely regarded as bad" ?

patrickstox 4y ago

Folks here have a really strange view of SEO. These days it's really following guidelines, creating content that people want to see and is useful to searchers, linking to sources, etc.

account42 4y ago
Content that people want to see ... plastered with affiliate links and/or biased to sell those people what you want them to buy. This is about extracting value and that value has to come from somewhere. Searchers are not benefiting here, they are the ones ultimately paying for it.
wonderbore 4y ago
It really depends on who you ask. The most "successful" companies don't whitehat their way there. Just open any modern social media website and tell me if they are respectful of the users — and you can see that part. SEO is mostly invisible so they really don't have to care about you.
endless1234 4y ago
The massive amount of github/stackoverflow content clones sitting at the top of the results for programming related queries seem to disagree with this sentiment. Ok I guess they are "creating content people want to see", but if just copypasting everyting from another source is enough for that...
- rapnie 4y ago
  Also the large amount of bots creating fake accounts on forums (my experience is with Discourse) with profiles and spam posts to SEO boost some site indicate a whole shady background business going here.
  - popcorncowboy 4y ago
    Yeah, reminds me of anecdotes from the Dan Lyons book [0] about Hubspot. There was the shiny fun stuff happening "front of house", but back in the boiler-room it was ugly. And everyone gets shown only what they need to see to make them feel happy/sign/pay, and nothing else. No-one actually wants to know how the sausage is made. [0] - https://www.goodreads.com/book/show/26030703-disrupted https://www.goodreads.com/book/show/26030703-disrupted

puranjay 4y ago

What exactly is “sleazy” about Ahrefs? Its literally just data.

dewey 4y ago

It sounds like you never actually used ahrefs, or even know what it does. You just read “SEO” and sent straight to “these are the people spamming Google with content marketing fluff posts when I search for a product review!” What it does is showing you errors on your page 404, slow urls, pages that link to your site, missing tags, images that are too big and load slowly, keyword research and other things. I use it even for my small projects (the free version) because it helps me fix broken links or improve the page speed which is all good for users.