7 ms·

Help Google just nerfed us out of the blue

90 points by useflyer 14y ago

jstanley 14y ago
Sadly, stories like this are getting more and more common.
- useflyer 14y ago
  The black box is getting more complex...this is a problem I hate to see ourselves and any other startups have...
j_s 14y ago
> We just completed the transition [301-ing from MelonCard.com to SafeShepherd.com] yesterday Clearly, your definition of 'out of the blue' is not the same as mine... good luck figuring out if the problem was on your end or theirs (hitting the front page of HN usually helps). Edit: clarified transition per comment below
- nowarninglabel 14y ago
  Yeah, this hardly got a mention, despite being the most obvious and influential factor. It's certainly not out of blue. Now, as to why Google failed to connect the dots for them, who knows.
- savories 14y ago
  Yup. It took us several months to get our page rank back. We did everything "by the book" too. Also: did you request a change of address in Webmaster tools? You have to control both domains, and request a "Change of Address" from old to new.
  - useflyer 14y ago
    I should clarify. We've been ranking for "Safe Shepherd" and SafeShepherd.com. A 301-redirect from MelonCard.com shouldn't bring DOWN our rank for "Safe Shepherd" on SafeShepherd.com. Unless I'm missing how the algorithm works..
    - MiguelHudnandez 14y ago
      If MelonCard was the primary source of "google juice" for SafeShepherd.com, then the change would certainly affect your rank.
      - Danieru 14y ago
        301 redirects are supposed to transfer the link juice so that couldn't have been the issue.
jrockway 14y ago
For me, all but one result on the first page of "Safe Shepherd" is them.
- useflyer 14y ago
  Are you in the US? We were told that changes roll out in the US first and then elsewhere. A friend in Japan said he sees as as #1 in Japan, and nowhere from a US search
  - jrockway 14y ago
    Yes. To be fair, their landing page is nowhere to be found. The first result is their blog. The second result is safeshepherd.com/beenverified. The rest are their blog or the press talking about them. But all in all, they've hardly disappeared from teh intarwebs :)
temphn 14y ago
Google should consider auctioning support time. Make it into a profit center rather than a cost center. No guarantee of a successful outcome, but it takes the situation out of the press.
ericabiz 14y ago
I hit this issue when I owned a .us site. After my site started making money, I arranged to buy the .com. When I made the switch, I did everything by the book (my background is in SEO) including properly 301 redirecting the .us to the .com and alerting Google in Webmaster Tools. The site completely disappeared from the Google search results--same symptoms as yours, with site: returning valid info but the site not showing up in search queries at all for the site's name. This went on for 3 weeks. Finally, after 22 days, it came back up in the rankings. Where previously it had been #4 for its keyword, it came back at #1. No explanation from Google or in Webmaster Tools. (If I may do a brief plug--our rank tracker will help you see if your site is ranking somewhere lower than the first few pages now, and will send you daily email updates so you'll know right away when you come back: http://whooshtraffic.com/rank-tracker/ http://whooshtraffic.com/rank-tracker/ ) Anyway, I'd have to say that this is par for the course for Google. It will likely come back in a few days or weeks. Time to play the waiting game, and develop some good links from your blog to your main site!
- motherwell 14y ago
  >Finally, after 22 days, it came back up in the rankings That is EXACTLY what is supossed to happen. Google runs a SE with over 5 billion URLs - if anyone tells you, or told you, that a transition was seemless and instantaneous, they lied. To explain why, Google has this flow: 1. Crawl the old site - once it finds 301 redirects, it kills the old site and has no data on the new site yet. 2. Google crawls the new site, and starts to apply the old sites criteria - this is NOT, repeat NOT instantaneous. 3. All the "pre processed" signals are applied from the old to the new site. 22 days is a pretty short time to see things come back better, and I'd thank my lucky stars!
keltex 14y ago
I think you've been hit by the "penguin" update, which is probably the biggest SEO embarassment from Google in a long time. Unlike other updates which attempted to remove "web spam" from the search results by tweaking some of the parameters, this update (according to many in the SEO community) is an active attempt to catch people doing black hat or over SEO optimization. Unfortunately many legit sites have gotten caught in its net. You can read about it in the WSJ among other places: http://online.wsj.com/article/SB10001424052702303505504577406751747002494.html http://online.wsj.com/article/SB1000142405270230350550457740...
- sixQuarks 14y ago
  I've been following the Penguin update and this doesn't sound like the cause. The OP said they did not do any crappy link building.
  - jeebus 14y ago
    I've even gone in to Alexa to start verifying all of our back-links. All of our links are from blogs / news / twitter , et al. That said, it doesn't mean Penguin didn't hurt us. Algorithms have corner cases.
    - keltex 14y ago
      Use Open Site Explorer to start on your bad link search: http://www.opensiteexplorer.org/domains?site=safeshepherd.com http://www.opensiteexplorer.org/domains?site=safeshepherd.co...
      - sixQuarks 14y ago
        OpenSE and Majestic SEO databases are not really going to find the spammy links that penguin is targeting: http://www.branded3.com/seo/penguin-update-renders-seomoz-majestic-link-data-unreliable/ http://www.branded3.com/seo/penguin-update-renders-seomoz-ma...
  - zackzackzack 14y ago
atticusfinch 14y ago
You're still number one when I search for you. Proof: http://i.imgur.com/HnTCn.png http://i.imgur.com/HnTCn.png
- chmod775 14y ago
  Same for me using ze German Google
  - cocoflunchy 14y ago
    Same for me in France.
- gee_totes 14y ago
  <evangelizing> Still number one on Duck Duck Go as well. http://duckduckgo.com/?q=safe%20shepherd http://duckduckgo.com/?q=safe%20shepherd </evangelizing>
- hluska 14y ago
  Number three in Canada...(your blog is number one and your article on the BeenVerified scam is two).
- sparknlaunch12 14y ago
  Number two in the UK - Number one is a film called 'Safe' showing tonight at a cinema in 'Sheperds Bush, London'. Number one in AU, BR, JP,...
drone 14y ago
Ok, so I thought, let me check on ours. Boom. Went from #1 when searching for our company name to not even present in the results. What gives? We had perfect placement, built not through any SEO, but just a lot of people talking about us - had the proper config and everything, recently had a site overhaul that was handled properly, using webmaster tools, and everything was updated in google indexes within a day.
ApolloRising 14y ago
Not my site but this may help you: http://www.sxw.org.uk/computing/robots/check.html http://www.sxw.org.uk/computing/robots/check.html ------------ Results for https://safeshepherd.com/robots.txt https://safeshepherd.com/robots.txt Error at line number 1: User-Agent: * Capitalization. Field names are case sensitive - the User-agent field should be written with that capitalization Error at line number 2: Allow: / No User Agent. An Allow line must have a User-agent line before it. As records are delimited by newlines, there cannot be new-lines between the User-agent and Allow lines. Warning at line number 2: Allow: / Allow is not widely supported. The Allow field was a late addition to the robots.txt standard, and is not currently widely supported by crawlers. You should consider alternative ways of constructing your robots.txt file Error at line number 3: Disallow: /login/auth No User Agent. A Disallow line must have a User-agent line before it. As records are delimited by newlines, there cannot be newlines between the User-agent and Disallow lines. Error at line number 4: Disallow: /users No User Agent. A Disallow line must have a User-agent line before it. As records are delimited by newlines, there cannot be newlines between the User-agent and Disallow lines. Error at line number 5: Disallow: /signin No User Agent. A Disallow line must have a User-agent line before it. As records are delimited by newlines, there cannot be newlines between the User-agent and Disallow lines. Error at line number 6: Disallow: /upgrade/submit No User Agent. A Disallow line must have a User-agent line before it. As records are delimited by newlines, there cannot be newlines between the User-agent and Disallow lines.
nhebb 14y ago
I get their blog as the first search result, but at least their blog links to their home page. I'm surprised by how often I see company blogs that don't link to their main site or make to hunt around the side bar for a link.
DanBC 14y ago
I'd be interested to know how Google protects trusty-worthy sites from the SEO equivalent of joe-jobbing.
AznHisoka 14y ago
"Build for users, and don't worry about search engines". I guess situations like this shows the absurdity and lunacy of people who say things like that. But I do see your blog as the first result. As for why your main landing page isn't #1, I suggest you just stay calm, and wait a few days. Google has a knack for bouncing results, especially these past few weeks with the Penguin update and all. And if you just recently did the 301-redirect, those things take time to get sorted out. (Since MelonCard was still a relatively young brand, a 301-redirect was harmless, but if you had an old brand/domain, a 301-redirect would be SEO suicide) You're still in the index. Your blog is ranking for your brand. You haven't been nerfed out of the blue, so stop worrying and be patient.
epoxyhockey 14y ago
Also, your redirection is weird.. You have a 302 on http://meloncard.com http://meloncard.com, that forwards to https://meloncard.com https://meloncard.com which then 301 redirects to safeshepherd.com If I were moving a domain, I would have 301'd http://meloncard.com http://meloncard.com to safeshepherd.com right out of the gate. Too add to everyone's anecdotal experiences, I 301'd a domain last week and the new domain only took a couple of days to show up in search. I was a happy customer.
SpiderX 14y ago
Buy ad words or quit complaining about it. You act like you are entitled to search ranking. You aren't.
TomAnthony 14y ago
Hopefully I can help some. :) I see a couple of problems with your redirects: 1) http://www.MelonCard.com http://www.MelonCard.com uses a 302 redirect to the https version. You have inbound links to that domain pointing to the http version, so the 302 negates the following 301. 2) Furthermore, your 301's seem to be implemented incorrectly. It works for me in Safari/Chrome, but if I use curl on the command line, or any crawling software I see an infinite redirect: www.MelonCard.com/?from=shadow&from=shadow&from=shadow..... This is likely interrupting Googlebot's crawl too. Certainly worth a fix! A couple of other bits to note: 1) The redirect in Chrome sends me to: www.safeshepherd.com/?from=shadow which Google believes could be different to: www.safeshepherd.com/ You should add a rel=canonical meta tag to the page to help Google out. 2) Your internal links point to safeshepherd.com without the www., but your MelonCard redirect redirects to the www. version. You should probably make this consistent, and also 301 one version to the other. This latter points may seem picky, but Google can be troublesome with this. Hope this helps! :)
- epoxyhockey 14y ago
  Since the OP is using nginx, here is how you properly do a 301 redirect straight from nginx.conf (in the 'server' section for meloncard.com): rewrite ^/(.*)$ http://safeshepherd.com/$1 permanent;
  - TomAnthony 14y ago
    Nice tip. Perhaps with https instead of http in the redirect URL, though? :)
    - epoxyhockey 14y ago
      True! It does look like Google will accept https as a canonical homepage link. They do it for Twitter.
  - k33l0r 14y ago
    That's actually more complicated than it needs to be, see http://wiki.nginx.org/Pitfalls#Taxing_Rewrites http://wiki.nginx.org/Pitfalls#Taxing_Rewrites Bad: rewrite ^/(.*)$ http://domain.com/$1 permanent; Good: rewrite ^ http://domain.com$request_uri? permanent; Better: return 301 http://domain.com$request_uri;
pierrefar 14y ago
Hi I work at Google helping webmasters like this. As far as I can tell, there are a couple of interacting issues we're seeing on the site that can be causing what you're seeing. It's a bit technical, but it's easy for you to implement a fix. Firstly, our algorithms recently have been picking one of the following URLs as the canonical URL for the homepage: http://www.safeshepherd.com/ http://www.safeshepherd.com/ https://www.safeshepherd.com/ https://www.safeshepherd.com/ https://safeshepherd.com/ https://safeshepherd.com/ For example, I see that the non-HTTPS pages redirect to the HTTPS pages (e.g. http://www.safeshepherd.com/ http://www.safeshepherd.com/ to https://www.safeshepherd.com/ https://www.safeshepherd.com/), but the non-www pages do not redirect to the www pages (both https://www.safeshepherd.com/ https://www.safeshepherd.com/ and https://safeshepherd.com/ https://safeshepherd.com/ return content). When we find the same content on multiple URLs like this, our algorithms pick one representative URL, and over the past few weeks the choice has been changing. As of 3 days ago, the current choice is https://safeshepherd.com/ https://safeshepherd.com/ . As it stands, our algorithms are trying to figure out the right canonical URL format, but it's difficult in this kind of situation. You can help by redirecting to your preferred URL format (say https://www.safeshepherd.com/* https://www.safeshepherd.com/*), and our systems will pick up this signal, and that will be reflected in the search results and reporting in Webmaster Tools. Secondly, Webmaster Tools treats these as different sites. For example, you would need to verify and check the statistics of both https://www.safeshepherd.com/ https://www.safeshepherd.com/ and https://safeshepherd.com/ https://safeshepherd.com/ (as well as the HTTP versions) as they're separate sites. It may be that you're checking (say) the stats for http://www.safeshepherd.com/ http://www.safeshepherd.com/ but if our algos have picked the https://www.safeshepherd.com/ https://www.safeshepherd.com/ URLs as canonical, the search queries of the former will suddenly be closer to zero but the latter will be a more accurate reflection of the site's traffic. Hope this helps, Pierre
- Codhisattva 14y ago
  Wow! Kudos to Google putting someone out in the wild answering these kinds of questions. Is there a normal place to ask for web/index help or does HN serve that purpose?
  - 14y ago
nicksergeant 14y ago
Um, still #1 for me: http://cl.ly/371t2b0z1S1C0A170y2e http://cl.ly/371t2b0z1S1C0A170y2e
- nicksergeant 14y ago
  I'm in the US, also (Rochester, NY)
andrewhillman 14y ago
If nothing else works, remove the domain from google webmaster tools - completely. I had two sites that lost top serp positioning and once i removed the domains from google webmaster tools, they went right back up to first 3 results on page 1. Google webmaster tools is more or less designed to help google, not you.
quangv 14y ago
sounds like you got slapped?
Monotoko 14y ago
Still number 1 in the UK too
deleted 14y ago
[deleted]
sevenstar 14y ago
You are number one on duckduckgo.Use social marketing if you really need the traffic.
unreal37 14y ago
Great to see Google responding here. Looks like the confusing 301/302 redirections are the problem. I love HN for this type of community stuff.
- lnanek 14y ago
  Agreed. Was very impressive seeing Googlers respond, and respond helpfully which is not always easy in tech esp to outside parties, and have forums and regular G+ hangouts so it isn't a one time thing. Not to mention other people with similar problems point out the delay in reapplying the signal, etc..
readme 14y ago
I just signed up for your service. Hope it helped. Your service will certainly help me. The infographic about beenverified was alarming: (https://www.safeshepherd.com/beenverified https://www.safeshepherd.com/beenverified) and I have actually tried to get my stuff off of there before and failed. I'm not even worried about my own information. What I'm worried about is the accuracy of it! How can I trust a third party who doesn't even know me to provide an accurate background check when it references PRIVATE databases that I can't even verify the integrity of? My biggest worry is that some day I will be screwed out of an opportunity because of a company like this that simply provides inaccurate data because they confused me with some other John Doe. It would be one thing to provide public records as a service, but been verified seriously ticks me off. To think they have the authority to 'verify' people irks me. Then again, it may be a good thing, because I sure as hell wouldn't want to work with anyone stupid enough to use a service like beenverified.
robdwoods 14y ago
I have to add my kudos to Matt and Pierre for taking the time to respond to these threads. They could clearly just refer users to the Google Webmaster forums rather than answering in detail here. For all the flack Google takes, I have to say that I'm encouraged that two senior engineers take the time to answer questions in a 3rd party discussion forum.

90 points by useflyer 14y ago

jstanley 14y ago

Sadly, stories like this are getting more and more common.

useflyer 14y ago
The black box is getting more complex...this is a problem I hate to see ourselves and any other startups have...

j_s 14y ago

> We just completed the transition [301-ing from MelonCard.com to SafeShepherd.com] yesterday Clearly, your definition of 'out of the blue' is not the same as mine... good luck figuring out if the problem was on your end or theirs (hitting the front page of HN usually helps). Edit: clarified transition per comment below

nowarninglabel 14y ago
Yeah, this hardly got a mention, despite being the most obvious and influential factor. It's certainly not out of blue. Now, as to why Google failed to connect the dots for them, who knows.
savories 14y ago
Yup. It took us several months to get our page rank back. We did everything "by the book" too. Also: did you request a change of address in Webmaster tools? You have to control both domains, and request a "Change of Address" from old to new.
- useflyer 14y ago
  I should clarify. We've been ranking for "Safe Shepherd" and SafeShepherd.com. A 301-redirect from MelonCard.com shouldn't bring DOWN our rank for "Safe Shepherd" on SafeShepherd.com. Unless I'm missing how the algorithm works..
  - MiguelHudnandez 14y ago
    If MelonCard was the primary source of "google juice" for SafeShepherd.com, then the change would certainly affect your rank.
    - Danieru 14y ago
      301 redirects are supposed to transfer the link juice so that couldn't have been the issue.

jrockway 14y ago

For me, all but one result on the first page of "Safe Shepherd" is them.

useflyer 14y ago
Are you in the US? We were told that changes roll out in the US first and then elsewhere. A friend in Japan said he sees as as #1 in Japan, and nowhere from a US search
- jrockway 14y ago
  Yes. To be fair, their landing page is nowhere to be found. The first result is their blog. The second result is safeshepherd.com/beenverified. The rest are their blog or the press talking about them. But all in all, they've hardly disappeared from teh intarwebs :)

temphn 14y ago

Google should consider auctioning support time. Make it into a profit center rather than a cost center. No guarantee of a successful outcome, but it takes the situation out of the press.

ericabiz 14y ago

I hit this issue when I owned a .us site. After my site started making money, I arranged to buy the .com. When I made the switch, I did everything by the book (my background is in SEO) including properly 301 redirecting the .us to the .com and alerting Google in Webmaster Tools. The site completely disappeared from the Google search results--same symptoms as yours, with site: returning valid info but the site not showing up in search queries at all for the site's name. This went on for 3 weeks. Finally, after 22 days, it came back up in the rankings. Where previously it had been #4 for its keyword, it came back at #1. No explanation from Google or in Webmaster Tools. (If I may do a brief plug--our rank tracker will help you see if your site is ranking somewhere lower than the first few pages now, and will send you daily email updates so you'll know right away when you come back: http://whooshtraffic.com/rank-tracker/ http://whooshtraffic.com/rank-tracker/ ) Anyway, I'd have to say that this is par for the course for Google. It will likely come back in a few days or weeks. Time to play the waiting game, and develop some good links from your blog to your main site!

motherwell 14y ago
>Finally, after 22 days, it came back up in the rankings That is EXACTLY what is supossed to happen. Google runs a SE with over 5 billion URLs - if anyone tells you, or told you, that a transition was seemless and instantaneous, they lied. To explain why, Google has this flow: 1. Crawl the old site - once it finds 301 redirects, it kills the old site and has no data on the new site yet. 2. Google crawls the new site, and starts to apply the old sites criteria - this is NOT, repeat NOT instantaneous. 3. All the "pre processed" signals are applied from the old to the new site. 22 days is a pretty short time to see things come back better, and I'd thank my lucky stars!

keltex 14y ago

I think you've been hit by the "penguin" update, which is probably the biggest SEO embarassment from Google in a long time. Unlike other updates which attempted to remove "web spam" from the search results by tweaking some of the parameters, this update (according to many in the SEO community) is an active attempt to catch people doing black hat or over SEO optimization. Unfortunately many legit sites have gotten caught in its net. You can read about it in the WSJ among other places: http://online.wsj.com/article/SB10001424052702303505504577406751747002494.html http://online.wsj.com/article/SB1000142405270230350550457740...

sixQuarks 14y ago
I've been following the Penguin update and this doesn't sound like the cause. The OP said they did not do any crappy link building.
- jeebus 14y ago
  I've even gone in to Alexa to start verifying all of our back-links. All of our links are from blogs / news / twitter , et al. That said, it doesn't mean Penguin didn't hurt us. Algorithms have corner cases.
  - keltex 14y ago
    Use Open Site Explorer to start on your bad link search: http://www.opensiteexplorer.org/domains?site=safeshepherd.com http://www.opensiteexplorer.org/domains?site=safeshepherd.co...
    - sixQuarks 14y ago
      OpenSE and Majestic SEO databases are not really going to find the spammy links that penguin is targeting: http://www.branded3.com/seo/penguin-update-renders-seomoz-majestic-link-data-unreliable/ http://www.branded3.com/seo/penguin-update-renders-seomoz-ma...
- zackzackzack 14y ago

atticusfinch 14y ago

You're still number one when I search for you. Proof: http://i.imgur.com/HnTCn.png http://i.imgur.com/HnTCn.png

chmod775 14y ago
Same for me using ze German Google
- cocoflunchy 14y ago
  Same for me in France.
gee_totes 14y ago
<evangelizing> Still number one on Duck Duck Go as well. http://duckduckgo.com/?q=safe%20shepherd http://duckduckgo.com/?q=safe%20shepherd </evangelizing>
hluska 14y ago
Number three in Canada...(your blog is number one and your article on the BeenVerified scam is two).
sparknlaunch12 14y ago
Number two in the UK - Number one is a film called 'Safe' showing tonight at a cinema in 'Sheperds Bush, London'. Number one in AU, BR, JP,...

drone 14y ago

Ok, so I thought, let me check on ours. Boom. Went from #1 when searching for our company name to not even present in the results. What gives? We had perfect placement, built not through any SEO, but just a lot of people talking about us - had the proper config and everything, recently had a site overhaul that was handled properly, using webmaster tools, and everything was updated in google indexes within a day.

ApolloRising 14y ago

Not my site but this may help you: http://www.sxw.org.uk/computing/robots/check.html http://www.sxw.org.uk/computing/robots/check.html ------------ Results for https://safeshepherd.com/robots.txt https://safeshepherd.com/robots.txt Error at line number 1: User-Agent: * Capitalization. Field names are case sensitive - the User-agent field should be written with that capitalization Error at line number 2: Allow: / No User Agent. An Allow line must have a User-agent line before it. As records are delimited by newlines, there cannot be new-lines between the User-agent and Allow lines. Warning at line number 2: Allow: / Allow is not widely supported. The Allow field was a late addition to the robots.txt standard, and is not currently widely supported by crawlers. You should consider alternative ways of constructing your robots.txt file Error at line number 3: Disallow: /login/auth No User Agent. A Disallow line must have a User-agent line before it. As records are delimited by newlines, there cannot be newlines between the User-agent and Disallow lines. Error at line number 4: Disallow: /users No User Agent. A Disallow line must have a User-agent line before it. As records are delimited by newlines, there cannot be newlines between the User-agent and Disallow lines. Error at line number 5: Disallow: /signin No User Agent. A Disallow line must have a User-agent line before it. As records are delimited by newlines, there cannot be newlines between the User-agent and Disallow lines. Error at line number 6: Disallow: /upgrade/submit No User Agent. A Disallow line must have a User-agent line before it. As records are delimited by newlines, there cannot be newlines between the User-agent and Disallow lines.

nhebb 14y ago

I get their blog as the first search result, but at least their blog links to their home page. I'm surprised by how often I see company blogs that don't link to their main site or make to hunt around the side bar for a link.

DanBC 14y ago

I'd be interested to know how Google protects trusty-worthy sites from the SEO equivalent of joe-jobbing.

AznHisoka 14y ago

"Build for users, and don't worry about search engines". I guess situations like this shows the absurdity and lunacy of people who say things like that. But I do see your blog as the first result. As for why your main landing page isn't #1, I suggest you just stay calm, and wait a few days. Google has a knack for bouncing results, especially these past few weeks with the Penguin update and all. And if you just recently did the 301-redirect, those things take time to get sorted out. (Since MelonCard was still a relatively young brand, a 301-redirect was harmless, but if you had an old brand/domain, a 301-redirect would be SEO suicide) You're still in the index. Your blog is ranking for your brand. You haven't been nerfed out of the blue, so stop worrying and be patient.

epoxyhockey 14y ago

Also, your redirection is weird.. You have a 302 on http://meloncard.com http://meloncard.com, that forwards to https://meloncard.com https://meloncard.com which then 301 redirects to safeshepherd.com If I were moving a domain, I would have 301'd http://meloncard.com http://meloncard.com to safeshepherd.com right out of the gate. Too add to everyone's anecdotal experiences, I 301'd a domain last week and the new domain only took a couple of days to show up in search. I was a happy customer.

SpiderX 14y ago

Buy ad words or quit complaining about it. You act like you are entitled to search ranking. You aren't.

TomAnthony 14y ago

Hopefully I can help some. :) I see a couple of problems with your redirects: 1) http://www.MelonCard.com http://www.MelonCard.com uses a 302 redirect to the https version. You have inbound links to that domain pointing to the http version, so the 302 negates the following 301. 2) Furthermore, your 301's seem to be implemented incorrectly. It works for me in Safari/Chrome, but if I use curl on the command line, or any crawling software I see an infinite redirect: www.MelonCard.com/?from=shadow&from=shadow&from=shadow..... This is likely interrupting Googlebot's crawl too. Certainly worth a fix! A couple of other bits to note: 1) The redirect in Chrome sends me to: www.safeshepherd.com/?from=shadow which Google believes could be different to: www.safeshepherd.com/ You should add a rel=canonical meta tag to the page to help Google out. 2) Your internal links point to safeshepherd.com without the www., but your MelonCard redirect redirects to the www. version. You should probably make this consistent, and also 301 one version to the other. This latter points may seem picky, but Google can be troublesome with this. Hope this helps! :)

epoxyhockey 14y ago
Since the OP is using nginx, here is how you properly do a 301 redirect straight from nginx.conf (in the 'server' section for meloncard.com): rewrite ^/(.*)$ http://safeshepherd.com/$1 permanent;
- TomAnthony 14y ago
  Nice tip. Perhaps with https instead of http in the redirect URL, though? :)
  - epoxyhockey 14y ago
    True! It does look like Google will accept https as a canonical homepage link. They do it for Twitter.
- k33l0r 14y ago
  That's actually more complicated than it needs to be, see http://wiki.nginx.org/Pitfalls#Taxing_Rewrites http://wiki.nginx.org/Pitfalls#Taxing_Rewrites Bad: rewrite ^/(.*)$ http://domain.com/$1 permanent; Good: rewrite ^ http://domain.com$request_uri? permanent; Better: return 301 http://domain.com$request_uri;

pierrefar 14y ago

Hi I work at Google helping webmasters like this. As far as I can tell, there are a couple of interacting issues we're seeing on the site that can be causing what you're seeing. It's a bit technical, but it's easy for you to implement a fix. Firstly, our algorithms recently have been picking one of the following URLs as the canonical URL for the homepage: http://www.safeshepherd.com/ http://www.safeshepherd.com/ https://www.safeshepherd.com/ https://www.safeshepherd.com/ https://safeshepherd.com/ https://safeshepherd.com/ For example, I see that the non-HTTPS pages redirect to the HTTPS pages (e.g. http://www.safeshepherd.com/ http://www.safeshepherd.com/ to https://www.safeshepherd.com/ https://www.safeshepherd.com/), but the non-www pages do not redirect to the www pages (both https://www.safeshepherd.com/ https://www.safeshepherd.com/ and https://safeshepherd.com/ https://safeshepherd.com/ return content). When we find the same content on multiple URLs like this, our algorithms pick one representative URL, and over the past few weeks the choice has been changing. As of 3 days ago, the current choice is https://safeshepherd.com/ https://safeshepherd.com/ . As it stands, our algorithms are trying to figure out the right canonical URL format, but it's difficult in this kind of situation. You can help by redirecting to your preferred URL format (say https://www.safeshepherd.com/* https://www.safeshepherd.com/*), and our systems will pick up this signal, and that will be reflected in the search results and reporting in Webmaster Tools. Secondly, Webmaster Tools treats these as different sites. For example, you would need to verify and check the statistics of both https://www.safeshepherd.com/ https://www.safeshepherd.com/ and https://safeshepherd.com/ https://safeshepherd.com/ (as well as the HTTP versions) as they're separate sites. It may be that you're checking (say) the stats for http://www.safeshepherd.com/ http://www.safeshepherd.com/ but if our algos have picked the https://www.safeshepherd.com/ https://www.safeshepherd.com/ URLs as canonical, the search queries of the former will suddenly be closer to zero but the latter will be a more accurate reflection of the site's traffic. Hope this helps, Pierre

Codhisattva 14y ago
Wow! Kudos to Google putting someone out in the wild answering these kinds of questions. Is there a normal place to ask for web/index help or does HN serve that purpose?
- 14y ago

nicksergeant 14y ago

Um, still #1 for me: http://cl.ly/371t2b0z1S1C0A170y2e http://cl.ly/371t2b0z1S1C0A170y2e

nicksergeant 14y ago
I'm in the US, also (Rochester, NY)

andrewhillman 14y ago

If nothing else works, remove the domain from google webmaster tools - completely. I had two sites that lost top serp positioning and once i removed the domains from google webmaster tools, they went right back up to first 3 results on page 1. Google webmaster tools is more or less designed to help google, not you.

quangv 14y ago

sounds like you got slapped?

Monotoko 14y ago

Still number 1 in the UK too

deleted 14y ago

[deleted]

sevenstar 14y ago

You are number one on duckduckgo.Use social marketing if you really need the traffic.

unreal37 14y ago

Great to see Google responding here. Looks like the confusing 301/302 redirections are the problem. I love HN for this type of community stuff.

lnanek 14y ago
Agreed. Was very impressive seeing Googlers respond, and respond helpfully which is not always easy in tech esp to outside parties, and have forums and regular G+ hangouts so it isn't a one time thing. Not to mention other people with similar problems point out the delay in reapplying the signal, etc..

readme 14y ago

I just signed up for your service. Hope it helped. Your service will certainly help me. The infographic about beenverified was alarming: (https://www.safeshepherd.com/beenverified https://www.safeshepherd.com/beenverified) and I have actually tried to get my stuff off of there before and failed. I'm not even worried about my own information. What I'm worried about is the accuracy of it! How can I trust a third party who doesn't even know me to provide an accurate background check when it references PRIVATE databases that I can't even verify the integrity of? My biggest worry is that some day I will be screwed out of an opportunity because of a company like this that simply provides inaccurate data because they confused me with some other John Doe. It would be one thing to provide public records as a service, but been verified seriously ticks me off. To think they have the authority to 'verify' people irks me. Then again, it may be a good thing, because I sure as hell wouldn't want to work with anyone stupid enough to use a service like beenverified.

robdwoods 14y ago

I have to add my kudos to Matt and Pierre for taking the time to respond to these threads. They could clearly just refer users to the Google Webmaster forums rather than answering in detail here. For all the flack Google takes, I have to say that I'm encouraged that two senior engineers take the time to answer questions in a 3rd party discussion forum.