10 ms·
Does anyone else find it disturbing that google employees are bending over for pg/hn? Seriously, if any other webmaster blocked googles bots they wouldn't chang
by hahla 15y ago
Does anyone else find it disturbing that google employees are bending over for pg/hn? Seriously, if any other webmaster blocked googles bots they wouldn't change their algorithms to accommodate, or see how they could use less of our resources.
Its pg's fault not googles, and I dont see why they should care. Maybe from their standpoint it would be more beneficial to google users who are used to typing in 'hacker news' to visit this site, but since when did that matter to google?
Also don't get me wrong I love both google and hackernews. I just find whats going on in this thread interesting..
- unreal37 15y agoIt's not disturbing at all. I am glad Google is a company made up of humans and not uncaring robots. Human nature is that people want to help others when they are in a position to do so. I would do the same if HN needed my help. Of course, there are people who want to take advantage of people like Matt to give themselves an economic advantage, and I have no sympathy for those people if they can't get someone in Google to help them as easily. It comes down to karma.
- felipe 15y ago> I am glad Google is a company made up of humans and not uncaring robots That's exactly the problem. Google is notorious for providing horrible customer support, so why some people get personalized help, while the rest of us are stuck with the uncaring robots?
- trickjarrett 15y agoGoogle does have bad customer service when they're working through the normal channels. But Google Employees are awesome. I got personal attention from core G+ devs when I broke my G+ profile. The point is if you can build relationships with Google employees rather than with Google itself then you too can get this sort of amazing help. There's nothing sinister or crazy about this. Rather than being one in a million, become one in a circle of friends or acquaintances.
- billpatrianakos 15y agoI see your point and maybe my attitude toward it is totally biased or morally/ethically off balance but I think it's just fine. PG knows his shit. He knows how to block crawlers, he knows how get SEO done, he's no dummy. "So what?" you say. Well I work in web development and I meet with people all the time and explain to them how to run a site and get rankings up, etc. I tell them I'll give them the tools, show them how to use them and even leave detailed manuals on how to work everything. These people are so into it at our first meeting and they're excited to get into it and work this stuff. Then the job is done, the tools are handed over, and the lessons are learned. What do they do? Nothing. Then they do nothing. Then even more nothing. The site is never updated and they do no maintenance whatsoever and eventually it becomes outdated and dead for all intents and purposes. A year later they want to know why they're not ranking in Google. I ask if they've checked or even once used any of the tools I gave them and the answer is always no. Then they demand I fix it. I kindly remind them they're the ones who wanted to have Larry in accounting run the site because he's "good at computers" and to save money by not hiring an IT guy. So the point I'm making is that I'm sure there are a ton of people out there who are trying to get customer service without trying to help themselves or using the resources readily available. I think it's great that Google is showing they're willing to help those who help themselves. I think google isn't treating everyone equally but they are treating people fairly. There's a difference. I'd prefer fairness. Not all "customers" are created equal so you compensate by treating them with fairness instead.
- wmf 15y agoPG knows his shit. He knows how to block crawlers, he knows how get SEO done, he's no dummy. Apparently he doesn't use Google Webmaster Tools, though.
- billpatrianakos 15y agoIt seems he doesn't want to. From what I understand there's the possibility that he'd prefer to accomplish his goal without the use of Wemaster Tools. Like I said, he knows his shit, so he's going to do things on his terms if he can. Webmaster Tools might be what the average guy wants to use but then there are some who follow the beat of their own drummer whenever possible for whatever reasons we may never know.
- jaylevitt 15y agoActually, much as I've blasted[1] Google over the years for not doing anything (like abuse-desk staffing) that requires People Not Algorithms... Giving occasional personalized help is a great way to avoid a total black hole while still creating an escalation path. It's like having EXPLAIN QUERY stack traces on NewRelic. You don't need a deep trace of every transaction; you need a deep trace from a few representative squeaky wheels, and you use that information to improve the underlying self-serve process. This is something Google, with their "1/3 of the company parses logs", excels at. You'll note neither Matt nor Pierre offered to do anything special for PG. They didn't say "Oh! I love HN, and we want it to succeed; I'll go push out a manual override that makes sure we use only the one crawler you unblocked." They gave him exactly the info he'd get from Webmaster Tools plus general knowledge, and told him how to use Webmaster Tools to fix it. Maybe they'll go figure out how to implement that "you're blocked!" hover-over in SERPs, but again, that would benefit everyone, not just HN. There's no special treatment here. [1] http://blog.wordtothewise.com/2008/04/troubleshooting-a-postini-block/#comment-865 http://blog.wordtothewise.com/2008/04/troubleshooting-a-post...
- 18pfsmt 15y agoTo be fair, I've seen Cutts jump in on "no-name" threads where there is an issue with Googlebot or some search-related thing. It does not appear to be favoritism, beyond choice of forums in which to participate.
- billpatrianakos 15y agoFairness is what it's all about. I like yiur choice of wording. A lot of people have a problem with Matt Cutts jumping in because they feel it's not fair. They're mistaking equality for fairness. This is perfectly fair but not equal. Equality sucks. Fairness is better. Fairness is treatment based on merit and equality is just equal treatment no matter what the situation. Equality can really screw you if you're on the wrong side of it.
- zerostar07 15y agoDisturbing yes but not surprising, HN is influential. Now whether it's hacking news or entrepreneur news is another issue.
- Kiall 15y agoI wouldn't call offering help/advice to a site you use as "bending over", I reckon any Google employee with access to the right tools would look into why one of their favourite sites has disappeared from the index.. Oh and, It could also have been a sign of a pretty major bug, considering the popularity of HN ;)
- AznHisoka 15y agoYes, it's VERY disturbing, no question about it. Matt can sound all helpful, and cuddly, and oxytocin-inducing all he wants, but he's basically stepping in and helping 1 site over another, when the whole process should NOT favor anyone. There are lots of webmasters out there that do stupid stuff like override a robots.txt accidentally, yet I don't see Matt sending them an email, asking them to check up on it.. Come'n... this is lame!
- apetresc 15y agoAre you kidding? Matt is basically a superhero who helps anyone he can find. He's solved problems for people complaining on Twitter, on Google+, he hosts regular video office hours when anyone can come to him with problems, etc. It may be the first time YOU'VE seen him rush in to help someone, but this is Matt Cutts' standard MO.
- AznHisoka 15y agoNo, he's not a super hero.. and no he doesn't help everyone that asks for it. I've brought up many issues over the years to him via Twitter, blog comments, emails, etc, and none of them.. count them.. ZERO have been addressed by him. He just gives preferential treatment to those sites that make a lot of noise, and might give Google or Google's quality team a lot of negative press. That's all he does. I've even brought up 100% apparent, clear spammy sites and he doesn't respond, and doesn't do anything about them. They're still there. Lots of folks here like to give Matt Cutts this aura of an angel, or some sort of saint. Those who have known his actions over the past 5-10 years know better. Just ask guys like Aaron Wall and Rand. Matt Cutts is just a pawn that is there to do damage control for Google. That's all he is.
- Matt_Cutts 15y agoHi AznHisoka. A lot of people send me spam reports, and I'm happy to pass them on. But normally we investigate the spam reports without sending back specific feedback of what action we took in response. P.S. Just curious--is this the same AznHisoka from BlackHat World, the "Blackhat SEO forum"? http://www.blackhatworld.com/blackhat-seo/members/137345-aznhisoka.html http://www.blackhatworld.com/blackhat-seo/members/137345-azn...
- pierrefar 15y agoFrom my personal point of view, I help webmasters when I see an issue. It's part of my job and it's something that gets me fired up. I help when I go on our forums, I do that here, and I'm regularly helping webmasters on Google+ (my team does regular webmaster hangouts that anyone can join - see my profile: https://plus.google.com/115984868678744352358 https://plus.google.com/115984868678744352358 ) and also on Twitter. We can't cover everything, we can't be everywhere, but when we come across something we can help with, we try to help! Another important point about this thread: this is a very common issue that regularly comes up, and no site is immune from it. I regularly see major sites have a firewall that auto-configures itself to block Googlebot, and the webmaster doesn't know what's going on. Raising awareness about this problem and how to fix it adds to the importance of replying.
- novedieci 15y agoAnyone know if this is also a common issue with BingBot? Our site is doing fine in Google at the moment, but was completely deindexed by Bing, and we're totally baffled.
- 1010101111001 15y agoWas the query for a domain name or just hacker+news? How do we know? Does <title> make any difference? I don't think of this site as "Hacker News". I think of it as ycombinator, and the subdomain, news. Should users of hackernews.com think of that site as something else, e.g. whatever is between the title tags? A searchable list of domain names, ranked by popularity. Or even a searchable list of main page titles. Is that how some users are using Google? If so, Google does not need a full, current cached copy of the crawlable web to provide that.
- tripzilch 15y agoUnsure. On several occasions I got the distinct impression that a lot of members aren't actually interested in "hacking" (for any of its definitions), instead you get people asking about the value of jailbreaking a phone: http://news.ycombinator.com/item?id=3169000 http://news.ycombinator.com/item?id=3169000 I suppose it is because the top hits below HN are perhaps even less about "hacker news" ?
- ig1 15y agoIf you use Google Webmaster Tools it will in-fact tell you all of this information without having to ask someone at Google.
- MarkMc 15y agoInteresting, but not disturbing. I think it's just a consequence of people at Google initially reading HN for the stories, then finding that the HN community forms a useful 'brains trust'. I for one am glad to see that HN has a hotline to the Kremlin ;)
- njloof 15y agoIf you don't know somebody that works for Google at this point in your career, you're working in the wrong industry.
- xd 15y ago~30k employees at google .. what makes you think they hold all the keys to the millions of web developers across the globe?
- umarmung 15y agoDamned if you and damned if you dont. All the people in this thread, including Google-related users, are trying to do is figure out what changed. That could just as easily have been at Google or HN's! It could also just as easily been a correct change. If it turns out it was something HN changed and they did not use or even aware of the proper Google tools available for everyone to optimize their site, then I hardly think any reasonable person could object if it is pointed out to them - even if it's by Google personnel.
- blhack 15y ago>Does anyone else find it disturbing that google employees are bending over for pg/hn? Umm...no. They're not tweaking the algo, they're explaining to PG that he should stop blocking the crawlers, or that he should verify the site in google webmaster tools, then change the crawl rate.
- zem 15y ago> Maybe from their standpoint it would be more beneficial to google users who are used to typing in 'hacker news' to visit this site, but since when did that matter to google? from google's point of view, a user who types "hacker news" into google is almost certainly looking for news.ycombinator. therefore, it is highly desirable for the site to be the #1 result.
- product50 15y agoBaseless comment. If they don't help, you will say - HN is one of the top sources of tech news in the world and it is surprising no one from Google is responding to such an obvious mistake. If they help, you comment as stated. Secondly, HN is one of the most popular sources of tech news. By responding to this issue, Google is helping bring out possible solutions to a common problem - that crawlers are too frequent and affect the performance of a site. People like you and me can read the responses and learn what can be done to control this. Third, popular websites like HN deserve attention like this because of the following they have garnered over the years. If,say "hahla blog" doesn't show as a top result in Google - it will take time for Google to verify whether it should even be shown as a top site. But, if something like HN or Amazon.com doesn't show as top result when users explicitly searches for it - that is a darn good use case for Google to fix the issue. It is an indication of something is definitely wrong somewhere that needs to be fixed.
- moses1400 15y agoCompletely agree with you Hahla - amazing to see the Googlers jump in including Matt Cutts so quickly to support PG and HN and YC yet 99.99999999% would never ever see support like this.
- billpatrianakos 15y agoLet's be real here. Do 99.9999...% of users need or deserve this kind of support? It's easy to have sour grapes and say "oh the big shots get attention but I don't! Not fair!". But then you have to remember that people like PG worked their ass off to get a site like this to become so valuable and popular. We're not talking about some guy's blog about his cat that no one cares about, we're talking about HN here. Is it equal treatment? Of course not but is it fair? I say it is fair. If fairness is treating people based on merit then it's totally fine. Matt isn't in here saying he's going to manually manipulate the result or change anything on Google's end (save for maybe correcting a flaw that would benefit everyone who's in a similar situation as HN). All he's doing is suggesting possible causes, trying to make a diagnosis, and just troubleshooting. Most people who want this attention don't deserve it. They're the type of people who won't use the resources available like the google help documents or even learning how to administrate a site for best results in search engines. Plus, most people's website just aren't worthy of this attention. I could go on but lets look at the reality here. We're all knowledgeable of how the web works here so let's just admit that HN is a damn popular site and it definitely seems like a fluke to have it rank as it was when this was submitted.
- csallen 15y agoWhat's more interesting is that people ONLY feel the way you do when the company in question is big. If it's a tiny startup doing some sort of unscalable marketing or customer support, nobody blinks an eye. But if a Google-sized organization does it, it's shocking or disturbing. I think your best bet is to realize that, regardless of size, companies are always made up of people. And people are opinionated, subjective, and prone to making decisions that fall outside of some standard set of rules.
- brown9-2 15y agoWhat algorithm changes?