11 ms·
Suddenly, Hacker News is not the first result for 'Hacker News'
- AretNCarlsen 15y agoOr even on the first page. (I notice because I always get to HN this way. Today is the first time I have ever seen it below #1 or #2.)
- deleted 15y ago[deleted]
- lhnz 15y agoI actually think this might be positive for the quality of the conversation on this site.
- _ndrw 15y agoTrue. No reason not to keep HN on the hush hush, eh?
- billpatrianakos 15y agoI disagree. I mean, we don't want to keep it a secret but at the same time this site has always been really close knit and the people all really just "get it". If HN grows too large I fear for it. The web is full of Cretans and people who think they're thoughts are so worthy they must be heard. Lots of ignorance out there. I've seen some it here already. I learned the culture and studied the etiquette and what qualifies as a useful submission or comment before I really joined in. Not everyone is so thoughtful and it could really sour a once great community. You're obviously new around here so I'm not surprised you'd say that. I mean no disrespect - I do think you'll eventually change your mind though.
- _ndrw 15y agoTrue. No reason not to keep HN on the hush hush, eh?
- cletus 15y agoRaised here already: http://news.ycombinator.com/item?id=3277365 http://news.ycombinator.com/item?id=3277365 FWIW I've raised this issue.
- Matt_Cutts 15y agoThanks, cletus. I'll file one too, just to help make sure people check it out.
- tpr1m 15y agoThat website sucks too :\
- deleted 15y ago[deleted]
- Matt_Cutts 15y agoI think I know what the problem is; we're detecting HN as a dead page. It's unclear whether this happened on the HN side or on Google's side, but I'm pinging the right people to ask whether we can get this fixed pretty quickly. Added: Looks like HN has been blocking Googlebot, so our automated systems started to think that HN was dead. I dropped an email to PG to ask what he'd like us to do.
- deleted 15y ago[deleted]
- moe 15y agoDoes it perhaps return something to the tune of "Unknown or expired link"? :-P
- Matt_Cutts 15y agoNope, it looks like we haven't been able to fetch even the robots.txt file for a few days, so our system is trying to play it safe by assuming that HN is unreachable/dead.
- asto 15y agoWhoooosh!
- pg 15y agoI sent you an email about this. (A couple weeks ago I banned all Google crawler IPs except one. Crawlers are disproportionately bad for HN's performance because HN is optimized to serve recent stuff, which is usually in memory.)
- tibbon 15y agoCould you use some sort of sitemap or other way to provide the data to Google that isn't so damaging to site performance? Or in Google Webmaster tools turn down the rate of crawling? Just realized that this could be a problem for lots of sites, and I'm curious as to what the best solution is, since not everyone has Matt Cutts reading their site and helping out.
- pierrefar 15y agoHi I work at Google helping webmasters. It seems something has been blocking Googlebot from crawling HN, and so our algorithms think the site is dead. A very common cause is a firewall. I realize that pg has been cracking down on crawlers recently. Maybe there was an unexpected configuration change? If Googlebot is crawling too fast, you can slow it down in Webmaster Tools. I'm happy to answer any questions. This is a common issue. Pierre
- joelhaasnoot 15y agoWhy is it that "our algorithms think the site is dead" so soon, yet when I search for a keyword, I still get sites that our dead or no longer contain the given keyword? Bad algorithms or is this "thinking dead" a time delayed thing?
- Matt_Cutts 15y agoNormally there's a lag between when a site goes dead and when we crawl the site to see that it's dead. It's also tricky because you don't want a single transient glitch to cause a site to be removed from Google, so normally our systems try to give a little wiggle room in case individual sites are just under unusual load or the site is down only temporarily.
- yeggeyeggeyegge 15y agoNot a webmaster, but would like to know: Does/can the google crawler system send a notification email to the registered webmaster of a website, if something like this happens? This could be deemed unsolicited, but would in a huge majority of the cases be more than welcome!
- pierrefar 15y agoAll these errors are reported in Webmaster Tools in the Diagnostics section. Verify your site and you'll have access to this and a wealth of more data. Also, we do send notifications in Webmaster Tools, and you can confgure those to be delivered by email too. I'm not sure if we send messages for these kinds of serious crawl errors, so I'll need to check. If not, that's an interesting idea I can ask the team to think about. Thanks for the feedback :) Pierre
- markerdmann 15y agoHas Google been making significant changes to the search ranking algorithms in the past couple months? I've noticed a significant decline in the quality of results, to the point that (for the first time in years), I've bounced over to DuckDuckGo or Bing to try my luck there. I love Google as a company, so (if this isn't just in my head) I'd love to see things get better again. EDIT: Looks like the change to HN's ranking is related to a change that pg made, so my comment is now less relevant to the parent post. I still stand by it, though. :-)
- Matt_Cutts 15y agoIf you can look back in your search history to find the specific searches that didn't do well for you, we're always happy to get really concrete examples. The best reports look like "I did a search for [Bavarian red widgets] and the results weren't good because e.g. you were missing a specific page X that you used to return or should return, or you returned Austrian red widgets" or whatever. Lots of Googlers are clearly hanging out on HN over Thanksgiving while they're stuck at relatives' houses. :)
- markerdmann 15y agoThat's helpful, thanks. I haven't kept a record of those searches (they were mostly programming-related queries), but I'd be happy to submit reports in the future. Is there a URL or email address for this purpose?
- pierrefar 15y agoAt the bottom of every search results page is a link that says something like "give us feedback". That's your first point of call for reporting bad results, spam, etc. If you have a whole bunch queries that are somehow related, like say we do a bad job at Python function names, then you can collect them and post them on our forums in one go. And please give us details as Matt said. In particular, If your refer to a specific result, please say its URL not "5th result" because rankings fluctuate.
- alpb 15y agoI remember times a few months ago when Hacker News was not the first result again. Just wanted to point this out.
- w1ntermute 15y agoOut of curiosity, how did the OP even find out that this was going on? He appears to be a long-time user of the site, and therefore would have no reason to Google "Hacker News". The most common other reason would be that some people use Google as their URL bar - instead of typing "hackerne.ws" or "news.ycombinator.com" into the URL bar, they type "Hacker News" into Google and click on the first result. However, I would've thought that the types of people using HN would have the tech savvy to use a keyword bookmark, or at least the URL.
- narcissus 15y agoCould've been from telling a friend "hey yeah... found this on Hacker News". Then the friend went looking for 'Hacker News' on Google and came across the current top result. At that point that friend asks the OP: "by Hacker News, you mean <this other site>, right?". Then the OP goes looking...
- yonran 15y agoWhen I'm on someone else's phone or computer, I just search for [hn] to geT here quickly.
- pbhjpbhj 15y agoSometimes I want to find HN but am using a different device, sometimes that device is also a touchscreen and typing is a pain. Usually Google is very close to hand and the autocomplete saves typing out a full address or full set of keywords. So I'd only need to type "hacke" in google and click "I'm feeling lucky" in the drop down rather than type "http://news.ycombinator.com/ http://news.ycombinator.com/ into an address bar. Didn't know about the hackerne.ws domain name however.
- 9ec4c12949a4f3 15y agoThat's why you use duck duck go. http://duckduckgo.com/?q=hacker+news http://duckduckgo.com/?q=hacker+news
- hahla 15y agoDoes anyone else find it disturbing that google employees are bending over for pg/hn? Seriously, if any other webmaster blocked googles bots they wouldn't change their algorithms to accommodate, or see how they could use less of our resources. Its pg's fault not googles, and I dont see why they should care. Maybe from their standpoint it would be more beneficial to google users who are used to typing in 'hacker news' to visit this site, but since when did that matter to google? Also don't get me wrong I love both google and hackernews. I just find whats going on in this thread interesting..
- unreal37 15y agoIt's not disturbing at all. I am glad Google is a company made up of humans and not uncaring robots. Human nature is that people want to help others when they are in a position to do so. I would do the same if HN needed my help. Of course, there are people who want to take advantage of people like Matt to give themselves an economic advantage, and I have no sympathy for those people if they can't get someone in Google to help them as easily. It comes down to karma.
- felipe 15y ago> I am glad Google is a company made up of humans and not uncaring robots That's exactly the problem. Google is notorious for providing horrible customer support, so why some people get personalized help, while the rest of us are stuck with the uncaring robots?
- trickjarrett 15y agoGoogle does have bad customer service when they're working through the normal channels. But Google Employees are awesome. I got personal attention from core G+ devs when I broke my G+ profile. The point is if you can build relationships with Google employees rather than with Google itself then you too can get this sort of amazing help. There's nothing sinister or crazy about this. Rather than being one in a million, become one in a circle of friends or acquaintances.
- 15y ago
- williamle8300 15y agoI want to "Learn ethical hacking training."
- braga 15y agoStay away from venture capitalists ;)
- StatHacking 15y agoThis is an example of why Goog's search algorithms (and others') should be open: http://news.ycombinator.com/item?id=3268371 http://news.ycombinator.com/item?id=3268371 A subtle attack may be by making bots stop indexing it or using SEO practices to lower it enough so it would become unsearchable, and therefore, non-existent. Or just crack into Google...
- StatHacking 15y agoUPDATE: http://www.itworld.com/software/228393/free-software-activists-take-google-new-free-search-engine http://www.itworld.com/software/228393/free-software-activis...
- deleted 15y ago[deleted]
- ludwigvan 15y agoA few months ago, I was talking about the Stanford ai/ml-classes to one of my friends. He asked me if there were any more classes and I said "Yes, take a look at the front page in Hacker News, there are some links. I visit that site frequently, it's very helpful." There was one little issue though. The poor guy didn't know what hackernews was, so found that site (hackernews.com). He then scanned the Twitter stream over that site several times to find those links and started visiting the site for several days to find those other helpful links. When I saw him again a few days later, he told me: "What a silly site HackerNews is! And I couldn't find the links to those classes over there." He also told me that he was disappointed of me for visiting such a silly site. Now, can you guess the look on his face when I told him that he was visiting the wrong site for the last few days?
- tillk 15y agogreat story, bro
- kizel 15y agoYeah, I usually get here that way, but today was the first time I haven't seen it on the front page, let alone first result.
- test5625 15y agoDon't link to Google search results! It's personalized - everyone sees different results. Even if you don't have a Google account. For me http://news.ycombinator.com http://news.ycombinator.com is the top page. But when I use TOR, http://www.hackernews.com http://www.hackernews.com and http://thehackernews.com/ http://thehackernews.com/ are on top. I don't think it's possible to get a real "invariant" result page. It all depends on which computer you use (cookies, language setting, ip address).
- waitwhat 15y agohttp://googlesystem.blogspot.com/2007/04/how-to-disable-google-personalized.html http://googlesystem.blogspot.com/2007/04/how-to-disable-goog...
- lowglow 15y agoMatt Cutts is my hero. I've never seen anyone from google (or any company) be as proactive in explaining, interacting, and helping this community. Thanks.
- ammarkalim 15y agopatio11, do your magic...for the HN community!
- baby 15y agoIt has never been the first result for me.
- TastyFish 15y agoOh shit, now I'm gonna have to remember the url.
- skillachie 15y agolol, i checked google twice, then checked to make sure it was actually google that I was searching "Hacker News" in
- deepkut 15y agoScrew it--don't crawl HN. Keep the community small and passionate :)
- chorola 15y agoYes.What's wrong?
- Craiggybear 15y agoStill number one via DuckDuckGo. Google is now officially useless.
- noduerme 15y agoDDG is using Bing for most general web results. My site mysteriously disappeared from Bing and DDG a few days ago at the same time; but Gabriel said if I'm not cool enough to get a top ranking on HN for six hours to complain about it, I'll just have to go through the normal process Bing sets up for webmasters.
- mangodrunk 15y agoYour comments are officially useless.
- yllus 15y agoI definitely didn't expect to see my own blog as the last result on page one. Or is that because I've shared it via Google Plus and it's being injected into my personal results?
- rplnt 15y agoNor second or anywhere in the top five (seven it is).
- mmaunder 15y agoPG: welcome to the woes of being an alexa top 1000 site with over 1 million pages of dynamic content. HN has roughly 1.3 million pages indexed by google. 1.3M pages at 43k per page is 53 gigs to cache static versions of all pages on the site. Quadruple that for a worst case scenario and it'll still easily fit on a single drive. When your site gets this popular you tend to have to re-architect your application to solve perf issues. You could serve googlebot UA's 1 week old cached pages for example. I'd encourage you to start thinking of yourself as a utility providing a valuable and necessary resource to the Net and take the time and energy to solve this properly.
- resnamen 15y agoI'd like him to fix the experience for non-bot users first, namely the "Unknown or expired link" issue.
- BadassFractal 15y agoI don't think it's all bad, we don't want this community to turn into another reddit.
- TylerBrock 15y agoWho cares, let paul graham deal with it.
- gaza3g 15y agoI think it makes a a lot of sense for Matt Cutts to intervene in this case since: 1) Matt browses HN. 2) HN is a high-volume site and whatever suggestions that were discussed and implemented here can be noted and learned by everyone else.
- bezza1 15y agoBottom line is that if you don't trust Google & use their tools (GWT) then you can end up in sticky situations like this one. My experience with the Crawl Rate feature via GWT is that they do honour it pretty strictly, but for large sites Gbot can cause a lot of extra load even if pages are static. A good CDN and stateless cache server will help but for sites as large as HN every request adds up!