8 ms·
Jason Calacanis Knows He's Spamming Google, He Just Thinks It's No Big Deal
- dangrossman 17y agoWhile it's always a bit annoying to see a meme take up article slots on Hacker News regularly, I don't mind this one. I see sites like Mahalo as doing massive disservice to huge numbers of people -- * thousands of AdWords advertisers that have paid for their ads to be matched with content sites, not scraper pages with a huge ad to text ratio * thousands of publishers whose work is being scraped, aggregated and outranked without as much as a backlink * millions of web searchers that are hitting these pages instead of the real sources of the content they were searching for And calling out companies that harm the fabric of the web for everyone else is worth doing.
- aresant 17y agoHere's one thing to consider : Google's algorithms for detecting the actual value of clicks for advertisers has improved greatly over the past few years. If Mahalo’s traffic was utter crap, it would be dropped.
- mvandemar 17y ago"If Mahalo’s traffic was utter crap, it would be dropped." That's not even close to true. If it were, there would be no AdSense for parked pages program. Their eye is still very much on their own bottom line, not the advertisers.
- hernan7 17y agoThere's also the possibility that aggregator sites are actually useful for some portion of the Internet users (read: the "facebook login" set) and the advertisers that cater to them. Too bad "we" have to share Google with "them".
- aaronwall 17y agoHere is a link about how search monetizes when compared to content sites (~ 30 to 1!) http://www.emarketer.com/Article.aspx?R=1007053 http://www.emarketer.com/Article.aspx?R=1007053 As an extension of that, sites which get most of their traffic from search often have the highest ad CPM values because the ad targeting is so relevant on the AdSense ads. And this is true even moreso now that Google is also using the referrer for ad targeting http://adsense.blogspot.com/2010/02/better-contextual-matching.html http://adsense.blogspot.com/2010/02/better-contextual-matchi... Scraper sites do quite well on the ad revenues front because most of the traffic is search driven, and because the content ends up so bad that the ads look all the more appealing.
- aresant 17y ago"Their eye is still very much on their own bottom line, not the advertisers." I know for a fact that Google is balancing advertiser ROI against advertising spend and ensuring that they don't upset the apple cart. However you are absolutely right that Google is maximizing their profit – having the ROI numbers in hand lets them drive down overall sales-from-adsense margins while still keeping the customer. So my original point is that if Mahalo was flat out non-performing crap, it would be tossed. An argument could be made though that they use the Mahalo stuff to grow ad inventory, while balancing out the ROI to their advantage since it likely converts on the lower end of the scale.
- dangrossman 17y agoMy conversion rates from the content network have never been as high as search. I try new campaigns targeting both networks every once in a while but always end up either opting out or setting a much lower bid on the content side.
- mwcremer 17y ago[citation needed]
- qeorge 17y agoIts really a shame how low Google's standards are for AdSense. I've tried advertising in the content network several times, but each time I end up wasting so much time blocking MFA sites that I give up.
- wheaties 17y agoThe rub is, if Google were to block all of Mahalo, a lawsuit would surely be filed.
- sjsivak 17y agoI do not think anyone is asking for all of Mahalo to be blocked. The major issue is the insane amount of auto-generated pages that are not given a noindex call and are also added to the site map xml file. These actions appear to be conscious choices to make sure that google will index these pages, which according to several sources, have nothing but "scraped" content and are therefore spam.
- jonknee 17y agoAnd Mahalo would lose. Google penalizes SEO spammers hundreds of times every day. It's very easy to see that Mahalo does not follow the rules.
- seldo 17y agoGoogle is providing a free service in a field in which there is intense competition. I don't see what basis anyone would have for filing a lawsuit for not being indexed in Google.
- moultano 17y agoRegardless, it still happens all the time.
- mars 17y agoagreed.
- deleted 17y ago[deleted]
- smallblacksun 17y agoThey could argue that Google has a monopoly on online search, and that excluding people from their search results is an illegal restraint of trade. I don't think that such a case would be a cut-and-dry victory for Google if the spammer (as in this case) has the money to fight them.
- w00pla 17y agoThe question is: why doesn't google do anything about this? There are countless such pages. Why not just blacklist domains? Or artificially give them lower pagerank? Or is it okay as long as they get ad-sense money?
- vaksel 17y agothey do it all the time, as long as it's a small site, Mahalo's existence basically flies in the face of all of Google's rules and regulations
- fnid2 17y agoAn old boss told me, "When you're the man, the rules don't apply to you." Not to say that Calacanis is the man, but to illustrate that rules are there to suit the rule makers and in this case, it suits the rule makers to ignore their own rules.
- gcheong 17y agoSo why didn't Google nip Maholo in the bud when it was starting out? Insider connections?
- vaksel 17y agobecause when they launched they were being touted as a search engine, not a content farm
- skinnymuch 17y agoThey went through a bunch of different iterations and stuff before they established as the site they are now. And look at the funding of Mahalo and Google. Same early people.
- noodle 17y agothey do block domains, at least from their organic search engine results. i accidentally got one of mine blocked when playing around with some random software that produced a site that was somehow against their TOS. i didn't have it live long, but long enough for the spiders to grab and flag. still waiting on a reconsideration request. why not mahalo? who knows. if i had to venture a guess, i'd say that the larger the site and the more revenue google sees from that site due to adwords and the like, the higher up the chain the decision to block it has to come from.
- access_denied 17y agoYou frogot the search engines; search results quality is lowered (at least until they work against it).
- danik 17y agoAlthough I understand your points the AdWords and search issues is googles problem. People pay for a product with flaws in it and google is selling, not Mahalo. Mahalo has the freedom to do whatever they want with their pages and they can't be held responsible for what google in it's turn does with them. Now, stealing other peoples work, that is obviously Mahalos doing and should be a case for the court. I'm not really sure what the point with this article is. Mahalo spams google - so what? It's up to them to do this and up to google to prevent it. Business is business, even on the internet.
- axod 17y ago>> "Currently when I look, Google tells me that Mahalo has 356,000 pages indexed" I see 'Results 1 - 10 of about 2,200,000 from mahalo.com' Have things stepped up a gear or am I misunderstanding?
- mvandemar 17y agoNo, different datacenters will show different results... sometimes very different. It also matters if you are visiting Google.com or one of the country variants. According to what Jason said in another comment, however, all of their pages are listed in their xml sitemap, and all of those are listed in a master xml sitemap index located here (warning! huge files if you follow the links in the first one!): http://www.mahalo.com/sitemapindex.xml http://www.mahalo.com/sitemapindex.xml Based on what I saw 2 million+ looks like a huge overestimate, if what Jason said is true. Edit: My bad, frederickcook's answer was the right one. I didn't realize you were doing a regular text search.
- chronomex 17y agoThere are 12 sitemap-mahalo.xml files, each with 50,000 urls in it. The 12th has 48,661 urls by my count. That gives a total size of 598,661 pages. Methodology: wget http://www.mahalo.com/sitemapindex.xml cat sitemapindex.xml |sed -e's/[<>]/\n/g'|grep ^http|xargs wget for P in sitemap-mahalo.xml* ; do echo $P ; cat $P |sed -e's/[<>]/\n/g'|grep ^http|wc -l ; done Edit: Formatting
- frederickcook 17y agoSearch "site:mahalo.com" Simply "mahalo.com" lists every indexed page with that text on it, such as this one.
- axod 17y agoYup that's what I did. Maybe the particular google datacenter I hit had a high estimate there. It also says 59,300,000 for scribd.com which I can't help but draw comparisons with this.
- mcutts 17y agocan't do anything about Mahalo's spam, it's off limits
- pclark 17y agohow so?
- thehodge 17y agoI doubt this is the real account, created 37 minutes ago..
- mcutts 17y agoSequoia Capital backed both Google and Mahalo
- prodigal_erik 17y agoI have trouble believing post-IPO Google is still so beholden to Sequoia that they must tolerate "publishers" poisoning their well.
- skinnymuch 17y agoSure if Mahalo was really screwing with Google, Sequoia backing wouldn't help, but to say Google + Sequoia aren't incredibly buddy-buddy would be difficult to say.
- mvandemar 17y agoCan't you just see Matt Cutts, sitting in his office, connecting to sites like HN through multiple proxy levels creating fake Matt accounts so he can leave truthful comments without his bosses at Google finding out... and when they question him he can just say, oh, of course that wasn't me!
- icey 17y agoI think it's safe to say that mcutts is an impostor account. As far as I can tell, the actual Matt Cutts is here: http://news.ycombinator.com/user?id=MattCutts http://news.ycombinator.com/user?id=MattCutts
- CoachRufus87 17y agothis is getting very, very old. can we move on? please??
- prawn 17y agoIt's obviously of interest and importance to a number of people involved in this field or troubled by poor quality material showing up in Google. If you're not one of those people, it's pretty easy to identify these links and not upvote them or visit the articles/comments, etc. There are countless articles on HN that I have no interest in (e.g., I don't even know what Clojure is), but I just don't click through to them.
- moe 17y agoOne of the things I really miss in google is a persistent blocking preference a.k.a. site blacklist. Mahalo would go straight in there, along with expert-sexchange, sedo parking and a few others.
- ryoshu 17y agoDepending on the browser you use, this is easy to do. I have experts-exchange and other useless sites blocked using greasemonkey/greasemetal.
- mvandemar 17y agoYeah, sedo parking is a bitch tho, since by it's very nature it's tons of different domains. You would need a way to block by app.
- pclark 17y agoa year on, maybe not so crazy now eh? http://news.ycombinator.com/item?id=485618 http://news.ycombinator.com/item?id=485618
- jachee 17y agoSEO is snake oil. This is further proof that the whole "industry" is ruining the integrity and usefulness of the internet.
- lmkg 17y agoNot everything that uses the phrase "SEO" should be painted with the same brush. A lot of SEO best practices are making the HTML markup more semantic and human-readable. White-hat SEO dovetails pretty well with a human-readable web and accessibility standards, and is one of the better business cases for kicking the Flash habit. Just because Calacanis makes his name by abusing loopholes in the algorithms doesn't mean that's the only sort of thing that the name "SEO" applies to.
- aaronwall 17y agoThat is the angle Jason used when he created his steaming pile. It doesn't mean that anyone in the industry agrees with Jason's strategy. And if you want to place the blame where it belongs remember that Google is the company funding all this content scraping with their ads programs. I just tried searching for a recent post from an official Google blog (about AdSense using referral data for more relevant ad targeting) and found a scraper site with their ads outranking them for their own content. Pretty sad.
- olefoo 17y agoActually, pretty savvy. Google gets paid for the scraped site because of ad impressions and possible clicks. Google does not get paid for running an informative blog, in fact it's a cost to them. Google cares about getting paid.
- jsz0 17y agoIt's really too bad Google doesn't allow you to simply blacklist domains from your search results permanently. The thing that frustrates me the most about these SPAM sites is the fact they're constantly popping up and my down voting of the result seems to do absolutely nothing unless I'm using the exact same search query. Just let me blacklist Mahalo and other sites like it permanently. Better yet make it possible to subscribe to a blocklist so the community can pool it's resources and fight back.
- Devilboy 17y agoI request this from Google every couple of months. If I could remove an entire domain from all my personalized search results, I'd be soooo happy.
- benatkin 17y agoI've requested this, too. I even switched to Yahoo! for a few hours because they let me blacklist expertsexchange.com (since changed to redirect to experts-exchange.com). I don't see the option on Yahoo!'s page anymore.
- elai 17y agoexpert sex change .com?
- benatkin 17y agoYes. It used to show up in Google search results as expertsexchange.com. After being thoroughly razzed, they made it redirect to experts-exchange.com. Since it was called expertsexchange.com at the time I blocked it in the Yahoo! search results, I took the liberty of referring to it by its old name.
- aaronwall 17y agoWith Google custom search engine you can create your own personalized search service...built off the back of Google, but while delisting any sites you do not want results from. And it only takes about 30 seconds to set one up. Then you embed the search box wherever you like. http://www.google.com/cse/ http://www.google.com/cse/
- jasonmcalacanis 17y agothis is getting really old and we're not interested in doing anything black hat or even gray hat. as such we're doing the following: 1. we're removing (or building out) any page in our system created by our users with under 200 words of original content. This will take a couple of weeks but it's tarted. 2. we're not letting users create stub pages (short pages) until we can noindex them and put them in a different directory (i.e. /stubs/) so google can easily tell the difference between them. these pages are < 1% of our revenue and low single digits of our traffic. we don't benefit from them materially, and I think we're being targeted by Aaron Wall and other SEOs for my "seo is bullshit" comment from 2005 or so. I guess that is fine... I gotta live with the ramifications of what I say. however, for the record I don't believe that SEO is BS any more... when i said that it was when we were building joystiq and autoblog and we spent zero time on SEO. All that being said, we're being targeted by a small group of folks who want to take us down. we're only going to get stronger from this because our hundreds of contributors are rallying around building out the short pages. Topix, Kosmix, NYTimes and Zimbio are all making quality topic pages and are not getting attacked over it. not sure why there is some double standard. regardless.... this is not a material thing for us. we're flushing all these pages and moving them to a different directory going forward so that search engines know where they are located (i.e. /stubs/ ). thanks for the ass kicking.... having a horrible day today over this. jcal http://bit.ly/jasondown http://bit.ly/jasondown
- tdm911 17y agowe're removing (or building out) any page in our system created by our users with under 200 words of original content. This will take a couple of weeks but it's tarted. or i'm also getting a list of every page under 300 words and having the page managers build them out in 30 days or deleting them. from: http://news.ycombinator.com/item?id=1143512 http://news.ycombinator.com/item?id=1143512 is it under 200 words or under 300? are the goal posts moving already?
- jasonmcalacanis 17y ago300 is the goal right now.... we're making plans right now. .... keep the attacks up, only makes us more motivated.
- fjabre 17y agoThink this thread has gotten way out of hand. A lot of the comments in here seem like more of a personal attack than anything else. You might as well change the title of this post to "Jason Calacanis ruined the Internet" or something to that effect.. Can we stop the drama already? I think we're going to need a hose to control this mob.
- japherwocky 17y agoJason Calacanis is the Paris Hilton of the web. Mahalo is not particularly interesting, not particularly evil, he doesn't really do anything, and yet - we keep talking about him and putting his shit on the front page of hacker news. It's a mediocre aggregator/linkfarm, with some mechanical turk style incentives for humans to contribute, and a nice chunk of $ in the bank. Just ignore/mock him for another year or two until the funding runs dry.
- mvandemar 17y agoYou are absolutely right, it is mediocre crap. The issue is that through the spam techniques described in the article, he can now gain undeserved top 10 rankings for many phrases using pages that have no business being there. If you are not part of the web development community then these discussions will most likely bore the hell out of you. However, if you are, and if you are aware of how many innocent sites Google bans or penalizes on a daily basis, or AdSense accounts that get canceled with no appeal for offenses much less than his, then this stuff actually matters.