16 ms·
Google Search Results Plagued with spam “.it” domains
- pyinstallwoes 4y agoAlso noticed here: https://support.google.com/websearch/thread/118733416/lot-of-it-domains-polluting-search-results?hl=en https://support.google.com/websearch/thread/118733416/lot-of... Locked?
- rwmj 4y agoI've been gettng these spam .it domains for years and years, this is nothing at all new.
- politelemon 4y agoWhat does Cloudflare normally do with spam sites, is it a hands off approach or do they do some policing?
- yellow_lead 4y agoHands off till it gets upvoted on HN
- charcircuit 4y agoThey do the bare minimum. You can report sites for abuse and they will take them down. It doesn't appear like they do anything to proactively stop similar sites so the person can just make a new account and domain and be back in business.
- miyuru 4y agocloudflare abuse department is really lacking. Their abuse form is getting abused too. It sends an email to site operator and the server hosting company in single submit so its getting abused. It not even have a captcha. https://abuse.cloudflare.com/ https://abuse.cloudflare.com/
- deleted 4y ago[deleted]
- breakingcups 4y agoThey will do nothing, and it is a feature. The only thing they'll do is forward the complaint to the user. Leaving you with no recourse other than to take legal action before Cloudflare will lift a finger. Unless there's CSAM, of course.
- sammy2244 4y agoYou need to remember that Cloudflare isnt a host
- r1ch 4y agoThat isn't always the case these days, web apps can run on Cloudflare without an origin.
- sammy2244 4y ago
- breakingcups 4y agoI've often ranted about this, because for all intents and purposes they are. They store the content of the website on their drive to serve to visitors. Whatever processes lie in the backend of whatever website to fetch up-to-date content from an upstream source is not my concern. They are NOT a neutral ISP, they are providing a service to their customer which includes hosting (doesn't matter if it's temporary hosting because they expire files). From our point of view, it is their IP addresses that are hosting the website. They have all the responsibilities a traditional hoster has, no motter how they try to frame this debate.
- bad416f1f5a2 4y agoThis… seems about right? A trademark dispute is a civil issue between two parties. We have legal systems to solve these. Cloudflare should ensure that their customers get timely notification of complaints, and that’s pretty much it.
- beardyw 4y agoPresumably these folk take advantage of cheap/free domain offers wherever in the world they are.
- pyinstallwoes 4y agoI've actually experienced this and it is not related at all to the device. It was related to the signed in google account across networks and devices.
- 01acheru 4y agoI think you are right, https://www.register.it/ https://www.register.it/ is offering free domains for 1 year since some time.
- deleted 4y ago[deleted]
- peoplefromibiza 4y agoit's not that straightforward though. to register an .it you must prove you are a person or a business working or residing in one of the EU member states and need to provide the ID of a person who's gonna be listed as admin-c of the domain.
- gsich 4y agoNo you don't. I had a .it domain too, yes there is a field in regstritation where you should enter a "identity card id", but I didn't have one so I entered something random. Worked of course.
- peoplefromibiza 4y ago> No you don't. Yes, you do! of course it worked. you just committed a crime. you can fake your id everywhere in the World, it is a crime everywhere in the world and if something happens doesn't mean you won't get caught. you can drive a stolen car, it will work. > yes there is a field in regstritation where you should enter a "identity card id" so it is required! you simply ignored it, lied and broke the law. your criminal behaviour doesn't imply laws do not exist. if you tried to buy an insurance policy with that fake ID, you would be in troubles now.
- mkl95 4y ago.it may be the .tk of the 2020s
- toastal 4y agoShame too. I was hosting my personal site on .tk when I was broke out of college, but often a link too it was automatically flagged as spam.
- password1 4y agoWhy? .tk was popular because it was free, so it was really useful for teens and young adults in an era when you still had to host things somewhere if you wanted them online. On the other hand .it is the tld of Italy and used legit by all businesses of EU's third largest economy.
- ricardobeat 4y ago.it is currently being offered for free / €1
- peoplefromibiza 4y ago.com are being offered free or €1 as well, but you don't need to be an EU member with a valid EU ID to register a .com https://www.register.it/domains/?lang=en https://www.register.it/domains/?lang=en https://imgur.com/W0XkZIj https://imgur.com/W0XkZIj https://imgur.com/a/p9sFsKj https://imgur.com/a/p9sFsKj
- deleted 4y ago[deleted]
- qalmakka 4y agoJust, no. Plenty of legitimate websites under .it, basically every single Italian company plus all localized versions of international websites (apple.it, google.it, ...)
- jwally 4y agoAnecdotally gmail has been doing a miserable job filtering spam for the last 5 months or so. For me it used to be pretty bulletproof - one of its best features. Now I get something from McAffee Pratners(sic) every other day warning my computer is about to expire. Back in May I kept winning things from Home Depot and Lowes; and gmail would categorize it as "forums". No idea if its related, just odd.
- pwiecz 4y agoOn the other hand over last few months majority of post from sma few of private Google Groups, I'm member of, keep getting wrongly clsssified as spam. I don't know if it's related either.
- sofixa 4y agoI occasionally have spam that slips through Gmail's filters, but when I explicitly mark it as spam it disappears nothing of the same type reappears again.
- silversmith 4y agoAdd another anecdote to the anecdata pile. Past three months, McAffee and north american shopping chain spam is breaking through Gmail filters. And reporting as spam does not help. I assume they've been somehow building Google reputation for the spam accounts.
- edvards 4y agoBeen having the same issue on my old email. a LOT of spam going past the filter.
- thelollies 4y agoSimilarly I've been getting smashed with gmail spam, google calendar spam and google drive spam for around 6 months now after never previously getting any and despite reporting most of it.
- tempest_ 4y agoYeah the drive spam started for me last year or the year before. It took a break but in the last couple months has returned with a vengeance. I had the Slack google drive integration and I needed to mute it because it was couple doc invites every few hours.
- krono 4y agoEvery single one of these results carries the `html` filetype as part of their URL is my experience. This is likely a consequence of the useragent-based switcheroo technique they use to fool Google. Just blanket block the lot with the following uBlock Origin filter: google.*##.g:has(a[href*=".it"][href$=".html"]) Google ain't going to fix itself ;)
- permo-w 4y agoI'm sure there are plenty of non-spam html pages based in Italy too
- krono 4y agoConsidering the crowd that trade-off that seemed too obvious to mention.
- jacooper 4y agoOr use Brave search, which honestly from my experience is much better.
- peoplefromibiza 4y agocool! now s/\.it/every TLD/ and you solved domain spam forever. /s You might not know that 99.99% of .it domains with urls ending up in .html are completely legit, including some official government one.
- nkrisc 4y agoSince uBlock is run on the client, unless you’re Italian or interested in Italian sites it doesn’t really seem like much of an issue. I could block all .it sites on my network and I’d likely never even notice.
- peoplefromibiza 4y agoyeah, right, unless you're american, why should you care about .com domains? ¯\_(ツ)_/¯ the problem is not .it domains, it's clearly stated in the linked post A large number of spam pages are indexed when searching by our product name. It’s very similar to Japanese Keyword hack, but the difference is that our site is not hacked so it's definitely an indexing issue, those .it domains are being indexed for the Japanese word hack for some reason, it's not that .it domains are particularly spammy per se. Your "solution" would filter the vast minority of the abusers at the cost of banning an entire TLD, not much different than turning off the internet connection entirely. Most of the spam on the internet comes from .com domains though, even more so because registering a .com domain is much easier than getting an .it Are you willing to ban .com too?
- oefrha 4y agoWhat can search engines do about user-agent based content differentiation? Say my robots.txt allows Googlebot and nothing else. If Google attempts to double-check with a covert user agent, robots.txt is violated. Assign humans to review reported pages? It’s pretty easy to swamp a manual system like that. Just forget about robots.txt?
- foobarbecue 4y agoI remember reading on HN years ago that Google bots have never honored robots.txt, but I don't actually know
- hombre_fatal 4y agorobots.txt is just a guideline between well-meaning actors for the majority of their traffic, like helping a bot not waste its time nor your bandwidth by crawling dynamically-generated, endless-scrolling /calendar.php pages. Google does use it to that extent. It's not a firewall. Seems like you're describing cloaking (https://developers.google.com/search/docs/advanced/guidelines/cloaking https://developers.google.com/search/docs/advanced/guideline...), one of the oldest SEO tricks, and you can imagine that search engines started defeating it on Day 2 of crawling the web.
- fredgrott 4y agothere is also the spam of name.ru.com domains as well Warning, do not click on those links as you will get your PC infected.
- sammy2244 4y agoWow from clicking on a link on a modern browser? New 0-day?
- sub7 4y agoThis headline would be accurate without the “.it” domains part
- jeroenhd 4y agoI don't think this problem should be solved by Cloudflare. Cheap domains will always exist and they shouldn't be a problem. The problem lies with Google and its failure to detect these spam sites. Surely Google can spare an engineer or two to do a deep dive into the way any one of these spam sites manages to get itself to the first page of Google, work out their scheme, and fix the algorithm? This problem isn't exactly hard to reproduce!
- Hrundi 4y agoThe fact that this has been going on for several years makes me believe Google either doesn't care or the problem is particularly hard to fix (less believable)
- frankfrankfrank 4y agoGoogle has at the very least neglected its search for many years now and recently has also actively made it worse through all the censorship and thought control stuff. I find it rather surprising because essentially all of google’s success is lynchpinned by search. All it would take is for a narrative to dominate that the best results can be found elsewhere, which does not seem particularly remote, considering how much damage google has done to its search.
- ehnto 4y agoIt has been an ongoing battle for 10-15 years at this point. Search engines are constantly battling people trying to game their systems. I have to wonder if Google hasn't lost the thread a bit, inside their surely quite complex algorithm black boxes. For a while now Google has suggested that the best way to rank well is to have human readable content and focus on user experience. At the same time, natural language generation has come leaps and bounds, to the point where sometimes even I, a human, can't tell if an article has been spun by a bot or not. So if Google starts ranking human readable content, and robots can now produce human readable content, what is the next ranking signal they can use to differentiate spam from humans? Are we going to end up with "Verified Websites" ala verified Twitter handles? A huge portion of the web at this point is just bots communicating with eachother, and legitimate business systems having to process bots participation on the internet. I imagine the portion of the web that Google crawls that is legitimate versus that which is bot generated would surely be majority bots, just because of how fast they can generate content. One thing they can't do as easily though is register domains, so it may be one of the better points of defense.
- thejosh 4y agoThere has been a huge influx this year with the amount of sites that simply scrape SO and then have the exact content on their site. It's a pain, and there is no official way to remove them. I thought that this was a massive nono from Googles side, has something changed?
- disruptiveink 4y agoI'd like to get actual confirmation of this, but my vague feeling is that, once upon a time, Google Search would get "updates", as in, actually deployed code that would change the rule of the game and most of the previous dirty tricks would become unusable, leading to people to go out and find out new ones. This changed with the Google "machine learning" days, where you no longer have humans at the helm laying down explicit rules, so no more "change the world" updates, you can only slightly nudge the parameters towards what you want, meaning the same old tricks keep being effective for far too long.
- nerdawson 4y agoThe “May Core Update” which recently rolled out impacted every site. A lot of updates are targeted at specific problems such as low quality product reviews but there are still broader updates taking place.
- dewey 4y ago> Google Search would get "updates", as in, actually deployed code that would change the rule of the game That's just what the scheduled "core update" days are now: https://developers.google.com/search/blog/2022/05/may-2022-core-update https://developers.google.com/search/blog/2022/05/may-2022-c...
- atwood22 4y agoI have no evidence of this, but the ad load on the returned results has gotten way higher. In theory, ranking sites that display Google ads higher would be a very easy knob for Google to turn to increase profit. The SO scrapers probably have Google ads on them, making them more profitable for Google.
- nottorp 4y agoNo, google search is plagued with spam from any domain. And even the non spam results are useless.
- H8crilA 4y agoIt may be that deep learning is now increasingly used to generate the spam. It either is or will be used for spam generation A LOT. Frankly it seems to be the most promising commercial use-case for the large language models.
- mrkramer 4y agoThis is 2nd and 3rd search page results spam, I get spam phishing websites on 1st page of search results when I search for certain ecommerce websites. Google is done.
- jl6 4y agoI recently cursed google search results when trying to research an actor's birth date. There were two dates given on Wikipedia and I wanted to see which one (if either) was correct. Google returned the actor's IMDB page (which listed a third date, and no source), and then pages upon pages of what appeared to be auto-generated sites that clearly scraped from Wikipedia, repeating one or the other of the Wikipedia dates. This is not helping to organize the world's knowledge.
- hericium 4y ago> This is not helping to organize the world's knowledge. And Google is not about organizing world's knowledge but creeping on people for YoY financial results.
- richbell 4y agoThey're quoting Google's own mission statement; though, you are correct. > Google's mission is to organize the world's information and make it universally accessible and useful.
- znpy 4y ago> This is not helping to organize the world's knowledge. Oh they stopped doing that long long ago...
- dazc 4y agoA lot of actors lie about their age so I wouldn't hold out too much hope on getting an accurate result on that one. I get your point though about the multiple results for something where there clearly is no authoritative answer.
- seydor 4y agoIt must be the mafia. No other explanation
- tremon 4y agoYeah, but which one? The Camorra, Ndrangheta, Stidda, RIAA or FAANG?
- randomperson_24 4y agoI think Google can probably not fix it. Users will have to be manually reporting as spam. These websites on seeing traffic from Google's crawler bots show a perfectly legit and highly SEO optimized website, but for anything else show other spam. If Google starts indexing from random IP ranges, most websites would probably block indexing from “unofficial IPs” or some companies (esp in EU) would file some lawsuit against Google. The reason being that some pay-walled news article websites won't be indexed properly, as the “unofficial IP-ed” Googlebot will not get the paywalled content. If a website lies to Google itself, I believe the only way to solve it is by reporting the search result as spam or Google contracts people to somehow visit all billions of web pages (again the same problem – from different IP ranges) to verify it as a legit page. I would like to know how Google currently handles it and probably how it could be improved
- Nextgrid 4y agoThe practice of "cloaking" has been around for ages and I'm sure Google has (or at least had) solutions against it. I'm not sure on what grounds could someone sue for crawling from random, unaffiliated addresses as long as the crawling isn't causing a denial of service (they can always check robots.txt using the main IP then use that to throttle crawling from random IPs as to remain compliant). > The reason being that some pay-walled news article websites won't be indexed properly, as the “unofficial IP-ed” Googlebot will not get the paywalled content. Good riddance? That would be a welcome change.
- panarky 4y agoGoogle has all the text they've scraped. They can see all the domains that have served a given snippet. They also have history to identify where each snippet was first seen. If SO has a lot of traffic and a good reputation, and if the same snippet is found first at SO and then later at bunch of newly created, low volume, low reputation domains, then show the SO result and not the others.
- Thorentis 4y agoThe obsession with "machine learning" is actually making systems dumber. Google Search and Gmail spam filters are getting worse with each passing week, and I am almost certain the increasing reliance on ML is to blame.
- patentatt 4y agoI chalk it up to a cost benefit calculation. Google clearly isn't trying to eliminate all spam in search. It's not their goal. They are not trying to optimize for the user experience. They're trying to optimize revenue.
- mod 4y agoThey're trying to keep spam out of my inbox, and the spam rate has been increasing for me (and other HN commenters who frequently talk about it)
- tremon 4y agoThe competing explanation is that "machine learning" is actually making spam generator systems smarter, so spam gets harder to detect.
- Thorentis 4y agoThe type of spam I see in my inbox is anything but smart. But I agree that this has always been a cat and mouse game.
- Ueland 4y agoNote that the discussion is a year old. Around one year ago I wrote more about this "phenonomen" here: https://news.ycombinator.com/item?id=27993123 https://news.ycombinator.com/item?id=27993123 tl;dr: I managed to find the servers behind it, most likely anybody who are still affected can do the same thing I did pretty easily. We also followed the money, which is a tad more work.
- kmfrk 4y agoRemember the good old days of talking about a "semantic web"? Now we just get one Google results page of SEO'd garbage with no way to process them. I can't help plug kagi.com, which has the amazing feature of grouping SEO'd stuff like recommendation lists together, so a thing that's contextually useful is still available but without polluting the other contexts.
- aaron695 4y ago
- midislack 4y agoGoogle hasn't given a shit about search since at least a decade ago. It's all about data collection via Android and Chrome OS, and gmail and docs. They don't need search to collect your data any more. Don't people actually know this? LOL
- DharmaPolice 4y agoAll that data they've collected is only useful if they can sell something (i.e. ads) based on the data. AFAIK the majority of their income they get for ads is from search based ads.
- midislack 4y agoGovernments are happy to buy the data actually. In-Q-Tel, aka the CIA, was the first large investor in Google.
- worldofmatthew 4y agoGoogle appears to stopped caring after Matt Cutts left......
- jrockway 4y agoThe Internet is quite a bit bigger than it was in 2016: https://www.internetworldstats.com/emarketing.htm https://www.internetworldstats.com/emarketing.htm (Have no idea how reputable that data is, but it seems about right to me. In 2016 there were 3.6 billion Internet users. Now there are 5.3 billion.)
- Proven 4y ago
- ehnto 4y agoAlso imagine how many bots there are, and how fast they can generate content now.
- manquer 4y agoMost that growth is in non English language users . There are only so many English speakers, we are talking about quality of English search results, they number of users for that has not doubled in 6 years.
- jrockway 4y agoIsn't the article we're discussing about a .it domain impersonating a Japanese product?
- manquer 4y agoContent moderation and SEO in non English languages is far worse than English. I meant that google dropping search quality for English has not much to do with growth users in the last few years as that growth has largely been non english
- meerita 4y agoDealwith.it
- oliwarner 4y agoThe real question we should be asking is: Why Google don't care about their index? Junk copypasta and news-squatting (posting regularly about the same thing with no additional data) is a decade(s) old problem. Crowd sourcing and verifying junk domains could be a weekend project. But nothing.