7 ms·
Our company [0] was also hit by this too. We receive email for our customers and a portion of that is spam (given the nature of email). Google decided out of
by veesahni 6y ago
Our company [0] was also hit by this too.
We receive email for our customers and a portion of that is spam (given the nature of email). Google decided out of the blue to mark our attachment S3 bucket as dangerous, because of one malicious file.
What's most interesting is that the bucket is private, so the only way they could identify that there is something malicious at a URL is if someone downloads it using Chrome. I'm assuming they make this decision based on some database of checksums.
To mitigate, we now operate a number of proxies in front of the bucket, so we can quickly replace any that get marked as dangerous. We also now programmatically monitor presence of our domains in Google's "dangerous site" database (they have APIs for this).
0: https://www.enchant.com https://www.enchant.com - software for better customer service
- gomox 6y agoAuthor here. I'm not sure exactly how they actually decide to flag. Alternatively, Amazon might somehow be reporting files in S3 onto the Google blacklist. It would seem surprising, but it's the other possibility.
- judge2020 6y ago> What's most interesting is that the bucket is private, so the only way they could identify that there is something malicious at a URL is if someone downloads it using Chrome. I'm assuming they make this decision based on some database of checksums. Doesn't Chrome upload everything downloaded to VirusTotal (a Google product)?
- deleted 6y ago[deleted]
- londons_explore 6y agoThe hashes of all things that match a "probably evil" bloom filter, yes. Hosting a virus on a domain and then downloading it a few times with different chrome installations sounds like a good way to get the whole domain blacklisted...
- zerkten 6y agoThat's why user uploads are worth some thought and consideration. File uploads normally gets treated as a nuisance by developers because it can become kind of fiddly even when it works and you are getting file upload bugs from support. It normally isn't that much of a challenge to mitigate the issues, but other things get priorities. Companies end up leaving pivots to XSS attacks and similar bugs too.
- xmprt 6y agoGoogle has a great service for this called Checksum. You upload a file checksum and it validates it against the database of all known bad checksums that might flag your website as unsafe. The pricing is pretty reasonable too and you can proxy file uploads through their service directly. I'm actually not telling the truth but at what point did you realize that? And what would be the implications if Google actually did release a service like this? It feels a bit like racketeering.
- cryptoquick 6y agoHa! You got me. I was like, wow, that sounds really useful. I'd love to sign up for that, and built my app to use it, if that were the case. But then, I realized: 1). I'd be integrating further into Google because of a problem they created (racketeering), and 2). They seem to really dislike having paying customers (even if they made it, they'd kill it before long).
- monsieurbanana 6y agoAnd 3), they would later update their evil-bloom-filter and all of the sudden the file you paid to get verified is now an Evil File, and they blacklist you anyway. They actually blacklist you even faster, because of course they have in their database that you have the now-evil-file.
- zbrozek 6y agoReal shame if this domain got blocked because of a contraband file, eh? Just pay us and we'll make sure you don't have any problems.
- lrem 6y agoSounds rather too resource-intensive? I've just tried with current Chrome on Windows and a 32MB zip on my personal domain, Wireshark says the file has not been sent anywhere.
- kyrra 6y agoI believe there are limits on the virus checking size. You can see this when trying to download really large files from Google drive (> 100mb)
- lrem 6y agohttps://developers.virustotal.com/v3.0/reference#files-scan https://developers.virustotal.com/v3.0/reference#files-scan Seems I might have just hit the limit? ... Nope, 8.1MB zip file also wasn't sent anywhere.
- tapland 6y agoWouldnt it be more efficient to grab it in parallel to your download?
- lrem 6y agoThat only shifts the bandwidth cost between the original server and the user, Google's resources are unaffected. And it's not what GP claimed. I just checked the nginx access logs - both the 32MB and the 8MB zip files have been accessed only once (both were created only for this experiment).
- vaduz 6y ago> Doesn't Chrome upload everything downloaded to VirusTotal (a Google product)? It doesn't, unless you opt for SafeSearch "Enhanced Protection" or enable "Help improve security on the web for everyone" in "Standard Protection". Both are off by default, IIRC. Without it, it periodically downloads what amounts to bloom filter of "potentially unsafe" URLs/domains. On the other hand, GMail and GDrive do run the checks via VirusTotal, as far as we know - which means that OP case may have been caused by having some of the recipients having their incoming mail automatically scanned. It's similar for Microsoft version (FOPE users provide input for Defender Smart Screen), at least last time I checked.
- ehsankia 6y agoWhat happens if it is a hit against the bloom filter / checksum? Would it transmit the URL so that it can be blocklisted?
- kevincox 6y agohttps://developers.google.com/safe-browsing/v4/update-api#checking-urls https://developers.google.com/safe-browsing/v4/update-api#ch... TL;DR is you download a chunk of SHA-256 hashes and check if the hash for your URL is there. There is of course the chance of collision but that is minuscule.
- richardwhiuk 6y agoOr you could screen your attachments for malware?
- veesahni 6y agoWe do, but it's not good enough for Google.
- deleted 6y ago[deleted]