5 ms·

Google beefs up its security by acquiring online virus scanner VirusTotal

53 points by mukyu 14y ago

ChuckMcM 14y ago
Good move on their part, although I'm surprised they weren't able to create something like this in-house. Basically if you crawl every freakin' web page on the internet you pretty much should find all the bad guy's landing pages.
- throwaway54-762 14y ago
  It might be prohibitively expensive (think: CPU time and memory required to run a virus scanner (multiple virus scanners?) on every unique page on the internet). Edit: Plus, per-machine licenses, and profitability -- how much is it worth vs blacklisting manually?
  - ChuckMcM 14y ago
    Heh, you can't put 'expensive CPU time and memory' and Google in the same sentence :-) That said, its a great use for the crawler, as long as you are reading web pages to figure out how to index them, might as well scan them for malware. As a marginal cost I expect it to be quite low once you have the crawl infrastructure in place.
  - toomuchtodo 14y ago
    Computing time will ALWAYS be cheaper than human labor.
    - michael_miller 14y ago
      I'm not sure I agree with this. When operating at a very large scale, like Google, eeking out tiny gains in efficiency can yield huge cost savings. A 1% saving of CPU time on a 100K core cluster means 1,000 cores can be eliminated from the system. Figure $50 per core, and you've just saved $50,000 in base cost, plus a ton on cooling + power. Meaning, if it takes less than 3 dev-months, it is absolutely worth the developer time to optimize.
      - ChuckMcM 14y ago
        This is absolutely a true statement, and its wrong :-) Until the proposals for energy proportional computing [1] bear fruit there is a lot of 'sunk cost' going into the infrastructure. So finding useful work to do with that sunk cost is a net win. [1] http://research.google.com/pubs/pub33387.html http://research.google.com/pubs/pub33387.html
- adsr 14y ago
  They've been maintaining and updating the safe browsing database for some time now. It shows a warning in supported browsers for potentially malicious web pages. There's also a web diagnostics page available btw, here testing hacker news: http://www.google.com/safebrowsing/diagnostic?site=news.ycombinator.com/ http://www.google.com/safebrowsing/diagnostic?site=news.ycom...
- sharjeel 14y ago
  Its probably more about the huge dataset VirusTotal has gathered over time.
mmaunder 14y ago
Question: Does virustotal scrape data from Norton, McAfee and the 20 to 50 scanning engines they use, or do they have a paid licensing arrangement with them? [I use the term scraping loosely e.g. they could run the file against a local instance of each scanner, upload the file to other websites, etc]
- huhtenberg 14y ago
  I'm pretty sure they have a set of servers each running a licensed copy of every AV software they support, all scripted and automated in some form or fashion. I am really puzzled as to why Google needed to acquire them. It surely wasn't a technology acquisition.
  - dpeck 14y ago
    They've got a hell of a dataset and lot of users/partners that push sample data to them all the time. Even if they lose some of those because people don't want to deal with GOOG its still huge information source that they will probably be able to put to better use than most anyone else.

53 points by mukyu 14y ago

Good move on their part, although I'm surprised they weren't able to create something like this in-house. Basically if you crawl every freakin' web page on the internet you pretty much should find all the bad guy's landing pages.

throwaway54-762 14y ago
It might be prohibitively expensive (think: CPU time and memory required to run a virus scanner (multiple virus scanners?) on every unique page on the internet). Edit: Plus, per-machine licenses, and profitability -- how much is it worth vs blacklisting manually?
- ChuckMcM 14y ago
  Heh, you can't put 'expensive CPU time and memory' and Google in the same sentence :-) That said, its a great use for the crawler, as long as you are reading web pages to figure out how to index them, might as well scan them for malware. As a marginal cost I expect it to be quite low once you have the crawl infrastructure in place.
- toomuchtodo 14y ago
  Computing time will ALWAYS be cheaper than human labor.
  - michael_miller 14y ago
    I'm not sure I agree with this. When operating at a very large scale, like Google, eeking out tiny gains in efficiency can yield huge cost savings. A 1% saving of CPU time on a 100K core cluster means 1,000 cores can be eliminated from the system. Figure $50 per core, and you've just saved $50,000 in base cost, plus a ton on cooling + power. Meaning, if it takes less than 3 dev-months, it is absolutely worth the developer time to optimize.
    - ChuckMcM 14y ago
      This is absolutely a true statement, and its wrong :-) Until the proposals for energy proportional computing [1] bear fruit there is a lot of 'sunk cost' going into the infrastructure. So finding useful work to do with that sunk cost is a net win. [1] http://research.google.com/pubs/pub33387.html http://research.google.com/pubs/pub33387.html
adsr 14y ago
They've been maintaining and updating the safe browsing database for some time now. It shows a warning in supported browsers for potentially malicious web pages. There's also a web diagnostics page available btw, here testing hacker news: http://www.google.com/safebrowsing/diagnostic?site=news.ycombinator.com/ http://www.google.com/safebrowsing/diagnostic?site=news.ycom...
sharjeel 14y ago
Its probably more about the huge dataset VirusTotal has gathered over time.

mmaunder 14y ago

Question: Does virustotal scrape data from Norton, McAfee and the 20 to 50 scanning engines they use, or do they have a paid licensing arrangement with them? [I use the term scraping loosely e.g. they could run the file against a local instance of each scanner, upload the file to other websites, etc]

huhtenberg 14y ago
I'm pretty sure they have a set of servers each running a licensed copy of every AV software they support, all scripted and automated in some form or fashion. I am really puzzled as to why Google needed to acquire them. It surely wasn't a technology acquisition.
- dpeck 14y ago
  They've got a hell of a dataset and lot of users/partners that push sample data to them all the time. Even if they lose some of those because people don't want to deal with GOOG its still huge information source that they will probably be able to put to better use than most anyone else.