3 ms·
It might be prohibitively expensive (think: CPU time and memory required to run a virus scanner (multiple virus scanners?) on every unique page on the internet)
by throwaway54-762 14y ago
It might be prohibitively expensive (think: CPU time and memory required to run a virus scanner (multiple virus scanners?) on every unique page on the internet).
Edit: Plus, per-machine licenses, and profitability -- how much is it worth vs blacklisting manually?
- ChuckMcM 14y agoHeh, you can't put 'expensive CPU time and memory' and Google in the same sentence :-) That said, its a great use for the crawler, as long as you are reading web pages to figure out how to index them, might as well scan them for malware. As a marginal cost I expect it to be quite low once you have the crawl infrastructure in place.
- toomuchtodo 14y agoComputing time will ALWAYS be cheaper than human labor.
- michael_miller 14y agoI'm not sure I agree with this. When operating at a very large scale, like Google, eeking out tiny gains in efficiency can yield huge cost savings. A 1% saving of CPU time on a 100K core cluster means 1,000 cores can be eliminated from the system. Figure $50 per core, and you've just saved $50,000 in base cost, plus a ton on cooling + power. Meaning, if it takes less than 3 dev-months, it is absolutely worth the developer time to optimize.
- ChuckMcM 14y agoThis is absolutely a true statement, and its wrong :-) Until the proposals for energy proportional computing [1] bear fruit there is a lot of 'sunk cost' going into the infrastructure. So finding useful work to do with that sunk cost is a net win. [1] http://research.google.com/pubs/pub33387.html http://research.google.com/pubs/pub33387.html