4 ms·
Have a bunch of pages with no real content (but have millions of pages). Everytime someone tries to load a page, do some intensive task (ex: mining bitcoins).
by kyrra 12y ago
Have a bunch of pages with no real content (but have millions of pages). Everytime someone tries to load a page, do some intensive task (ex: mining bitcoins).
If you just make it appealing to GoogleBot and no one else, you get free computational resources.
- xigency 12y agoIs that really how it works?
- belorn 12y agoIf the mining are done at Google Cloud Storage, initiated by a google search bot, can't Google then identify and handle such abuse? I assume Google already scans for multiple types of abuse, such as sites that spread malware.
- skj 12y agoNow you're suggesting that Google basically devotes a team to detecting "crawler-free-quota abuse", when the real solution needs to handles crawlers from many different sources that aren't all Google.
- nemothekid 12y agoGoogle shouldn't really have to do this. Replace GoogleBot with BingBot or GCE with AWS and you still have the same problem. A website operator should be working to make sure search crawlers don't consume too many resources given that the bots follow rules. Otherwise you'd have a team at every cloud provider trying to figure out how to manage bots.
- fragmede 12y agoSorry, I still don't get it. How does not charging for outgoing network traffic make computation free? You'd still be paying for everything else, eg the instances themselves, datastore storage, read/write datastore calls, using the logs API, which means mining bitcoins wouldn't be free.
- brianpgordon 12y agoOP's concern isn't with network traffic, it's with GAE compute time. Googlebot keeps causing instances to run. If requests initiated by Googlebot were free to run, you could make a giant website full of garbage and use each free request to spend 50ms mining bitcoin.