5 ms·

Why not use something like http://www.google.com/xhtml?q=google%20mobile%20search http://www.google.com/xhtml?q=google%20mobile%20search ?

by medecau 16y ago

PostOnce 16y ago
I misunderstood you for a moment. I thought you meant to use that instead of scroogle. Now that I know what you mean, yes, that looks like an ideal solution. The Scroogle people are moaning for no reason. It's not Google's job to provide them with the layout they prefer. How hard is it to scrape Google anyway? It's not Flash-based or anything. Jeez.
- jacquesm 16y ago
  Scraping google for an extended period is in fact incredibly hard, but not because of the lay out. I'm fairly sure that someone on googles management has had to give scroogle a free pass on the scraping because at any serious usage level they'd normally have been blocked long ago. I'm guessing they have weighed the advantages of having scroogle out there versus the disadvantages of a bunch of bad press. They might even think it is good to have sites that are openly critical of google out there. Scroogle should count their blessings that it has lasted as long as it did, now they should get their act together and fix the bugs in their scraper.
  - PostOnce 16y ago
    Does Scroogle really generate enough traffic for Google to even notice, or bother taking any action? Does anyone use it on a regular basis?
    - jacquesm 16y ago
      About 6K uniques on a daily basis. If they sent that to google without some trickery they'd be banned within a day.
est 16y ago
any more like this?
fragmede 16y ago
I would think that the ajax API could also be twisted to their needs. It requires an API key that Google puts some limits on, limits that they might want to apply to Scroogle in any case. http://code.google.com/apis/ajaxsearch/ http://code.google.com/apis/ajaxsearch/
- drm237 16y ago
  The ajax api returns completely different results than the normal search and in a different order, so it's not worth using if you want to "actual" google search results.
- goodness 16y ago
  Actually, their docs claim that the limits aren't really related to the API key. The API key is just so they can contact you if they start blocking you: http://code.google.com/apis/ajaxsearch/key.html That may just be disinformation though. For example, if you can spread your usage over multiple IPs, you might be allowed more searches than if you told them your API key for all your requests on multiple IP addresses.

Why not use something like http://www.google.com/xhtml?q=google%20mobile%20search http://www.google.com/xhtml?q=google%20mobile%20search ?

by medecau 16y ago

Why not use something like http://www.google.com/xhtml?q=google%20mobile%20search http://www.google.com/xhtml?q=google%20mobile%20search ?

PostOnce 16y ago

I misunderstood you for a moment. I thought you meant to use that instead of scroogle. Now that I know what you mean, yes, that looks like an ideal solution. The Scroogle people are moaning for no reason. It's not Google's job to provide them with the layout they prefer. How hard is it to scrape Google anyway? It's not Flash-based or anything. Jeez.

jacquesm 16y ago
Scraping google for an extended period is in fact incredibly hard, but not because of the lay out. I'm fairly sure that someone on googles management has had to give scroogle a free pass on the scraping because at any serious usage level they'd normally have been blocked long ago. I'm guessing they have weighed the advantages of having scroogle out there versus the disadvantages of a bunch of bad press. They might even think it is good to have sites that are openly critical of google out there. Scroogle should count their blessings that it has lasted as long as it did, now they should get their act together and fix the bugs in their scraper.
- PostOnce 16y ago
  Does Scroogle really generate enough traffic for Google to even notice, or bother taking any action? Does anyone use it on a regular basis?
  - jacquesm 16y ago
    About 6K uniques on a daily basis. If they sent that to google without some trickery they'd be banned within a day.

est 16y ago

any more like this?

fragmede 16y ago

I would think that the ajax API could also be twisted to their needs. It requires an API key that Google puts some limits on, limits that they might want to apply to Scroogle in any case. http://code.google.com/apis/ajaxsearch/ http://code.google.com/apis/ajaxsearch/

drm237 16y ago
The ajax api returns completely different results than the normal search and in a different order, so it's not worth using if you want to "actual" google search results.
goodness 16y ago
Actually, their docs claim that the limits aren't really related to the API key. The API key is just so they can contact you if they start blocking you: http://code.google.com/apis/ajaxsearch/key.html That may just be disinformation though. For example, if you can spread your usage over multiple IPs, you might be allowed more searches than if you told them your API key for all your requests on multiple IP addresses.