4 ms·

Ask HN: Best web search API(s)

1 points by will_brown 13y ago

I have previously coded a project using both Google search API and Bing API. Google was problematic because of query limits without paying. Bing addressed the issue with the query limits, but I figured I would get some HN opinions. I looked into DuckDuckGo because I liked the idea of a search engine that did not collect info of searches, but my understanding of the DDG APIs are that it is not the full DDG web results but there "0 click answers" (I could be wrong but likely due to DDG's use of other search APIs for their web results). So perhaps another question of interest is what API(s) does DDG use?

masswerk 13y ago
On Google's Custom Search API: As far as I am understanding this you won't be able to retrieve a "normal" result as provided by coogle.com by the API, even if you are paying them. The API is always bound to a custom search engine (which won't be updated as often as Google's standard index).
- will_brown 13y ago
  Per the API TOS and my personal experience you are correct, the API results are not identical to actual google results (at least the non-paid, but I believe you are correct and even if paying API results will not be the same as google). That said I never had complaints regarding the quality of the results on my custom engine using google API, only that the query limits make it impractical for my purpose. Do you have a suggestion for search API, other than bing with no query limits?
  - masswerk 13y ago
    Sorry to say no. I've done some with the old REST API (using the free quota, cf: http://www.masswerk.at/google60 http://www.masswerk.at/google60 http://www.masswerk.at/googleBBS http://www.masswerk.at/googleBBS http://www.masswerk.at/googleAsteroids http://www.masswerk.at/googleAsteroids ) which is due to expire sometime in the near future, but I don't see any backup for this with same functionality. Bing killed their API in favor for the Azure Market Place last year (in August). DuckDuckGo's API looks nice, but results are not on the same level (as of writing this). BTW: Google's results using the REST API (free quota) seem to vary with usage. When there's some load, there will be only a limited set of result on the next "page". Edit: As free quotas are very limited with the new APIs (10/day), I'ld guess the days for any experiments with search operations are over. (If a project gains some interest, you are due for 1M hits/day, and the fees for this are more than anyone would afford for a pet project.) You could try to make a deal with the vendor in question before, but this would result in an amount of planning that is out of scope for a project, which is essentially shoot-and-go.

1 points by will_brown 13y ago

masswerk 13y ago

On Google's Custom Search API: As far as I am understanding this you won't be able to retrieve a "normal" result as provided by coogle.com by the API, even if you are paying them. The API is always bound to a custom search engine (which won't be updated as often as Google's standard index).

will_brown 13y ago
Per the API TOS and my personal experience you are correct, the API results are not identical to actual google results (at least the non-paid, but I believe you are correct and even if paying API results will not be the same as google). That said I never had complaints regarding the quality of the results on my custom engine using google API, only that the query limits make it impractical for my purpose. Do you have a suggestion for search API, other than bing with no query limits?
- masswerk 13y ago
  Sorry to say no. I've done some with the old REST API (using the free quota, cf: http://www.masswerk.at/google60 http://www.masswerk.at/google60 http://www.masswerk.at/googleBBS http://www.masswerk.at/googleBBS http://www.masswerk.at/googleAsteroids http://www.masswerk.at/googleAsteroids ) which is due to expire sometime in the near future, but I don't see any backup for this with same functionality. Bing killed their API in favor for the Azure Market Place last year (in August). DuckDuckGo's API looks nice, but results are not on the same level (as of writing this). BTW: Google's results using the REST API (free quota) seem to vary with usage. When there's some load, there will be only a limited set of result on the next "page". Edit: As free quotas are very limited with the new APIs (10/day), I'ld guess the days for any experiments with search operations are over. (If a project gains some interest, you are due for 1M hits/day, and the fees for this are more than anyone would afford for a pet project.) You could try to make a deal with the vendor in question before, but this would result in an amount of planning that is out of scope for a project, which is essentially shoot-and-go.