6 ms·

I should add that HNSearch has an API http://www.hnsearch.com/api http://www.hnsearch.com/api which should be a lot faster than crawling HN itself to get the

by pg 15y ago

I should add that HNSearch has an API http://www.hnsearch.com/api http://www.hnsearch.com/api which should be a lot faster than crawling HN itself to get the same data.

k7d 15y ago
This is cool, but the API looks a bit cryptic. Is there a way to simply lookup an item by URL? Edit: Nevermind, figured it out: http://api.thriftdb.com/api.hnsearch.com/items/_search?pretty_print=true&filter[fields][url][]=http%3A%2F%2Fycombinator.com%2Fnewsnews.html%234jun11 http://api.thriftdb.com/api.hnsearch.com/items/_search?prett...
- news-yc 15y ago
  Or, by "id"? I looked, but I wasn't able to figure out, for example, how I'd get pg's top-level comment here from the id "2619846". I was able to find it with this search, but it requires a second-part to the "_id" that I don't know where to find: http://api.thriftdb.com/api.hnsearch.com/items/_search?q=&filter[fields][_id]=2619846-a30f9&pretty_print=true http://api.thriftdb.com/api.hnsearch.com/items/_search?q=&#3... (Trying to filter for "id" instead of "_id" throws an error, for some reason.)
  - andres 15y ago
    The `_id` attribute for each item is its submission id plus a signature. ThriftDB references objects by their unique `_id` attributes. The `id` attribute on the other hand is an item's submission id. It's not indexed though so you can't do lookup by id.
    - milkshakes 15y ago
      could you please index the submission ids? i can think of a few reasons this would be useful
      - andres 15y ago
        Would you mind emailing me with your ideas? andres@octopart.com
      - lt 15y ago
        Exactly. How do I find this comment through the API? How do I find its replies?
    - ars 15y ago
      Well that explains why I failed at fetching an item by id, and I tried a whole bunch of different ways. You should really mention somewhere that the "id" referenced is an internal id, and not the HN id. I would like to lookup items by HN id.
- andres 15y ago
  The API docs are a work in progress. Please let me know which parts need work.
  - k7d 15y ago
    For example what's the difference between "filter[queries][]" and "q" arguments for http://www.thriftdb.com/documentation/rest-api/search-api http://www.thriftdb.com/documentation/rest-api/search-api? The way I understand, q does a simple keyword matching on all fields while "filter[queries][]" is more like SQL "where" query. In that case, what operators are supported?
    - andres 15y ago
      The `q` argument is used to perform a full-text search across all fields. One benefit to using `q` is that you can rank results based on field matches (using the `weights` argument). The `filter` arguments are used to cut the data before performing a full-text query: `filter[fields][fieldname]` can be used to filter on fieldnames: http://api.thriftdb.com/api.hnsearch.com/items/_search?filter[fields][username][]=andres http://api.thriftdb.com/api.hnsearch.com/items/_search?filte... `filter[queries][]` can be used to add arbitrary filters: http://api.thriftdb.com/api.hnsearch.com/items/_search?filter[queries][]=username:andres http://api.thriftdb.com/api.hnsearch.com/items/_search?filte... http://api.thriftdb.com/api.hnsearch.com/items/_search?filter[queries][]=create_ts:[NOW-5DAYS%20TO%20NOW] http://api.thriftdb.com/api.hnsearch.com/items/_search?filte... The syntax for filter queries is the same as solr: http://wiki.apache.org/solr/CommonQueryParameters#fq http://wiki.apache.org/solr/CommonQueryParameters#fq
k7d 15y ago
So after some quick hacking I created a new HN bookmarklet based on the new API: http://kaspa.rs/hn-bookmarklet/ http://kaspa.rs/hn-bookmarklet/ If the current URL is added, it will go straight to discussion thread (without voting). If not, it will ask if you want to submit it.
- JoachimSchipper 15y ago
  Why not model it after the submission page (submit or upvote)?
  - k7d 15y ago
    Original HN bookmarklet already works that way http://ycombinator.com/bookmarklet.html http://ycombinator.com/bookmarklet.html However - I don't always want to upvote a post just because I want to check the discussion. Published this with an assumption that I'm not the only one.
vladocar 15y ago
I will be super cool if JSONP support is added. If we want to build web app with JavaScript it is much easier to use directly JSONP. When I build this web app http://www.vcarrer.com/2010/11/hacker-news-mobile-front-page-reader.html http://www.vcarrer.com/2010/11/hacker-news-mobile-front-page... I need to use Yahoo YQL to obtain JSONP. So pretty please, add JSONP.
- k7d 15y ago
  There is already JSONP support, see callback argument: http://www.thriftdb.com/documentation/rest-api http://www.thriftdb.com/documentation/rest-api
  - vladocar 15y ago
    Cool! Thanks!
jrussbowman 15y ago
I'm already working on integrating the api in unscatter.com using a /hn tag - http://www.unscatter.com/search/?q=%2Fhn+facebook http://www.unscatter.com/search/?q=%2Fhn+facebook (currently only functional)
nrbafna 15y ago
Browser notification when someone comments on your post. Would that be useful?
evanrmurphy 15y ago
Should be faster for the developer and also good for HN's servers to not have so many independent apps scraping the forum.

by pg 15y ago

I should add that HNSearch has an API http://www.hnsearch.com/api http://www.hnsearch.com/api which should be a lot faster than crawling HN itself to get the same data.

k7d 15y ago

This is cool, but the API looks a bit cryptic. Is there a way to simply lookup an item by URL? Edit: Nevermind, figured it out: http://api.thriftdb.com/api.hnsearch.com/items/_search?pretty_print=true&filter[fields][url][]=http%3A%2F%2Fycombinator.com%2Fnewsnews.html%234jun11 http://api.thriftdb.com/api.hnsearch.com/items/_search?prett...

news-yc 15y ago
Or, by "id"? I looked, but I wasn't able to figure out, for example, how I'd get pg's top-level comment here from the id "2619846". I was able to find it with this search, but it requires a second-part to the "_id" that I don't know where to find: http://api.thriftdb.com/api.hnsearch.com/items/_search?q=&filter[fields][_id]=2619846-a30f9&pretty_print=true http://api.thriftdb.com/api.hnsearch.com/items/_search?q=&#3... (Trying to filter for "id" instead of "_id" throws an error, for some reason.)
- andres 15y ago
  The `_id` attribute for each item is its submission id plus a signature. ThriftDB references objects by their unique `_id` attributes. The `id` attribute on the other hand is an item's submission id. It's not indexed though so you can't do lookup by id.
  - milkshakes 15y ago
    could you please index the submission ids? i can think of a few reasons this would be useful
    - andres 15y ago
      Would you mind emailing me with your ideas? andres@octopart.com
    - lt 15y ago
      Exactly. How do I find this comment through the API? How do I find its replies?
  - ars 15y ago
    Well that explains why I failed at fetching an item by id, and I tried a whole bunch of different ways. You should really mention somewhere that the "id" referenced is an internal id, and not the HN id. I would like to lookup items by HN id.
andres 15y ago
The API docs are a work in progress. Please let me know which parts need work.
- k7d 15y ago
  For example what's the difference between "filter[queries][]" and "q" arguments for http://www.thriftdb.com/documentation/rest-api/search-api http://www.thriftdb.com/documentation/rest-api/search-api? The way I understand, q does a simple keyword matching on all fields while "filter[queries][]" is more like SQL "where" query. In that case, what operators are supported?
  - andres 15y ago
    The `q` argument is used to perform a full-text search across all fields. One benefit to using `q` is that you can rank results based on field matches (using the `weights` argument). The `filter` arguments are used to cut the data before performing a full-text query: `filter[fields][fieldname]` can be used to filter on fieldnames: http://api.thriftdb.com/api.hnsearch.com/items/_search?filter[fields][username][]=andres http://api.thriftdb.com/api.hnsearch.com/items/_search?filte... `filter[queries][]` can be used to add arbitrary filters: http://api.thriftdb.com/api.hnsearch.com/items/_search?filter[queries][]=username:andres http://api.thriftdb.com/api.hnsearch.com/items/_search?filte... http://api.thriftdb.com/api.hnsearch.com/items/_search?filter[queries][]=create_ts:[NOW-5DAYS%20TO%20NOW] http://api.thriftdb.com/api.hnsearch.com/items/_search?filte... The syntax for filter queries is the same as solr: http://wiki.apache.org/solr/CommonQueryParameters#fq http://wiki.apache.org/solr/CommonQueryParameters#fq

k7d 15y ago

So after some quick hacking I created a new HN bookmarklet based on the new API: http://kaspa.rs/hn-bookmarklet/ http://kaspa.rs/hn-bookmarklet/ If the current URL is added, it will go straight to discussion thread (without voting). If not, it will ask if you want to submit it.

JoachimSchipper 15y ago
Why not model it after the submission page (submit or upvote)?
- k7d 15y ago
  Original HN bookmarklet already works that way http://ycombinator.com/bookmarklet.html http://ycombinator.com/bookmarklet.html However - I don't always want to upvote a post just because I want to check the discussion. Published this with an assumption that I'm not the only one.

vladocar 15y ago

I will be super cool if JSONP support is added. If we want to build web app with JavaScript it is much easier to use directly JSONP. When I build this web app http://www.vcarrer.com/2010/11/hacker-news-mobile-front-page-reader.html http://www.vcarrer.com/2010/11/hacker-news-mobile-front-page... I need to use Yahoo YQL to obtain JSONP. So pretty please, add JSONP.

k7d 15y ago
There is already JSONP support, see callback argument: http://www.thriftdb.com/documentation/rest-api http://www.thriftdb.com/documentation/rest-api
- vladocar 15y ago
  Cool! Thanks!

jrussbowman 15y ago

I'm already working on integrating the api in unscatter.com using a /hn tag - http://www.unscatter.com/search/?q=%2Fhn+facebook http://www.unscatter.com/search/?q=%2Fhn+facebook (currently only functional)

nrbafna 15y ago

Browser notification when someone comments on your post. Would that be useful?

evanrmurphy 15y ago

Should be faster for the developer and also good for HN's servers to not have so many independent apps scraping the forum.