4 ms·

Looks like there's some bugs here. Just claimed 'esi' was available on Twitter and GitHub, but it's not.

by jschuur 14y ago

cbhl 14y ago
Hmm. Judging from the way the server.js is written, I'd expect he's being rate-limited, and the server returns free whenever the HTTP call doesn't return 200. Edit: Just to be sure, I threw in a breakpoint on line 18 of generate.js, and set name to my username (cbhl) -- and got a false "free" result for both Twitter and GitHub. The OP might need to be getting OAuth tokens and using the GitHub/Twitter APIs.
- trevorah 14y ago
  Any tips to sort this out? How are things being rate limited? edit: just realised you meant that twitter/github etc are rate limiting my requests. Bummer.
  - jschuur 14y ago
    Just hit the profile page URL and get the HTTP staus code. If https://twitter.com/ksdhbfslbflsdbs https://twitter.com/ksdhbfslbflsdbs returns 404, it's not available. Not quit the same (a username might be reserved internally), but either way, nobody else can use the name.
    - trevorah 14y ago
      Yep, thats pretty much how it works at the moment (see https://github.com/trevorah/usernique/blob/master/server.js https://github.com/trevorah/usernique/blob/master/server.js). The issue is that the server is sending out quite a few of these hits to twitter, and twitter is smart enough to start reject requests when things get a bit too much.
      - nsp 14y ago
        I'm not a node guy, but I found this module for Whois checks - https://npmjs.org/package/whois-available https://npmjs.org/package/whois-available It uses the IANA Whois service, an admittedly cursory look doesn't turn up anything re: rate limits, but you could just do a get request on the root url and only check Whois if it 404s to be safe. http://www.iana.org/whois http://www.iana.org/whois
        jlogsdon 14y ago
        Whois wouldn't help here. The problem is that so many requests are going to twitter.com/[username] that they start sending false 404s (or some other error, not 200) because of rate limiting.
        nsp 14y ago
        I meant to fix the false positives on the domain name availability, or is that issue related somehow?
  - cbhl 14y ago
    If you can figure out how to do the request on the client side, that would mitigate the issue. Otherwise, I'd say you're probably out of luck. (You could try doing the query over IPv6, but eventually they'll just block the entire subnet, so I don't know if it's worth the trouble.)

Looks like there's some bugs here. Just claimed 'esi' was available on Twitter and GitHub, but it's not.

by jschuur 14y ago

Looks like there's some bugs here. Just claimed 'esi' was available on Twitter and GitHub, but it's not.

cbhl 14y ago

Hmm. Judging from the way the server.js is written, I'd expect he's being rate-limited, and the server returns free whenever the HTTP call doesn't return 200. Edit: Just to be sure, I threw in a breakpoint on line 18 of generate.js, and set name to my username (cbhl) -- and got a false "free" result for both Twitter and GitHub. The OP might need to be getting OAuth tokens and using the GitHub/Twitter APIs.

trevorah 14y ago
Any tips to sort this out? How are things being rate limited? edit: just realised you meant that twitter/github etc are rate limiting my requests. Bummer.
- jschuur 14y ago
  Just hit the profile page URL and get the HTTP staus code. If https://twitter.com/ksdhbfslbflsdbs https://twitter.com/ksdhbfslbflsdbs returns 404, it's not available. Not quit the same (a username might be reserved internally), but either way, nobody else can use the name.
  - trevorah 14y ago
    Yep, thats pretty much how it works at the moment (see https://github.com/trevorah/usernique/blob/master/server.js https://github.com/trevorah/usernique/blob/master/server.js). The issue is that the server is sending out quite a few of these hits to twitter, and twitter is smart enough to start reject requests when things get a bit too much.
    - nsp 14y ago
      I'm not a node guy, but I found this module for Whois checks - https://npmjs.org/package/whois-available https://npmjs.org/package/whois-available It uses the IANA Whois service, an admittedly cursory look doesn't turn up anything re: rate limits, but you could just do a get request on the root url and only check Whois if it 404s to be safe. http://www.iana.org/whois http://www.iana.org/whois
      - jlogsdon 14y ago
        Whois wouldn't help here. The problem is that so many requests are going to twitter.com/[username] that they start sending false 404s (or some other error, not 200) because of rate limiting.
        nsp 14y ago
        I meant to fix the false positives on the domain name availability, or is that issue related somehow?
- cbhl 14y ago
  If you can figure out how to do the request on the client side, that would mitigate the issue. Otherwise, I'd say you're probably out of luck. (You could try doing the query over IPv6, but eventually they'll just block the entire subnet, so I don't know if it's worth the trouble.)