3 ms·

That's odd. UrlRoulette does some sanity checking on the content of the URL. Thank you for this bug report!

by mmathias 10y ago

Lorenzo45 10y ago
I also had it fail for www.autodraw.com
Clex 10y ago
You test the URL with a HEAD method that often 404. You should try with GET instead.
- hk__2 10y ago
  A server should respond with the same headers to a HEAD request as to a GET one.
  - Daviey 10y ago
    SHOULD and DOES are not the same thing. Returning 405 'Method Not Allowed' on a HEAD isn't unreasonable, and that is the issue we are seeing. EDIT: did say GET on prior edit.
    - hk__2 10y ago
      > Returning 405 'Method Not Allowed' on a GET isn't unreasonable I think you meant "on a HEAD".
      - Daviey 10y ago
        er yes, my bad. Fixed
  - tyingq 10y ago
    They should, but many good sites just don't, for whatever reason. Even ones you know are run by smart people: $ curl -I -X HEAD https://redis.io/ HTTP/1.1 404 Not Found For this app, being more forgiving is a better idea.
    - jaredsohn 10y ago
      https://www.stumbleupon.com https://www.stumbleupon.com had a similar issue.
    - deleted 10y ago
      [deleted]
artursapek 10y ago
https://cryptowat.ch https://cryptowat.ch fails too. seems to fail at very basic URL parsing.
- bigbugbag 10y ago
  it fails at a lot of other basic things: I was unable to post anything because my firefox 52 is not supported to get a recaptcha challenge. Please upgrade to a supported browser to get a reCAPTCHA challenge. Alternatively if you think you are getting this page in error, please check your internet connection and reload. I could reload to infinity, this would not fix this broken website or change the useless error message.
Nycto 10y ago
Fails for Amazon URLs, too (And no, I didn't include a referral link in there.)
- tyingq 10y ago
  Amazon rejects obvious non real browser user agents. This app tries to verify urls before putting them in the queue. I think they may have more anti scraper measures as well.

That's odd. UrlRoulette does some sanity checking on the content of the URL. Thank you for this bug report!

Lorenzo45 10y ago

I also had it fail for www.autodraw.com

Clex 10y ago

You test the URL with a HEAD method that often 404. You should try with GET instead.

hk__2 10y ago
A server should respond with the same headers to a HEAD request as to a GET one.
- Daviey 10y ago
  SHOULD and DOES are not the same thing. Returning 405 'Method Not Allowed' on a HEAD isn't unreasonable, and that is the issue we are seeing. EDIT: did say GET on prior edit.
  - hk__2 10y ago
    > Returning 405 'Method Not Allowed' on a GET isn't unreasonable I think you meant "on a HEAD".
    - Daviey 10y ago
      er yes, my bad. Fixed
- tyingq 10y ago
  They should, but many good sites just don't, for whatever reason. Even ones you know are run by smart people: $ curl -I -X HEAD https://redis.io/ HTTP/1.1 404 Not Found For this app, being more forgiving is a better idea.
  - jaredsohn 10y ago
    https://www.stumbleupon.com https://www.stumbleupon.com had a similar issue.
  - deleted 10y ago
    [deleted]

artursapek 10y ago

https://cryptowat.ch https://cryptowat.ch fails too. seems to fail at very basic URL parsing.

bigbugbag 10y ago
it fails at a lot of other basic things: I was unable to post anything because my firefox 52 is not supported to get a recaptcha challenge. Please upgrade to a supported browser to get a reCAPTCHA challenge. Alternatively if you think you are getting this page in error, please check your internet connection and reload. I could reload to infinity, this would not fix this broken website or change the useless error message.

Nycto 10y ago

Fails for Amazon URLs, too (And no, I didn't include a referral link in there.)

tyingq 10y ago
Amazon rejects obvious non real browser user agents. This app tries to verify urls before putting them in the queue. I think they may have more anti scraper measures as well.