4 ms·

There's a few things I'll push back on: I could harm your service by making too many API-key signed, OAuth-signed requests too. I could harm your service by hi

There's a few things I'll push back on: I could harm your service by making too many API-key signed, OAuth-signed requests too. I could harm your service by hitting your website a lot too. We have ways of dealing with people who intentionally or unintentionally launch the equivalent of denial of service attacks: you block their IPs and move on. There's no need to have a special magical way of doing it with an API. The point is the whole concept of an API should be unnecessary. We have a way of saying where data is: URLs. We have a way of specifying what format the client wants it in: content negotiation (Accept headers). We have a way of retrieving that data: HTTP GET. There's a reason why BugMeNot exists for websites. API keys are basically pointless registration pages for access to the same data that is being published on the web. As for the code samples: if there's little more needed than "here's the URL of our data", I don't need a code sample. I only need a code sample when it's been made ridiculously over-complicated. (My favourite API recently: clockworksms.com - all of the other SMS providers I've looked at want me to talk to some salesman and/or read complicated docs. Clockwork just let me send an HTTPS POST message. They have an API key, sure, but they required only an email address to get it. And I can pay for credits with PayPal. Ridiculously simple. I like.) As for arrogancy? Guilty as charged. I'll say in my defence that it's more that the data I wanted to retrieve from the service in question (which I won't name) was exceptionally simple, had no commercial value in itself, but would send referrals to the site that they could monetise (and there's no affiliate scheme, I wasn't gonna profit off this). There literally is no business reason to lock that kind of API down. It's just cargo-cult API design: everyone else has API keys, they must have a reason, so I better have that too.

overbroad 14y ago
"cargo-cult". That's the term I was looking for. Monkey see, monkey do. Code monkey mimicry. If thinking for yourself is "arrogance", then we need more "arrogant" people involved in web design. It's not the "API" that is the method of providing and restricting access. It's the IP address. Any reasonably smart user can figure out the "API". They do not need a lengthy manual. A few examples, mere hints, is all that is needed.
opminion 14y ago
Rants are always best with actual examples of what's being ranted. Just as http://tommorris.org/posts/2451 http://tommorris.org/posts/2451
- voyou 14y ago
  That post is totally wrong, though. There's no way in which URLs with opaque IDs in are better for humans than URLs with stubs. If the stub happens to jog a users memory about the content of the post, it might be useful, and even if it doesn't sum up the post very well, it's still no worse than an opaque ID.
  - tommorris 14y ago
    Try writing down "example.org/posts/1234" on a post-it note and then try writing down "example.org/posts/thing-with-a-long-name-with-lots-of-dashes". One is a lot more fun than the other.
    - 1349615402 14y ago
      see http://qntm.org/urls http://qntm.org/urls
georgebashi 14y ago
I think the point of API keys is so that if you were to build and publish an abusive client (think badly written iOS app), it's easier to stop the DDoS, rather than having to ban thousands of end-user IPs.
- tehwebguy 14y ago
  Absolutely! This helps stop developers from putting lazy API calls on the client side instead of caching, for example. For instance if my website includes a feed of my latest posts to a third party service I should be caching them myself and serving direct to visitors, rather than using a client side request coming from every visitor's browser. Obviously this sounds like common sense but you might be surprised :)
eps 14y ago
Psst... Not sure if you care, but viewing your site on iPad shows links overlapping with surrounding text, some text inexplicably clipped as well as few other rendering issues.
- tommorris 14y ago
  Yep, I need to fix that. Sorry about that. I was going to do iPad testing yesterday.
lazyjones 14y ago
In general, blocking IPs is not really useful anymore since far too much abuse comes over Tor, open proxies, or potentially short-lived dynamic IPs (major German ISPs let you switch IPs as often as you want) and if you care about legitimate users, you cannot block these. The point that websites without API-key get hit by this anyway is valid though and legitimate users won't be annoyed too much by IP-blocking abusers of the API for API access only (block all Tor exit nodes for the API too, who cares ...). You could also mention that authentication, registration etc. probably make APIs slower and more of a burden for the servers too ...
- tommorris 14y ago
  So, on the Tor front: Wikipedia has blocked Tor users from editing since, well, since forever. For the occasional Chinese dissident or whoever who actually wants to legitimately edit over Tor or another blocked IP from an account, they can request IP block exemption from an administrator. Another solution is to actually use User-Agent strings. Nominatim, OpenStreetMap's reverse geocoder, recommends that legitimate users put an email address in their User-Agent string. So it might read "My Craptastic Mashup v1. Maintained by: <whoever@gmail.com>". If there's a legitimate problem, email them.
steverb 14y ago
Wow. They charge 8 cents per text? Yeah, their API should come give you a massage at that price.
- tommorris 14y ago
  I don't particularly care about the price. I'm using it for things like sending myself a small number of simple notifications. I have a script that monitors a web app I maintain for work, and if it isn't responding, it sends me an SMS. When the site is down, people can't do any work. If they can't do any work, they lose clients. If they lose clients, I don't have a job anymore. If the point of using SMS were to actually build an SMS service, I'd spend more time worrying about it and choose one of the more fully-featured SMS providers. And if it were to be a commercial thing, sure, I'd talk to their business development people. But everything starts at the micro level: what could be a business idea in six months starts as a crappy little hack now. Here's a point of comparison. Esendex are usually considered one of the better SMS providers. How much do they cost? Oh, let's have a look at their pricing page. Doesn't tell me. Lots of blather. And a nice phone number I can ring and talk to a salesman or I can sign up for a no-commitment free trial. I don't want to talk to a salesman, I want to send a flipping text to myself in this crappy little Python script I'm writing.
ucee054 14y ago
A URL is not a way of saying where data is. It is a way of naming a data query. HTTP GET is not a way of retrieving data. It is a a way of launching a query. In the general case, the data is going to be in an enterprise datawarehouse divided over 200 tables each of which could be terabytes in size. The schema design will mean that each table individually is meaningless, and you will probably not understand it all without a lot of documentation. If you were allowed to pull a table using a GET it would almost certainly be very expensive for the service provider, and might kill your download connection. APIs were invented to deal with these cases.
- tommorris 14y ago
  The data is there in an HTML page. I'm looking at it. I just want to stick ".xml" or ".json" on the end and see the exact same thing in a machine readable form. Whatever complex enterprise data warehousing you are doing to get it in my browser as an HTML page is very impressive, I'm sure. It can be as complicated as you want it to be if you want to offer something more complicated than that. But for the basic use case, I fail to see why exactly it has to be more complicated than using HTTP to GET things that are on the damn web. And all the talk of APIs makes people think it needs to be complicated when it doesn't.
  - ucee054 14y ago
    On your web page you say "I just want the data on your website in a machine readable format. XML, JSON, RDF, CSV, YAML, I don’t particularly give a fuck. ... I don’t want a relationship with you, I just want to have a fiddle with your data." But in your comment you say "The data is there in an HTML page. ... I just want to stick .xml or .json on the end and see the exact same thing in a machine readable form." In the first case, you are asking for the absence of an API. In the second case, you are asking for an API that apes the user interface. I just wanted to point out that that's an important difference. EDIT: It is not correct to say the data is in the web page already, it is more accurate to say the needed query result is in the web page.
    - tommorris 14y ago

overbroad 14y ago

"cargo-cult". That's the term I was looking for. Monkey see, monkey do. Code monkey mimicry. If thinking for yourself is "arrogance", then we need more "arrogant" people involved in web design. It's not the "API" that is the method of providing and restricting access. It's the IP address. Any reasonably smart user can figure out the "API". They do not need a lengthy manual. A few examples, mere hints, is all that is needed.

opminion 14y ago

Rants are always best with actual examples of what's being ranted. Just as http://tommorris.org/posts/2451 http://tommorris.org/posts/2451

voyou 14y ago
That post is totally wrong, though. There's no way in which URLs with opaque IDs in are better for humans than URLs with stubs. If the stub happens to jog a users memory about the content of the post, it might be useful, and even if it doesn't sum up the post very well, it's still no worse than an opaque ID.
- tommorris 14y ago
  Try writing down "example.org/posts/1234" on a post-it note and then try writing down "example.org/posts/thing-with-a-long-name-with-lots-of-dashes". One is a lot more fun than the other.
  - 1349615402 14y ago
    see http://qntm.org/urls http://qntm.org/urls

georgebashi 14y ago

I think the point of API keys is so that if you were to build and publish an abusive client (think badly written iOS app), it's easier to stop the DDoS, rather than having to ban thousands of end-user IPs.

tehwebguy 14y ago
Absolutely! This helps stop developers from putting lazy API calls on the client side instead of caching, for example. For instance if my website includes a feed of my latest posts to a third party service I should be caching them myself and serving direct to visitors, rather than using a client side request coming from every visitor's browser. Obviously this sounds like common sense but you might be surprised :)

eps 14y ago

Psst... Not sure if you care, but viewing your site on iPad shows links overlapping with surrounding text, some text inexplicably clipped as well as few other rendering issues.

tommorris 14y ago
Yep, I need to fix that. Sorry about that. I was going to do iPad testing yesterday.

lazyjones 14y ago

In general, blocking IPs is not really useful anymore since far too much abuse comes over Tor, open proxies, or potentially short-lived dynamic IPs (major German ISPs let you switch IPs as often as you want) and if you care about legitimate users, you cannot block these. The point that websites without API-key get hit by this anyway is valid though and legitimate users won't be annoyed too much by IP-blocking abusers of the API for API access only (block all Tor exit nodes for the API too, who cares ...). You could also mention that authentication, registration etc. probably make APIs slower and more of a burden for the servers too ...

tommorris 14y ago
So, on the Tor front: Wikipedia has blocked Tor users from editing since, well, since forever. For the occasional Chinese dissident or whoever who actually wants to legitimately edit over Tor or another blocked IP from an account, they can request IP block exemption from an administrator. Another solution is to actually use User-Agent strings. Nominatim, OpenStreetMap's reverse geocoder, recommends that legitimate users put an email address in their User-Agent string. So it might read "My Craptastic Mashup v1. Maintained by: <whoever@gmail.com>". If there's a legitimate problem, email them.

steverb 14y ago

Wow. They charge 8 cents per text? Yeah, their API should come give you a massage at that price.

tommorris 14y ago
I don't particularly care about the price. I'm using it for things like sending myself a small number of simple notifications. I have a script that monitors a web app I maintain for work, and if it isn't responding, it sends me an SMS. When the site is down, people can't do any work. If they can't do any work, they lose clients. If they lose clients, I don't have a job anymore. If the point of using SMS were to actually build an SMS service, I'd spend more time worrying about it and choose one of the more fully-featured SMS providers. And if it were to be a commercial thing, sure, I'd talk to their business development people. But everything starts at the micro level: what could be a business idea in six months starts as a crappy little hack now. Here's a point of comparison. Esendex are usually considered one of the better SMS providers. How much do they cost? Oh, let's have a look at their pricing page. Doesn't tell me. Lots of blather. And a nice phone number I can ring and talk to a salesman or I can sign up for a no-commitment free trial. I don't want to talk to a salesman, I want to send a flipping text to myself in this crappy little Python script I'm writing.

ucee054 14y ago

A URL is not a way of saying where data is. It is a way of naming a data query. HTTP GET is not a way of retrieving data. It is a a way of launching a query. In the general case, the data is going to be in an enterprise datawarehouse divided over 200 tables each of which could be terabytes in size. The schema design will mean that each table individually is meaningless, and you will probably not understand it all without a lot of documentation. If you were allowed to pull a table using a GET it would almost certainly be very expensive for the service provider, and might kill your download connection. APIs were invented to deal with these cases.

tommorris 14y ago
The data is there in an HTML page. I'm looking at it. I just want to stick ".xml" or ".json" on the end and see the exact same thing in a machine readable form. Whatever complex enterprise data warehousing you are doing to get it in my browser as an HTML page is very impressive, I'm sure. It can be as complicated as you want it to be if you want to offer something more complicated than that. But for the basic use case, I fail to see why exactly it has to be more complicated than using HTTP to GET things that are on the damn web. And all the talk of APIs makes people think it needs to be complicated when it doesn't.
- ucee054 14y ago
  On your web page you say "I just want the data on your website in a machine readable format. XML, JSON, RDF, CSV, YAML, I don’t particularly give a fuck. ... I don’t want a relationship with you, I just want to have a fiddle with your data." But in your comment you say "The data is there in an HTML page. ... I just want to stick .xml or .json on the end and see the exact same thing in a machine readable form." In the first case, you are asking for the absence of an API. In the second case, you are asking for an API that apes the user interface. I just wanted to point out that that's an important difference. EDIT: It is not correct to say the data is in the web page already, it is more accurate to say the needed query result is in the web page.
  - tommorris 14y ago