5 ms·

Exactly. There's no "just" when you're adding an API. APIs require a lot of work and complexity to do right: APIs need to be well-designed or the client may ne

Exactly. There's no "just" when you're adding an API. APIs require a lot of work and complexity to do right: APIs need to be well-designed or the client may need to make multiple API calls when one should suffice. Or the API could be confusing and people will call it wrong or fail to use it at all. APIs require authentication and authorization. This means setting up OAuth2 or at least having a secure API token generation, storage, and validation. APIs need handle data securely or you'll leak data you shouldn't, or allow modification you shouldn't. APIs need to be performant or you database may be crushed under the load. This may involve caching which then adds the complexity of cache invalidation and other servers/processes to support caching. APIs need to be rate-limited or sloppy clients will hammer your API. APIs require thorough documentation or they're useless. You may even have to add SDKs (libraries for different programming languages) to make it easier for people to use the API. APIs need good error messages or users who are getting started will not know why their calls are failing.

eddythompson80 2y ago
lol, I can already hear the responses of plenty of managers/management I worked with to these points. > APIs need to be well-designed or the client may need to make multiple API calls when one should suffice. Or the API could be confusing and people will call it wrong or fail to use it at all. That's fine. It's a first iteration we can change it later if people complain. Lets get something out there and iterate. > APIs require authentication and authorization. Lets not worry about authorization/permissions. It's just one key or whatever account they use to log in with now. > APIs need handle data securely or you'll leak data you shouldn't, or allow modification you shouldn't. You're saying you don't know how to do that? > APIs need to be rate-limited or sloppy clients will hammer your API. Either "Lets worry about that later when it happens" or "Here is the first github link from a google result for 'api rate limit open source free'. Lets use that" > APIs require thorough documentation or they're useless. Either "We need to have the API first, docs, clis, etc could come later after we have gauged the usage or had asks for them. We can handhold the customers asking for them for now." or "here is a github project that autogenerates docs, SKDs and clis from an OpenAPI spec" > APIs need good error messages or users who are getting started will not know why their calls are failing. You're saying you don't know how to do that? "It doesn't have to be perfect. We have customers who are asking for it and to win them we need to implement something then we can work with them to improve it. Otherwise we will lose them"™
- liontwist 2y ago
  Yep, helps a lot when your management is in the domain. Phased implementation isn’t the worst idea, they just have to be committed to ending the experiment or seeing it through.
- 9dev 2y ago
  Then again, these answers are valid. Your fancy API documentation will only be skimmed, nobody is going to notice the elaborate and elegant URI design or the discussion around whether to use PUT or PATCH, and the super-secure token mechanism you come up with will be written on a sheet of paper in the office. Most projects are far too fluid in their shape to warrant a proper design up-front anyway.
  - lazyasciiart 2y ago
    They’re really not. That approach is how you get “why do I need this account ID to look it up in the API but the only way I can get it is by using the UI? Why can I update this object but not see its current state?” and other complaints that users very reasonably have about a thing I worked on.
    - 9dev 2y ago
      Don’t get me wrong, I love to obsess over that kind of detail, but complaints like these usually aren’t a deal breaker, and yet it’s reasonable to count those complaining users as successful conversions regardless.
  - lucianbr 2y ago
    Recently I was working on a 12-year old project where the product people kept asking for "quick wins" and "leave the refactorings and in-depth analysis and difficult problems for later". When will they get to the "later", if 12 years was not enough? I eventually left. Of course, YMMV. Every company and manager is different.
    - justin_oaks 2y ago
nyrikki 2y ago
I disagree, those are all technical issues that are typically blocked by socio-technical and organizational problems. Retrofitting an API is a context specific problem, with few things that generalize but the above aren't the hard parts, just the complex ones. I am going to point you to a Public Policy book here, look at page 32 and their concept of 'wicked problems', which applies to tech but doesn't suffer the problems with the consulting industry co-opting as much as in the tech world. https://library.oapen.org/bitstream/id/f3358d25-56c0-47f2-9083-73332713b964/978-3-030-94580-0.pdf https://library.oapen.org/bitstream/id/f3358d25-56c0-47f2-90... Obviously most companies won't accept the Amazon style API edicts that require Externalizable interfaces and force a product mindset and an outside in view. Obviously an API is still a "Cognitively complex problem" as defined by the above. But what makes most API projects fail is the fact that aspects arise that result in ‘wickedness’ A.K.A intractability. API gateways, WAFs, RESTful libs with exponential backoff with jitter....etc... all exist and work well and almost any system that you have a single DB you are concerned about killing is possible to just add a anti-corruption layer if you don't have too much code debt, complicated centralized orchestration etc.... But these general, but difficult tasks like adding an API almost never fail due to tech reasons, but due to politics, poor communication, focusing on tech and not users, unrealistic timelines, and turf battles. You are correct that "APIs require thorough documentation or they're useless" but more importantly they need enforceable standards on contract ownership, communication. The direction of control, stability, audience, and a dozen other factors effect the tradeoffs there as to what is appropriate. Typically that control is dictated purely by politics and not focused on outcomes. That is often what pushes these complex problems into failed initiatives, and obviously this is far more complicated. Often orgs can't deal with the very real uncertainty in these efforts and waste lots of project time on high effort, low value tasks like producing gantt charts that are so beyond the planning horizon that they can only result in bad outcomes at best. The point being is that unless you are on the frontier of knowledge and technical capabilities, it is almost never the tech that causes these efforts to fail.
- harrall 2y ago
  When I read OP’s list of wants for an API, I don’t read it so literally. I read it as an example of the complexity required to implement something like an API. I agree that it’s also a people problem but I disagree that many of these failed initiatives aren’t due to bad tech as well. A complex initiative requires picking the right tech that gives you a good benefit/cost ratio. Knowledge of tech plays hand in hand with the political aspects and IMO many orgs lack both completely.
deleted 2y ago
[deleted]
Groxx 2y ago
Every single one of these is true of a website, but we have far more websites than APIs. If there's one thing APIs suffer from more, it's "social cost to make changes". A concurrent vN+1 largely resolves that though, unless your API consuming ecosystem is large enough to be worth investing vastly more resources into.
- saghm 2y ago
  Is "let's just make a website?" a common thing people say? It sounds like the issue isn't with "make an API", it's with "just".
mdaniel 2y ago
And there's an implied risk that once you have added an API for customers to exfiltrate data or integrate with their own system (or, worse, IMHO: a competitor's tool) then they'll never return to your site again I'm cognizant that if it truly is a make-or-break for the deal, there may not be any choice, but along with all the risks you cited is an underlying obsolescence one

eddythompson80 2y ago

lol, I can already hear the responses of plenty of managers/management I worked with to these points. > APIs need to be well-designed or the client may need to make multiple API calls when one should suffice. Or the API could be confusing and people will call it wrong or fail to use it at all. That's fine. It's a first iteration we can change it later if people complain. Lets get something out there and iterate. > APIs require authentication and authorization. Lets not worry about authorization/permissions. It's just one key or whatever account they use to log in with now. > APIs need handle data securely or you'll leak data you shouldn't, or allow modification you shouldn't. You're saying you don't know how to do that? > APIs need to be rate-limited or sloppy clients will hammer your API. Either "Lets worry about that later when it happens" or "Here is the first github link from a google result for 'api rate limit open source free'. Lets use that" > APIs require thorough documentation or they're useless. Either "We need to have the API first, docs, clis, etc could come later after we have gauged the usage or had asks for them. We can handhold the customers asking for them for now." or "here is a github project that autogenerates docs, SKDs and clis from an OpenAPI spec" > APIs need good error messages or users who are getting started will not know why their calls are failing. You're saying you don't know how to do that? "It doesn't have to be perfect. We have customers who are asking for it and to win them we need to implement something then we can work with them to improve it. Otherwise we will lose them"™

liontwist 2y ago
Yep, helps a lot when your management is in the domain. Phased implementation isn’t the worst idea, they just have to be committed to ending the experiment or seeing it through.
9dev 2y ago
Then again, these answers are valid. Your fancy API documentation will only be skimmed, nobody is going to notice the elaborate and elegant URI design or the discussion around whether to use PUT or PATCH, and the super-secure token mechanism you come up with will be written on a sheet of paper in the office. Most projects are far too fluid in their shape to warrant a proper design up-front anyway.
- lazyasciiart 2y ago
  They’re really not. That approach is how you get “why do I need this account ID to look it up in the API but the only way I can get it is by using the UI? Why can I update this object but not see its current state?” and other complaints that users very reasonably have about a thing I worked on.
  - 9dev 2y ago
    Don’t get me wrong, I love to obsess over that kind of detail, but complaints like these usually aren’t a deal breaker, and yet it’s reasonable to count those complaining users as successful conversions regardless.
- lucianbr 2y ago
  Recently I was working on a 12-year old project where the product people kept asking for "quick wins" and "leave the refactorings and in-depth analysis and difficult problems for later". When will they get to the "later", if 12 years was not enough? I eventually left. Of course, YMMV. Every company and manager is different.
  - justin_oaks 2y ago

nyrikki 2y ago

I disagree, those are all technical issues that are typically blocked by socio-technical and organizational problems. Retrofitting an API is a context specific problem, with few things that generalize but the above aren't the hard parts, just the complex ones. I am going to point you to a Public Policy book here, look at page 32 and their concept of 'wicked problems', which applies to tech but doesn't suffer the problems with the consulting industry co-opting as much as in the tech world. https://library.oapen.org/bitstream/id/f3358d25-56c0-47f2-9083-73332713b964/978-3-030-94580-0.pdf https://library.oapen.org/bitstream/id/f3358d25-56c0-47f2-90... Obviously most companies won't accept the Amazon style API edicts that require Externalizable interfaces and force a product mindset and an outside in view. Obviously an API is still a "Cognitively complex problem" as defined by the above. But what makes most API projects fail is the fact that aspects arise that result in ‘wickedness’ A.K.A intractability. API gateways, WAFs, RESTful libs with exponential backoff with jitter....etc... all exist and work well and almost any system that you have a single DB you are concerned about killing is possible to just add a anti-corruption layer if you don't have too much code debt, complicated centralized orchestration etc.... But these general, but difficult tasks like adding an API almost never fail due to tech reasons, but due to politics, poor communication, focusing on tech and not users, unrealistic timelines, and turf battles. You are correct that "APIs require thorough documentation or they're useless" but more importantly they need enforceable standards on contract ownership, communication. The direction of control, stability, audience, and a dozen other factors effect the tradeoffs there as to what is appropriate. Typically that control is dictated purely by politics and not focused on outcomes. That is often what pushes these complex problems into failed initiatives, and obviously this is far more complicated. Often orgs can't deal with the very real uncertainty in these efforts and waste lots of project time on high effort, low value tasks like producing gantt charts that are so beyond the planning horizon that they can only result in bad outcomes at best. The point being is that unless you are on the frontier of knowledge and technical capabilities, it is almost never the tech that causes these efforts to fail.

harrall 2y ago
When I read OP’s list of wants for an API, I don’t read it so literally. I read it as an example of the complexity required to implement something like an API. I agree that it’s also a people problem but I disagree that many of these failed initiatives aren’t due to bad tech as well. A complex initiative requires picking the right tech that gives you a good benefit/cost ratio. Knowledge of tech plays hand in hand with the political aspects and IMO many orgs lack both completely.

deleted 2y ago

[deleted]

Groxx 2y ago

Every single one of these is true of a website, but we have far more websites than APIs. If there's one thing APIs suffer from more, it's "social cost to make changes". A concurrent vN+1 largely resolves that though, unless your API consuming ecosystem is large enough to be worth investing vastly more resources into.

saghm 2y ago
Is "let's just make a website?" a common thing people say? It sounds like the issue isn't with "make an API", it's with "just".

mdaniel 2y ago

And there's an implied risk that once you have added an API for customers to exfiltrate data or integrate with their own system (or, worse, IMHO: a competitor's tool) then they'll never return to your site again I'm cognizant that if it truly is a make-or-break for the deal, there may not be any choice, but along with all the risks you cited is an underlying obsolescence one