5 ms·

I’m gonna say it: Many rest apis are lazy and developer friendly, not consumer friendly. If you have related resources, let’s say, product and product option

I’m gonna say it: Many rest apis are lazy and developer friendly, not consumer friendly. If you have related resources, let’s say, product and product options as two distinct endpoints: - /api/product - /api/options Then, and I want to be clear here, it is impossible for a client to perform an atomic operation on multiple distinct objects types. Let’s say the client needs to add a product with a single option or fail. You can create a product. You can create an option. You can add an option to a product. …but, at some point in time, a product will exist with no options on it. This kind of “pure” rest api is simply convenient to the developer because they push the problem of object consistency to the client. …but that’s not a client concern. If a product needs to be associated with an option on creation, then your api should offer that as an endpoint. It doesn’t meet the “standard” for a rest api? Too bad. Write APIs that do what the customer / consumer needs not lazy APIs that just make your life easier. I’ve worked with a lot of APIs and you know what I do not give the tiniest moment of care about? Consistency. I do. Not. Care. If POST /api/foo creates an object, or if the endpoint is /api/foo/create. Messy? Inconsistent? I don’t care. Just put it in the documentation and I’ll find it and use it. …but if your api forces object relational consistency onto me as an api consumer, you have no idea how much pain and hassle you’ve caused me having to implement my own set of fake transactions over your stupid api. Please, write APIs for consumers, not according to “the rules” of rest APIs. …and provide documentation. :)

1123581321 5y ago
Agree that the tendency exists but don't see the tradeoff between transactional and messy in this example--you should be able to create a new product with options [...] in the same request, perhaps using /options to ascertain what's available before posting to /products, depending on the situation.
- shoo 5y ago
  maybe a more illustrative example of where this can be messy is where a customer has placed an order for a bundle of products A B and C, and there's no "order bundle" API that can do that operation atomically, instead the client code trying to place the order has to call a bunch of random stateful create-product APIs to attempt to construct each product in the bundle, and deal with the mess if suddenly some bundle items error while others succeed. bonus points if some of the create-product APIs are not idempotent. "congratulations, we've provisioned the bundle A, C, C, C that you ordered! sorry about B!"
underwater 5y ago
Generally internal APIs are developed alongside one or two apps. They don't need a pure, resource-oriented, API that perfectly represents the domain models but ignores how the API is used in practice. A good example is the tip on PUT vs PATCH to update objects. That seems to be missing the point. Why are you forcing the clients to calculate the correct resource fields to PATCH to the server? This is supposed to be an API, not a database. Just expose methods that correspond to the actions that the users will perform. Sure, HTTP only has 5 verbs, but that doesn't mean your API should solely consist of five possible interactions on a resource.
- shoo 5y ago
  structured programming and rpc demonstrates you can get pretty far if your only verb is CALL
- kortex 5y ago
  > Sure, HTTP only has 5 verbs More like 40. You can also just like...make up your own verbs, the HTTP police won't arrest you, though this rustles the jimmies of HTTP purists. No guarantees any middleware will support it, and this is probably a bad idea if you actually do this in public apis, it's more so "hey, these verbs aren't magic incantations, it's just a string which gets switch-cased". Same with HTTP status codes. There's obviously some well known ones but you can just like...make up your own. https://www.iana.org/assignments/http-methods/http-methods.xhtml https://www.iana.org/assignments/http-methods/http-methods.x...
shoo 5y ago
i was in the process of writing a snarky reply describing how the next level enterprise approach is to host api/product in microservice A and api/options in microservice B, each with their own internal data store using completely different tech stacks. but, if you've already suffered through the pain of attempting to implement some kind of best-effort ad-hoc distributed commit logic in your client code, then needing to call two completely different services for api/product and api/options doesn't really make anything _worse_. on another hand, it doesn't make anything better.
parksy 5y ago
Everyone has different experiences but my view of the problem is that it's not so much down to developer laziness as it is technological ignorance and reactive direction from higher up - the people who actually fund and approve the work. Outside of the bigger providers I don't think most APIs are designed with general consumption in mind. The impetus usually seems to be reactive - "we've got this idea for a mobile app using data from our legacy WMS/CRM/whatever, and it needs to be finished yesterday!" So generally (not always but usually) what I've seen is whatever the underlying system supports and requires gets reflected into the API, and there's minimal proper engineering or sensible abstraction, partly because of time limits and also because the requirements are discovered mid-flight. "Oh... Products have options in a separate entity? Oops. Uh, we'll add an options endpoint. Easy." Several years later, the app is mildly successful, the business decides to whitelabel the API for some of their B2B clients, there's no budget for a redesign, "It works doesn't it? If it ain't broke don't fix it..." Where possible I try to design and build APIs with some forethought and push for as much information on requirements and capabilities of existing systems beforehand but quite often the stakeholders don't really know and don't want to pay for a few weeks of in-depth analysis. The endless facepalming of hearing the cycle of "Don't overengineer it" when I ask too many questions people can't answer, through to "Why didn't we pick up on this requirement" and having to listen to "we can all learn from this" when something fails because of some if/else statement buried in the legacy business logic, right back to "Don't overengineer it" when you propose the clean way of encapsulating that in the API, has left a permanent indent on my skull. So much leakage of black-box logic into the clients which bloats frontend logic and provides plenty of double handling and scope for unknown business logic to be missed, and no one in charge really gives a shit other than the developers and engineers, and that's fine because we can deal with it and it's their fault for not thinking of it anyway... they're the smart ones right? You'd think this is just a problem with SMEs but some of the biggest national and multinationals I've worked with have a ton of horrid shit in their API designs. One industry-wide government-mandated reporting spec you'd post a SOAP envelope that wraps an undocumented XML document (they had an example document and the envelope was documented, but no doctype or anything to go off for the contents), a major real estate interchange format has you polling a REST API for an update flag, then loading and saving CSVs over FTP. Some godawful affiliate marketing API essentially just exposed their nth normal form relational database as an API. One government department just used HTSQL for their API using a python script to reflect some of their silos into PGSQL and called it a day (even though whether or not your interactions would work or made sense depended entirely on their internal application logic which wasn't represented or documented). And too many projects where it turns out the promised APIs didn't even exist until the app was well into development. "Don't worry about it, that's their problem and gives us an excuse when the inevitable delays occur..." No wonder we're hearing of massive breaches daily, from top to bottom the entire industry is barely held together by glue and sticky tape. It seems like an engineering problem but I think it goes much higher than that, it's a cross-industry failure in planning and management. Engineers are not given the time and budget to do the job that the data deserves. The market self corrects though, massive breaches have closed more than one business down for good. It feels futile but I do support and encourage people to work towards just reasonable and well-documented endpoints, I doubt there's one true standard, but just some level above slapping it together at the last minute would be nice. I don't care if it's JSON or XML, or whether paths follow some semantics or another. As long as the relationships, types, and mandatory vs optional fields are explained clearly and succinctly, and the auth mechanism isn't a nightmare, and I can work with it. Sorry for the rant this is just one area that triggers me. I've seen too much bad practice, and fighting against it for decades and still hearing the same lack of concern in the architecture phase has left me more than a little jaded.
alkonaut 5y ago
If options are children of products and not many-to-many then the only logical way to have two endpoints in the API is if the parent (Aggregate root in DDD speak) items are readable/writable as a consistent tree, while the query for child items in /api/options is a read only api. There is nothing nonstandard about such a REST api. If a product must be associated with an option then the /product/create endpoint should only accept a product with an option already attached. > This kind of “pure” rest api is simply convenient to the developer because they push the problem of object consistency to the client. It's not like REST means that to be "pure" or "standard" you end up exposing all your database tables with read/write and just let callers CRUD anything they want, including creating inconsistent data? > “the rules” of rest APIs. The rules are very simple: the REST api should expose and enforce the consistency rules of the business logic!
bmn__ 5y ago
> it is impossible for a client to perform an atomic operation on multiple distinct objects types. > It doesn’t meet the “standard” for a rest api? Too bad. It is perfectly fine to model the creation of a product with options via a new resource. Another possibility is to model this as a transaction, a series of requests. Both meet the goals of atomicity and also following restful constraints. But since you disallowed those possibilities as the premise, there is no constructive way forward from there. Nice construction of a straw-man, must feel satisfying to see it topple over, but who do you hope to impress with that?

1123581321 5y ago

Agree that the tendency exists but don't see the tradeoff between transactional and messy in this example--you should be able to create a new product with options [...] in the same request, perhaps using /options to ascertain what's available before posting to /products, depending on the situation.

shoo 5y ago
maybe a more illustrative example of where this can be messy is where a customer has placed an order for a bundle of products A B and C, and there's no "order bundle" API that can do that operation atomically, instead the client code trying to place the order has to call a bunch of random stateful create-product APIs to attempt to construct each product in the bundle, and deal with the mess if suddenly some bundle items error while others succeed. bonus points if some of the create-product APIs are not idempotent. "congratulations, we've provisioned the bundle A, C, C, C that you ordered! sorry about B!"

underwater 5y ago

Generally internal APIs are developed alongside one or two apps. They don't need a pure, resource-oriented, API that perfectly represents the domain models but ignores how the API is used in practice. A good example is the tip on PUT vs PATCH to update objects. That seems to be missing the point. Why are you forcing the clients to calculate the correct resource fields to PATCH to the server? This is supposed to be an API, not a database. Just expose methods that correspond to the actions that the users will perform. Sure, HTTP only has 5 verbs, but that doesn't mean your API should solely consist of five possible interactions on a resource.

shoo 5y ago
structured programming and rpc demonstrates you can get pretty far if your only verb is CALL
kortex 5y ago
> Sure, HTTP only has 5 verbs More like 40. You can also just like...make up your own verbs, the HTTP police won't arrest you, though this rustles the jimmies of HTTP purists. No guarantees any middleware will support it, and this is probably a bad idea if you actually do this in public apis, it's more so "hey, these verbs aren't magic incantations, it's just a string which gets switch-cased". Same with HTTP status codes. There's obviously some well known ones but you can just like...make up your own. https://www.iana.org/assignments/http-methods/http-methods.xhtml https://www.iana.org/assignments/http-methods/http-methods.x...

shoo 5y ago

i was in the process of writing a snarky reply describing how the next level enterprise approach is to host api/product in microservice A and api/options in microservice B, each with their own internal data store using completely different tech stacks. but, if you've already suffered through the pain of attempting to implement some kind of best-effort ad-hoc distributed commit logic in your client code, then needing to call two completely different services for api/product and api/options doesn't really make anything _worse_. on another hand, it doesn't make anything better.

parksy 5y ago

Everyone has different experiences but my view of the problem is that it's not so much down to developer laziness as it is technological ignorance and reactive direction from higher up - the people who actually fund and approve the work. Outside of the bigger providers I don't think most APIs are designed with general consumption in mind. The impetus usually seems to be reactive - "we've got this idea for a mobile app using data from our legacy WMS/CRM/whatever, and it needs to be finished yesterday!" So generally (not always but usually) what I've seen is whatever the underlying system supports and requires gets reflected into the API, and there's minimal proper engineering or sensible abstraction, partly because of time limits and also because the requirements are discovered mid-flight. "Oh... Products have options in a separate entity? Oops. Uh, we'll add an options endpoint. Easy." Several years later, the app is mildly successful, the business decides to whitelabel the API for some of their B2B clients, there's no budget for a redesign, "It works doesn't it? If it ain't broke don't fix it..." Where possible I try to design and build APIs with some forethought and push for as much information on requirements and capabilities of existing systems beforehand but quite often the stakeholders don't really know and don't want to pay for a few weeks of in-depth analysis. The endless facepalming of hearing the cycle of "Don't overengineer it" when I ask too many questions people can't answer, through to "Why didn't we pick up on this requirement" and having to listen to "we can all learn from this" when something fails because of some if/else statement buried in the legacy business logic, right back to "Don't overengineer it" when you propose the clean way of encapsulating that in the API, has left a permanent indent on my skull. So much leakage of black-box logic into the clients which bloats frontend logic and provides plenty of double handling and scope for unknown business logic to be missed, and no one in charge really gives a shit other than the developers and engineers, and that's fine because we can deal with it and it's their fault for not thinking of it anyway... they're the smart ones right? You'd think this is just a problem with SMEs but some of the biggest national and multinationals I've worked with have a ton of horrid shit in their API designs. One industry-wide government-mandated reporting spec you'd post a SOAP envelope that wraps an undocumented XML document (they had an example document and the envelope was documented, but no doctype or anything to go off for the contents), a major real estate interchange format has you polling a REST API for an update flag, then loading and saving CSVs over FTP. Some godawful affiliate marketing API essentially just exposed their nth normal form relational database as an API. One government department just used HTSQL for their API using a python script to reflect some of their silos into PGSQL and called it a day (even though whether or not your interactions would work or made sense depended entirely on their internal application logic which wasn't represented or documented). And too many projects where it turns out the promised APIs didn't even exist until the app was well into development. "Don't worry about it, that's their problem and gives us an excuse when the inevitable delays occur..." No wonder we're hearing of massive breaches daily, from top to bottom the entire industry is barely held together by glue and sticky tape. It seems like an engineering problem but I think it goes much higher than that, it's a cross-industry failure in planning and management. Engineers are not given the time and budget to do the job that the data deserves. The market self corrects though, massive breaches have closed more than one business down for good. It feels futile but I do support and encourage people to work towards just reasonable and well-documented endpoints, I doubt there's one true standard, but just some level above slapping it together at the last minute would be nice. I don't care if it's JSON or XML, or whether paths follow some semantics or another. As long as the relationships, types, and mandatory vs optional fields are explained clearly and succinctly, and the auth mechanism isn't a nightmare, and I can work with it. Sorry for the rant this is just one area that triggers me. I've seen too much bad practice, and fighting against it for decades and still hearing the same lack of concern in the architecture phase has left me more than a little jaded.

alkonaut 5y ago

If options are children of products and not many-to-many then the only logical way to have two endpoints in the API is if the parent (Aggregate root in DDD speak) items are readable/writable as a consistent tree, while the query for child items in /api/options is a read only api. There is nothing nonstandard about such a REST api. If a product must be associated with an option then the /product/create endpoint should only accept a product with an option already attached. > This kind of “pure” rest api is simply convenient to the developer because they push the problem of object consistency to the client. It's not like REST means that to be "pure" or "standard" you end up exposing all your database tables with read/write and just let callers CRUD anything they want, including creating inconsistent data? > “the rules” of rest APIs. The rules are very simple: the REST api should expose and enforce the consistency rules of the business logic!

bmn__ 5y ago

> it is impossible for a client to perform an atomic operation on multiple distinct objects types. > It doesn’t meet the “standard” for a rest api? Too bad. It is perfectly fine to model the creation of a product with options via a new resource. Another possibility is to model this as a transaction, a series of requests. Both meet the goals of atomicity and also following restful constraints. But since you disallowed those possibilities as the premise, there is no constructive way forward from there. Nice construction of a straw-man, must feel satisfying to see it topple over, but who do you hope to impress with that?