7 ms·

Implementing Microsoft REST API Filter

67 points by skibish 4y ago

recursivedoubts 4y ago
don't make me tap the sign: https://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven https://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypert... > I am getting frustrated by the number of people calling any HTTP-based interface a REST API. Today’s example is the SocialSite REST API. That is RPC. It screams RPC. There is so much coupling on display that it should be given an X rating. > What needs to be done to make the REST architectural style clear on the notion that hypertext is a constraint?
- voidfunc 4y ago
  You can tap the sign as much as you want, that battle was lost a long time ago. REST is just the common term people use for HTTP+JSON RPC.
  - recursivedoubts 4y ago
    DONT THINK I WONT
  - lkxijlewlf 4y ago
    "Ugh... REST is too much work! We'll just change the meaning to suit us."
    - lliamander 4y ago
      Why should people do hypermedia if it is too much work? The Web (the prototypical REST application) had a lot of good ideas, not just hypermedia. It certainly makes sense to attribute those good ideas to REST.
    - SideburnsOfDoom 4y ago
      > "Ugh... REST is too much work! We'll just change the meaning to suit us." See also: Agile. DevOps. It's pretty common once you look.
  - rafale 4y ago
    You gotta say RESTful if you want to emphasis RESTfulness. But I personally don't care. I don't mind JSON RPC.
    - masklinn 4y ago
      Then say that? Rest pedants don’t care if you don’t use rest, as long as you don’t call it rest. It’s hardly a difficult concept.
      - voidfunc 4y ago
        Yea, but nobody cares what REST pedants think.
  - jgalt212 4y ago
    True that. I use it incorrectly with non-technical folk as they may understand what a REST API is, but when I say "it's more like an RPC interface" their eyes glaze over.
- deathanatos 4y ago
  … and it's not just the hyperlinking that's problematic. (Yeah, MS's unRESTful "RESTful" APIs do a huge amount of coupling in the form of URL building.) E.g., in Azure, which is also a "RESTful" API that has no idea what REST is about, MS completely misses Fielding points that most of the effort of definition should be spent defining the content / data's format, not things like URL structure. That way we can speak about MIME types / content-types, and know what structure we're describing. But Azure will happily describe in JSONSchema a single type, and declare that it is used for both PUT/GET, and … it's not. And discovering the additional constraints that exist on the type in the PUT is gleaned only through calling the API, certainly not through Azure's docs. And that's assuming you get a usable error in response. JSONSchema is also a bit of a disappointment. On the one hand — yay, a spec? But on the other hand, it fails to capture so, so much. Half the fields in the type will be required … and the schema will say they're optional. Sum types of any kind are particularly badly handled, and half the time are just "string" though I think this is more of a failing on MS/Azure than JSONSchema, for simple string-like enums; but more complicated sum types, IDK if JSONSchema can't cut it or if MS just doesn't get it or what. For example, to instantiate a VM, the request body looks something like: body: required struct { properties: optional struct { storageProfile: optional struct { imageReference: optional struct { communityGalleryImageId: optional string, exactVersion: optional string, id: optional string, offer: optional string, publisher: optional string, sharedGalleryImageId: optional string, sku: optional string, version: optional string, } osDisk: optional struct { createOption: optional enum { "Attach", "Empty", "FromImage" } image: optional struct { uri: optional string, } managedDisk: optionalStruct { id: optional string, // omitted fields } vhd: optional struct { uri: optional string, } } // omitted fields } // omitted fields } // omitted fields } I've listed only the fields used in determining where to source the VM's OS disk from. And it's nuts! "properties" and "osDisk" are actually required; if you specify "imageReference" or "image" or probably "vhd" (but I've never used that myself), "createOption" must be "FromImage", if you specify "managedDisk" it must be "Attach", and the docs don't describe what meaning "Empty" has. You can specify only one of those, because otherwise, you're saying to source the image from two things which would be nonsense (but is permitted by schema/docs?). "imageReference" itself is really a sum type; you must specify (offer, publisher, sku, version[, exactVersion]), or communityGalleryImageId, or sharedGalleryImageId. You could image it being, enum ImageReference { FromMarketplace { offer: String, publisher: String, sku: String, version: String, exactVersion: Option<String> }, SharedGallery(String), CommunityGallery(String), } And we've not even touched VHDs, managed disks, or VM images yet! And you don't need createOption. I think, again, I'm going off what I've learned the hard way about how Azure works. I shudder to think what the validation logic looks like. (I'm also reading the docs. Reading JSONSchema is … painful to start with, but Azure's schema's directory layout structure makes it triply painful.) But even that sum type is to miss the point of REST entirely. The RESTful definition would be: image: URI-reference and that's it. The Content-Type of the content at the provided URI provides the type of image that it is. Oh and while I'm here: don't choose a boneheaded page size if you paginate an API call. Half of Azure's services will trickle-feed you 100 records at a time, and so the response body is like 60 KiB. Since the payload also has the next page's URI, your calls get decimated by latency. Some bad offenders: listing images in a repo in ACR gets ~ a phone modems worth of overall throughput. It takes minutes to download single-digit megabytes of image metadata. The Azure pricing APIs are similar: it's ~58KiB per page. The entire VM pricing data is something like 131 MiB, and that requires 2,235 HTTP calls to fetch.
- derefr 4y ago
  A true REpresentational State Transfer API would give each version of each of the API's L7-protocol request and response messages its own media-(sub)type; and would then rely on HTTP content negotiation to allow clients to specify which messages they're expecting to receive in response to a request; where the difference between throwing/nonthrowing, sync/async, new/old variant, etc., would all come down to which response message types the client is declaring themselves to accept. Don't think I've literally ever seen this done in practice. Don't think I've ever seen a "REST client" library that even offers the possibility of negotiating with a backend that's attempting to do this — let alone doing it in a streamlined manner.
  - recursivedoubts 4y ago
    no, the whole media type thing was a moderately-interesting-idea-turned-ridiculous-navel-gazing turn that, among many other pedantic blind alleys, turned everyone away from the deeply innovative aspect of REST: the uniform interface, and, in particular, HATEOAS https://htmx.org/essays/hateoas/ https://htmx.org/essays/hateoas/ any idiot (such as myself) who has ever made a web 1.0 app has created a better REST API than 99.9% of all REST API engineers, so called
    - lliamander 4y ago
      I have created Hypermedia APIs using HTTP/JSON for service-to-service interactions, and it is definitely a lot of work for questionable benefit. The biggest problem with the bad style of RPC-style programming was the assumption that remote calls could be treated like local calls. Modern HTTP-based RPC APIs don't make that assumption and thr problems of distributed computing are handled explicitly. The semantics of HTTP verbs, semantically meaningful namespace trees for resources in the URL, standard authentication and encoding, etc are all nice benefits of HTTP APIs even without hypermedia. That said, I haven't given up on hypermedia APIs! I liken them to "fluent" APIs of OOP. It's a powerful design approach, but I just wonder how to capture the benefits of that approach over what most people are doing today.
      - pdhborges 4y ago
- vxNsr 4y ago
  Hey, I’m kinda new to this area, can you show with examples what the differences are? I read the article, he does a lot of explaining but doesn’t do much showing. Like why is a specific api more RPC and why a different one is a perfectly constructed REST api?
  - recursivedoubts 4y ago
    REST was the term Roy Fielding coined as description of the original web architecture. The big distinguishing feature of REST-ful systems vs. non-REST-ful systems was something called the uniform interface, where clients had zero knowledge about a given server's API surface beyond a URL entry point. Everything after that was encoded in hypermedia, giving us the term "Hypermedia As The Engine of Application State" (HATEOAS). Here is an article I wrote on it: https://htmx.org/essays/hateoas/ https://htmx.org/essays/hateoas/ RPC-style APIs require shared knowledge about a given end point: what arguments it expects, what data it returns, etc. Unfortunately, for historical reasons, we've come to call all HTTP JSON APIs "REST APIs". It's actually pretty funny.
    - jackling 4y ago
      Thanks! I was really confused with this topic, and your post cleared things up. But from what I gather, proper REST API's wouldn't pair nicely with modern day UI development. Maybe this is because I usually work with APIs that wouldn't be considered restful (from my understanding, my company's public API is here [https://developer.veevavault.com/api/22.1/ https://developer.veevavault.com/api/22.1/]), I have a hard time seeing how a modern React webapp would parse the HTTP form response given in your article.
      - recursivedoubts 4y ago
        That's right, REST concepts are largely wasted on modern SPA applications, which are fundamentally RPC-style clients. Instead, REST should be used in its original context: hypermedia clients (browsers) exchanging hypermedia (HTML) with the server. I have written an alternative front end library that takes this exact approach: https://htmx.org https://htmx.org Some more essays on the topic: https://htmx.org/essays/spa-alternative/ https://htmx.org/essays/spa-alternative/ https://htmx.org/essays/hypermedia-apis-vs-data-apis/ https://htmx.org/essays/hypermedia-apis-vs-data-apis/
- Lazare 4y ago
  One suggestion I've seen (eg here: https://cloud.google.com/blog/products/api-management/understanding-grpc-openapi-and-rest-and-when-to-use-them https://cloud.google.com/blog/products/api-management/unders...) is to distinguish between REST and OpenAPI designs. Because you're right, REST is a specific thing, but we do need some name for what everyone is talking about, and just saying "RPC" is much too vague.
- criddell 4y ago
  Why isn’t the meaning of REST allowed to drift? In the comment section of that page Roy mentions how the meaning of hypertext has drifted over time. Why shouldn’t REST evolve as well?
  - wizofaus 4y ago
    It has, regardless of what Roy believes - I've never come across any API that calls itself RESTful that fits with his idea of what REST is, nor is it clear what advantages it would have.
- zamalek 4y ago
  REST isn't REST anymore. Barely anyone does it "correctly." The term has gained an overload: anything that happens over HTTP that OpenAPI is likely able to describe. REmote Structured Transactions. Problem solved.
- aliswe 4y ago
  If a teacher has one failing student, blame the student. If a teacher has 99 failing students, blame the teacher.
  - yencabulator 4y ago
    Or, in this case, maybe: If the teacher has no students, blame the topic. People just don't want HATEOAS as much as some would like.
revskill 4y ago
To me, REST focus on Resource first. RPC is Producedure first.
qxmat 4y ago
I did the same a while back. I used ANTLR4 to generate a .NET tree walker that would build up an IQueryProvider expression. IQueryProvider would then compile to Expression<T> and self-optimise (simplifying boolean algebra and removing redundant expressions). We hooked this up to npgsql and viola - a sane, typesafe, query string DSL without a single line of SQL. Nowadays I hack together Terraform and Python ML- I miss .NET dearly... it was a simpler time.

67 points by skibish 4y ago

don't make me tap the sign: https://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven https://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypert... > I am getting frustrated by the number of people calling any HTTP-based interface a REST API. Today’s example is the SocialSite REST API. That is RPC. It screams RPC. There is so much coupling on display that it should be given an X rating. > What needs to be done to make the REST architectural style clear on the notion that hypertext is a constraint?

voidfunc 4y ago
You can tap the sign as much as you want, that battle was lost a long time ago. REST is just the common term people use for HTTP+JSON RPC.
- recursivedoubts 4y ago
  DONT THINK I WONT
- lkxijlewlf 4y ago
  "Ugh... REST is too much work! We'll just change the meaning to suit us."
  - lliamander 4y ago
    Why should people do hypermedia if it is too much work? The Web (the prototypical REST application) had a lot of good ideas, not just hypermedia. It certainly makes sense to attribute those good ideas to REST.
  - SideburnsOfDoom 4y ago
    > "Ugh... REST is too much work! We'll just change the meaning to suit us." See also: Agile. DevOps. It's pretty common once you look.
- rafale 4y ago
  You gotta say RESTful if you want to emphasis RESTfulness. But I personally don't care. I don't mind JSON RPC.
  - masklinn 4y ago
    Then say that? Rest pedants don’t care if you don’t use rest, as long as you don’t call it rest. It’s hardly a difficult concept.
    - voidfunc 4y ago
      Yea, but nobody cares what REST pedants think.
- jgalt212 4y ago
  True that. I use it incorrectly with non-technical folk as they may understand what a REST API is, but when I say "it's more like an RPC interface" their eyes glaze over.
deathanatos 4y ago
… and it's not just the hyperlinking that's problematic. (Yeah, MS's unRESTful "RESTful" APIs do a huge amount of coupling in the form of URL building.) E.g., in Azure, which is also a "RESTful" API that has no idea what REST is about, MS completely misses Fielding points that most of the effort of definition should be spent defining the content / data's format, not things like URL structure. That way we can speak about MIME types / content-types, and know what structure we're describing. But Azure will happily describe in JSONSchema a single type, and declare that it is used for both PUT/GET, and … it's not. And discovering the additional constraints that exist on the type in the PUT is gleaned only through calling the API, certainly not through Azure's docs. And that's assuming you get a usable error in response. JSONSchema is also a bit of a disappointment. On the one hand — yay, a spec? But on the other hand, it fails to capture so, so much. Half the fields in the type will be required … and the schema will say they're optional. Sum types of any kind are particularly badly handled, and half the time are just "string" though I think this is more of a failing on MS/Azure than JSONSchema, for simple string-like enums; but more complicated sum types, IDK if JSONSchema can't cut it or if MS just doesn't get it or what. For example, to instantiate a VM, the request body looks something like: body: required struct { properties: optional struct { storageProfile: optional struct { imageReference: optional struct { communityGalleryImageId: optional string, exactVersion: optional string, id: optional string, offer: optional string, publisher: optional string, sharedGalleryImageId: optional string, sku: optional string, version: optional string, } osDisk: optional struct { createOption: optional enum { "Attach", "Empty", "FromImage" } image: optional struct { uri: optional string, } managedDisk: optionalStruct { id: optional string, // omitted fields } vhd: optional struct { uri: optional string, } } // omitted fields } // omitted fields } // omitted fields } I've listed only the fields used in determining where to source the VM's OS disk from. And it's nuts! "properties" and "osDisk" are actually required; if you specify "imageReference" or "image" or probably "vhd" (but I've never used that myself), "createOption" must be "FromImage", if you specify "managedDisk" it must be "Attach", and the docs don't describe what meaning "Empty" has. You can specify only one of those, because otherwise, you're saying to source the image from two things which would be nonsense (but is permitted by schema/docs?). "imageReference" itself is really a sum type; you must specify (offer, publisher, sku, version[, exactVersion]), or communityGalleryImageId, or sharedGalleryImageId. You could image it being, enum ImageReference { FromMarketplace { offer: String, publisher: String, sku: String, version: String, exactVersion: Option<String> }, SharedGallery(String), CommunityGallery(String), } And we've not even touched VHDs, managed disks, or VM images yet! And you don't need createOption. I think, again, I'm going off what I've learned the hard way about how Azure works. I shudder to think what the validation logic looks like. (I'm also reading the docs. Reading JSONSchema is … painful to start with, but Azure's schema's directory layout structure makes it triply painful.) But even that sum type is to miss the point of REST entirely. The RESTful definition would be: image: URI-reference and that's it. The Content-Type of the content at the provided URI provides the type of image that it is. Oh and while I'm here: don't choose a boneheaded page size if you paginate an API call. Half of Azure's services will trickle-feed you 100 records at a time, and so the response body is like 60 KiB. Since the payload also has the next page's URI, your calls get decimated by latency. Some bad offenders: listing images in a repo in ACR gets ~ a phone modems worth of overall throughput. It takes minutes to download single-digit megabytes of image metadata. The Azure pricing APIs are similar: it's ~58KiB per page. The entire VM pricing data is something like 131 MiB, and that requires 2,235 HTTP calls to fetch.
derefr 4y ago
A true REpresentational State Transfer API would give each version of each of the API's L7-protocol request and response messages its own media-(sub)type; and would then rely on HTTP content negotiation to allow clients to specify which messages they're expecting to receive in response to a request; where the difference between throwing/nonthrowing, sync/async, new/old variant, etc., would all come down to which response message types the client is declaring themselves to accept. Don't think I've literally ever seen this done in practice. Don't think I've ever seen a "REST client" library that even offers the possibility of negotiating with a backend that's attempting to do this — let alone doing it in a streamlined manner.
- recursivedoubts 4y ago
  no, the whole media type thing was a moderately-interesting-idea-turned-ridiculous-navel-gazing turn that, among many other pedantic blind alleys, turned everyone away from the deeply innovative aspect of REST: the uniform interface, and, in particular, HATEOAS https://htmx.org/essays/hateoas/ https://htmx.org/essays/hateoas/ any idiot (such as myself) who has ever made a web 1.0 app has created a better REST API than 99.9% of all REST API engineers, so called
  - lliamander 4y ago
    I have created Hypermedia APIs using HTTP/JSON for service-to-service interactions, and it is definitely a lot of work for questionable benefit. The biggest problem with the bad style of RPC-style programming was the assumption that remote calls could be treated like local calls. Modern HTTP-based RPC APIs don't make that assumption and thr problems of distributed computing are handled explicitly. The semantics of HTTP verbs, semantically meaningful namespace trees for resources in the URL, standard authentication and encoding, etc are all nice benefits of HTTP APIs even without hypermedia. That said, I haven't given up on hypermedia APIs! I liken them to "fluent" APIs of OOP. It's a powerful design approach, but I just wonder how to capture the benefits of that approach over what most people are doing today.
    - pdhborges 4y ago
vxNsr 4y ago
Hey, I’m kinda new to this area, can you show with examples what the differences are? I read the article, he does a lot of explaining but doesn’t do much showing. Like why is a specific api more RPC and why a different one is a perfectly constructed REST api?
- recursivedoubts 4y ago
  REST was the term Roy Fielding coined as description of the original web architecture. The big distinguishing feature of REST-ful systems vs. non-REST-ful systems was something called the uniform interface, where clients had zero knowledge about a given server's API surface beyond a URL entry point. Everything after that was encoded in hypermedia, giving us the term "Hypermedia As The Engine of Application State" (HATEOAS). Here is an article I wrote on it: https://htmx.org/essays/hateoas/ https://htmx.org/essays/hateoas/ RPC-style APIs require shared knowledge about a given end point: what arguments it expects, what data it returns, etc. Unfortunately, for historical reasons, we've come to call all HTTP JSON APIs "REST APIs". It's actually pretty funny.
  - jackling 4y ago
    Thanks! I was really confused with this topic, and your post cleared things up. But from what I gather, proper REST API's wouldn't pair nicely with modern day UI development. Maybe this is because I usually work with APIs that wouldn't be considered restful (from my understanding, my company's public API is here [https://developer.veevavault.com/api/22.1/ https://developer.veevavault.com/api/22.1/]), I have a hard time seeing how a modern React webapp would parse the HTTP form response given in your article.
    - recursivedoubts 4y ago
      That's right, REST concepts are largely wasted on modern SPA applications, which are fundamentally RPC-style clients. Instead, REST should be used in its original context: hypermedia clients (browsers) exchanging hypermedia (HTML) with the server. I have written an alternative front end library that takes this exact approach: https://htmx.org https://htmx.org Some more essays on the topic: https://htmx.org/essays/spa-alternative/ https://htmx.org/essays/spa-alternative/ https://htmx.org/essays/hypermedia-apis-vs-data-apis/ https://htmx.org/essays/hypermedia-apis-vs-data-apis/
Lazare 4y ago
One suggestion I've seen (eg here: https://cloud.google.com/blog/products/api-management/understanding-grpc-openapi-and-rest-and-when-to-use-them https://cloud.google.com/blog/products/api-management/unders...) is to distinguish between REST and OpenAPI designs. Because you're right, REST is a specific thing, but we do need some name for what everyone is talking about, and just saying "RPC" is much too vague.
criddell 4y ago
Why isn’t the meaning of REST allowed to drift? In the comment section of that page Roy mentions how the meaning of hypertext has drifted over time. Why shouldn’t REST evolve as well?
- wizofaus 4y ago
  It has, regardless of what Roy believes - I've never come across any API that calls itself RESTful that fits with his idea of what REST is, nor is it clear what advantages it would have.
zamalek 4y ago
REST isn't REST anymore. Barely anyone does it "correctly." The term has gained an overload: anything that happens over HTTP that OpenAPI is likely able to describe. REmote Structured Transactions. Problem solved.
aliswe 4y ago
If a teacher has one failing student, blame the student. If a teacher has 99 failing students, blame the teacher.
- yencabulator 4y ago
  Or, in this case, maybe: If the teacher has no students, blame the topic. People just don't want HATEOAS as much as some would like.

revskill 4y ago

To me, REST focus on Resource first. RPC is Producedure first.

qxmat 4y ago

I did the same a while back. I used ANTLR4 to generate a .NET tree walker that would build up an IQueryProvider expression. IQueryProvider would then compile to Expression<T> and self-optimise (simplifying boolean algebra and removing redundant expressions). We hooked this up to npgsql and viola - a sane, typesafe, query string DSL without a single line of SQL. Nowadays I hack together Terraform and Python ML- I miss .NET dearly... it was a simpler time.