3 ms·

Ah, so you're looking for something more like var myData = RestApi->get('/records', id=myDataRecordID) myData = RestApi->post('/tools/strips-whitespace

Ah, so you're looking for something more like var myData = RestApi->get('/records', id=myDataRecordID) myData = RestApi->post('/tools/strips-whitespace', myData) So, you can certainly write such HTTP endpoints to work exactly as you've described, but it doesn't really make sense to make these network calls when you can just run such a simple function in memory. So, we usually say make a call to an api, do whatever manipulating we want in code, make a call to another api, manipulate, and so on and so on. If we removed the ability to manipulate in code and forced you to make calls to '/tools' endpoints for all of that, then we'd be in the same place as UNIX pipelines, no?

ehnto 6y ago
On any old API maybe, but not a REST API. It's ignoring the core ideologies of REST, which I'll get to below the code, but despite that it's also just not the same in real world usage either: cat myData | stripWhitespace | allToUppercase | removeTheLetterC > myData Versus var myData RestAPI->get'/records', id=myDataRecordID) myData = RestAPI->post('/tools/strips-whitespace', myData) myData = RestAPI->post('/tools/allToUppercase', myData) myData = RestAPI->post('/tools/removeTheLetterC', myData) It's outside the scope of a REST API to have transformation endpoints. A REST API is supposed to allow simple access to a systems state, most often records in a database, in order to retrieve, store and update that state. REST standing for Representational State Transfer, and transformations don't fit into that at all. It's not a transfer of state from one system to another if you are giving it your data and having it manipulate it then give it back, it's asking one system to do work for you and give you the result back. Of course in the real world, you can put whatever endpoints you want into your API, and there are dozens of wildly different APIs being called RESTful, but regardless of any of that, rarely are they conducive to being used in pipes, because the point the GP was getting at is that tools are made in the UNIX ecosystem with the pipeline in mind, and that is not the case with REST endpoints. Thanks for the discussion by the way!
- jmchuster 6y ago
  Right, I don't know if it's even appropriate really to be trying to compare HTTP APIs and Unix programs like this. As you've stated, they really are meant to serve different purposes. So the original discussion of "i wish i was building with legos" I'm not sure quite sure how you would even apply it to HTTP APIs. I do think that there is an interesting point around "designed with pipelines in mind" though. I don't know if that would specifically make things more "lego-like", or if that would even specifically be a good thing. But it would be certainly interesting to analyze the list of "average library interface, or framework, or domain-specific language, or REST API" and evaluate each of them along those lines, and consider what it would look like if each was designed them to be more pipeline-centric. Or maybe the original post came more from a feeling of "it's easy for me to think in terms of pipelines, so anything that forces me to do otherwise feels needlessly complicated"? But, if we start thinking in terms of pipelines, then that's essentially what functional programming is, run a function, get this output, pass it to another function, get its output, run another function, get its output. So, maybe the love for pipelining comes out of actually a desire for a functional core. So it'd be more like "i only want to think in terms of functions at the core and not have to worry about side effects, and unix does this on a program level". So, of course, you can't entirely get away from side effects, otherwise you'd literally be doing nothing. So we push it to the edges as far as we can. The analogy for side effects for Unix then would be `> myData` as the last step. So you could say that Unix's philosophy is to have programs that form a functional core, with there being only one possible side effect, which then only has one possible placement, which is at the end of everything. Sounds like a functional dream world. Back to the list, if we wanted to design "average library interface" with this idea, then maybe you'd say here is a package with all the functional methods. And here is a package with all the side effect methods. And the design is to run a bunch of methods from the functional package before finally making a call to the side effect package. If we then loop all the way back to REST APIs (and I mean REST spec this time) then I think I could make an argument that REST does in fact follow this functional core philosophy. You have GET endpoints which are your functions, that let you retrieve resources. And then you can POST against objects, which are your side effects, which let you create or update a specific resource. The spec limits the types of side effects you can make, almost to the point of making them dead simple. So, we then have a clear delineation between what is a function and what is a side effect, with a very limited and clear number of side effects that you're able to create.

ehnto 6y ago

On any old API maybe, but not a REST API. It's ignoring the core ideologies of REST, which I'll get to below the code, but despite that it's also just not the same in real world usage either: cat myData | stripWhitespace | allToUppercase | removeTheLetterC > myData Versus var myData RestAPI->get'/records', id=myDataRecordID) myData = RestAPI->post('/tools/strips-whitespace', myData) myData = RestAPI->post('/tools/allToUppercase', myData) myData = RestAPI->post('/tools/removeTheLetterC', myData) It's outside the scope of a REST API to have transformation endpoints. A REST API is supposed to allow simple access to a systems state, most often records in a database, in order to retrieve, store and update that state. REST standing for Representational State Transfer, and transformations don't fit into that at all. It's not a transfer of state from one system to another if you are giving it your data and having it manipulate it then give it back, it's asking one system to do work for you and give you the result back. Of course in the real world, you can put whatever endpoints you want into your API, and there are dozens of wildly different APIs being called RESTful, but regardless of any of that, rarely are they conducive to being used in pipes, because the point the GP was getting at is that tools are made in the UNIX ecosystem with the pipeline in mind, and that is not the case with REST endpoints. Thanks for the discussion by the way!

jmchuster 6y ago
Right, I don't know if it's even appropriate really to be trying to compare HTTP APIs and Unix programs like this. As you've stated, they really are meant to serve different purposes. So the original discussion of "i wish i was building with legos" I'm not sure quite sure how you would even apply it to HTTP APIs. I do think that there is an interesting point around "designed with pipelines in mind" though. I don't know if that would specifically make things more "lego-like", or if that would even specifically be a good thing. But it would be certainly interesting to analyze the list of "average library interface, or framework, or domain-specific language, or REST API" and evaluate each of them along those lines, and consider what it would look like if each was designed them to be more pipeline-centric. Or maybe the original post came more from a feeling of "it's easy for me to think in terms of pipelines, so anything that forces me to do otherwise feels needlessly complicated"? But, if we start thinking in terms of pipelines, then that's essentially what functional programming is, run a function, get this output, pass it to another function, get its output, run another function, get its output. So, maybe the love for pipelining comes out of actually a desire for a functional core. So it'd be more like "i only want to think in terms of functions at the core and not have to worry about side effects, and unix does this on a program level". So, of course, you can't entirely get away from side effects, otherwise you'd literally be doing nothing. So we push it to the edges as far as we can. The analogy for side effects for Unix then would be `> myData` as the last step. So you could say that Unix's philosophy is to have programs that form a functional core, with there being only one possible side effect, which then only has one possible placement, which is at the end of everything. Sounds like a functional dream world. Back to the list, if we wanted to design "average library interface" with this idea, then maybe you'd say here is a package with all the functional methods. And here is a package with all the side effect methods. And the design is to run a bunch of methods from the functional package before finally making a call to the side effect package. If we then loop all the way back to REST APIs (and I mean REST spec this time) then I think I could make an argument that REST does in fact follow this functional core philosophy. You have GET endpoints which are your functions, that let you retrieve resources. And then you can POST against objects, which are your side effects, which let you create or update a specific resource. The spec limits the types of side effects you can make, almost to the point of making them dead simple. So, we then have a clear delineation between what is a function and what is a side effect, with a very limited and clear number of side effects that you're able to create.