11 ms·

Distributed Systems and the End of the API

209 points by mcms 12y ago

richm44 12y ago
I stopped at the point where he claimed that APIs were always synchronous, this wasn't even true in the 80s. For example XLib is a rather well used API and is asynchronous (there are many others).
- koide 12y ago
  two paragraphs later he addresses that point, calling that support limited in current api designs
  - richm44 12y ago
    Not really, he talks about HTTP which wasn't really designed for that purpose. There are plenty of protocols that were. Does this actually have anything to add that isn't covered in http://en.wikipedia.org/wiki/Fallacies_of_Distributed_Computing http://en.wikipedia.org/wiki/Fallacies_of_Distributed_Comput... , if so then I'll read further.
    - pjscott 12y ago
      It has some very interesting stuff about CRDTs, which is definitely worth a look.
- koide 12y ago
  two paragraphs later he addresses that point, calling that support limited in current api designs
- koide 12y ago
  two paragraphs later he addresses that point, calling that support limited in current api designs
- dllthomas 12y ago
  My recollection was that Xlib is an unfortunately synchronous library written for the the wonderfully asynchronous X protocol.
  - bitwize 12y ago
    My understanding was that the Xlib protocol's asynchrony was hardly wonderful, and made syncing with vblank and pixel-perfect frames difficult, which motivates its abandonment in favor of the synchronous, local-host-only Wayland protocol.
    - dllthomas 12y ago
      Certainly, the X protocol's asynchronous nature isn't without some downsides (though I think you could address the vblank sync without discarding it). However, I maintain that Xlib itself was a synchronous interface in front of an asynchronous protocol - which gives us the worst of both worlds and motivated Xcb.
baldeagle 12y ago
TL;DR: APIs have issues with concurrency and latency, amongst others. Use Consistency As Logical Monotonicity (CALM) or Conflict-free Replicated Data Types (CRDTs) instead. Here is a little about how CRDTs work. btw: speech in NY on the 15th of May.
- lstamour 12y ago
  Thanks for the summary. Via Google, ended up at http://www.slideshare.net/jboner/the-road-to-akka-cluster-and-beyond http://www.slideshare.net/jboner/the-road-to-akka-cluster-an... which I've bookmarked to watch at a future date, since I'm new to these concepts.
cmeiklejohn 12y ago
I also covered a variety of similar issues when discussing that offline rich-web applications are perfect for CRDTs, because you are effectively building a distributed system, in my EmberConf 2014 talk [1][2] called Convergent/Divergent. [1] http://confreaks.com/videos/3311-emberconf2014-convergent-divergent http://confreaks.com/videos/3311-emberconf2014-convergent-di... [2] https://speakerdeck.com/cmeiklejohn/divergent https://speakerdeck.com/cmeiklejohn/divergent * edited to reformat list.
- cmeiklejohn 12y ago
  Also, very relevant: "A Note on Distributed Computing" 1994, Sun Microsystems Technical Report * http://lambda-the-ultimate.org/node/1450 http://lambda-the-ultimate.org/node/1450 * http://dl.acm.org/citation.cfm?id=974938 http://dl.acm.org/citation.cfm?id=974938
iadapter 12y ago
Of course APIs that serve as synchronous endpoints to distributed systems are a leaky abstraction. But its not the only one of its kind, there's also Guaranteed Message Delivery [1]. I find the philosophy behind Akka in this context a better fit - embrace that networks are unreliable and build your app around this limitation accordingly [2]. The cost is that it results in more work for the developer just like with the usage of CRDTs. [1] http://www.infoq.com/articles/no-reliable-messaging http://www.infoq.com/articles/no-reliable-messaging [2] http://doc.akka.io/docs/akka/2.1.0/general/message-delivery-guarantees.html http://doc.akka.io/docs/akka/2.1.0/general/message-delivery-...
- deleted 12y ago
  [deleted]
josephschmoe 12y ago
Many API users are not knowledgeable to the intricacies of network programming. This could definitely use an executive summary at the top with the following info: "What will these new libraries that replace APIs offer to single point API users?"
- cemerick 12y ago
  (Author here.) As a start, hopefully removing the blinders that makes statements like "many API users are not knowledgeable to the intricacies of network programming" oh so true. That that statement can be made without popular incredulity only reinforces the point that modern network API technologies have largely been built to sustain the illusion that there is no network, and you're just making a method call somewhere. Insert this-plt-life GIF here. :-P Building systems with things like CRDTs and tools and languages that support CALM will allow people using point-to-point APIs to continue to do the things they do now, but remove much of the incidental complexity from the equation. An example would be that, when you are relying upon N replication mechanisms to move CRDT state or operations from _here_ to _there_, you don't need complex timeout, retry, and backoff mechanisms to compensate for the realities of what's connecting the two parties. The message will arrive when it arrives...exactly the only guarantee that you can make in the general case of someone talking to external services.
jwingy 12y ago
CALM and CRDTs are interesting stuff. That being said, I feel like the author is confusing a bit the specific implementations of modern APIs vs the concept of an API which I see as simply some (somewhat standardized) interface to a system which you don't own. Those seem like two different problem domains to me, but perhaps I'm arguing over a different definition of APIs than from what the author is talking about....
- mantrax5 12y ago
  I think you got it exactly right. It feels like the author got a little too excited about CRDT-s and forgot all the other principles of good system design (it's about clear stable interface, low coupling, single responsibility and so on).
- cemerick 12y ago
  Hi, author here. (Somewhat standardized) interfaces are _fine_. My contention is that you can have an interface shared by disparate actors without the problematical bits of "APIs" (both in spirit and in their particular current best materializations), which provide no useful data model constraints, do not acknowledge the realities of the network, and inherently couple client and server. The point is that you can have a shared "interface" over _data_, in exactly the same way as producers and consumers share shapes/types of messages routed via queues — except that there are ways (CRDTs being one) to extend that dynamic so that data can be replicated along any topology, and shared and reacted to by N actors, not just a consumer downstream of your producer. I hope that clarifies. :-)
ddp 12y ago
I believe it was Leslie Lamport who said, "A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable."
- cmeiklejohn 12y ago
  The reference is here: http://research.microsoft.com/en-us/um/people/lamport/pubs/distributed-system.txt http://research.microsoft.com/en-us/um/people/lamport/pubs/d...
  - cemerick 12y ago
    Dammit, thank you. I've even read that exact message before. :-/ Post updated!
  - deleted 12y ago
    [deleted]
programminggeek 12y ago
At least the author is wise enough to see that REST/RPC are not so different from each other. I actually find it interesting that as I was learning about earlier networked "objects" type systems, programmers ran into problems where they were treating the networked objects as if they were local and that the network always works. Now, when we build REST api's they always ship with client libraries that feel like local objects and completely abstract away most notions of network failure, etc. I'm not saying we've made an unreasonable tradeoff, it's just interesting that we seem to be making more refined versions of the same solutions with the same fundamental problems. I guess the author was making a similar point.
- deleted 12y ago
  [deleted]
- steveklabnik 12y ago
  "Layman's REST" is very much RPC, yes. Fielding's REST is very much not.
  - mantrax5 12y ago
    Fielding's REST is pretty much CRUD in HTTP disguise. Don't get me wrong, this can be great for "hypermedia applications" as Fielding's paper argues. But "hypermedia applications" just doesn't fit what many distributed services do these days. Services are naturally centered arounds verbs (commands and queries) and not nouns (resources), so like with any other CRUD system, at some point a REST API that shoehorns everything into the four standard verbs HTTP commonly gives us, no longer adequately describes the business requirements of your app. You can definitely force things to be RESTful, but it's typically not the natural way to build an API. Feels akin to the ORM kind of impedance mismatch in some ways.
    - icedchai 12y ago
      Exactly this. I've seen people doing crazy, non-nonsensical stuff to make their APIs "RESTful". So RESTful that they make no sense.
mantrax5 12y ago
So he favors exposing a standard set of distributed data models instead of having APIs. What a horrible idea. Exposing implementations is bad because implementations change. Exposing implementations is bad because as you expose the intricacies of your data model to your client (which he claims is a benefit) you in turn obscure and hide the intricacies of your business domain, which will surely not allow you to patch a service's distributed data tree in an arbitrary fashion. It's in essence like having SQL as your underlying data model, and replacing your API with an open read/write/delete access to your SQL server to the entire world, and hoping everyone will run the right queries and all will be all right. It won't be all right. APIs will become more asynchronous and eventually all APIs will be seen as protocols, that don't necessarily follow a simple request/response pattern. But they'll remain in the form of abstract commands and queries modeled after the business domain of the application, and not the underlying data model of it.
- derefr 12y ago
  > It's in essence like having SQL as your underlying data model, and replacing your API with an open read/write/delete access to your SQL server to the entire world, and hoping everyone will run the right queries and all will be all right. I find it kind of amusing that this was the original purpose of having an "SQL server": letting people (e.g. auditors) submit arbitrary queries, so you won't have to anticipate what exactly they'll want to do with your data. (Write-access was intended to be segregated to particular database users writing to particular tables, though--basically parallel to using WebDAV with HTTP Basic Auth.)
  - mantrax5 12y ago
    It was, yes, and to this day read-only SQL access to certain tables is not that bad of a practice to allow for report-generating apps within a company. However the idea of exposing SQL databases publicly as an approach never took hold for many reasons we're today aware of. And the idea of public write access is ridiculous right from its premise. The anti-API rant of this author shows us that those who don't know their history are doomed to repeat it.
- Shorel 12y ago
kylebrown 12y ago
Distributed API's are a big part of Ethereum. I think the Merkle tree of the bitcoin blockchain (and the Patricia tree of the Ethereum blockchain) might even qualify as a semilattice. In fact, its by the physics of information theory that a cryptographic blockchain solves the consensus problem. Specifically, information theory emerges from the laws of thermodynamics: Maxwell's demon is essentially what secures one's private keys from brute-force cracking attempts. I'd like to see a comparison of how the blockchain solves the CAP problem, alongside CRDT's. Are they not both solutions to the same problem?
- marktangotango 12y ago
  FYI information theory entropy and physics entropy really aren't the same thing: http://physics.ucsd.edu/do-the-math/2013/05/elusive-entropy/ http://physics.ucsd.edu/do-the-math/2013/05/elusive-entropy/
  - kylebrown 12y ago
    Well that seemed excessively pedantic IMHO. It actually didn't touch much on information theory, and where it did, many of the comments disagree. I'll cite the Landauer limit[1] as what (yes, arguably) connects the entropy of information theory to the entropy of physics.[2] Also, I only mentioned physics because the article did, quoting Lamport "Most people view concurrency as a programming problem or a language problem. I regard it as a physics problem." Unfortunately the article didn't elaborate any more on the precise type of physics problem in question (Maybe Lamport does elsewhere), whether the physics of computational complexity or the physics of information theory, or something else. But even those two sub-fields have many connections and similarities (as does pretty much everything in physics and math. such connections are the bread-and-butter of theoreticians). 1. http://en.wikipedia.org/wiki/Von_Neumann-Landauer_limit http://en.wikipedia.org/wiki/Von_Neumann-Landauer_limit 2. http://en.wikipedia.org/wiki/Entropy_in_thermodynamics_and_information_theory http://en.wikipedia.org/wiki/Entropy_in_thermodynamics_and_i...
    - kylebrown 12y ago
      Here's an article which discusses Lamport's view: "The physics of distributed information systems"[1]. The first sentence: "This paper aims to present distributed systems as a new (interesting) area of applications of statistical physics, and to make the the case that statistical physics can be quite useful in understanding such systems." It has several mentions of statistical physics, but (curiously) no mentions of entropy. It does however discuss the Byzantine Generals problem, which of course is the problem the bitcoin blockchain solves. 1. http://iopscience.iop.org/1742-6596/473/1/012017/pdf/1742-6596_473_1_012017.pdf http://iopscience.iop.org/1742-6596/473/1/012017/pdf/1742-65...
logn 12y ago
Rest is less platform dependent than SOAP/RPC. I see that as the main benefit. JSON is easier to work with than XML. The whole idea of service oriented architectures is that users don't need to care about the tech stack details of your service. Rest and JSON do a better job of realizing that vision than SOAP/XML. I don't think anyone's claimed that Rest is a design pattern to end all woes. Maybe we haven't given due thought to what new design patterns (or data structures, architectures, etc) are emerging these days, and in that light, the article presents a lot of interesting pointers.
- virmundi 12y ago
  Actually JSON makes it damn near impossible to uniformally implement REST. You need HATEOS. This means that there has to be a semantic of following links in resources. JSON lacks this ability. ATOM or RSS, both XML, have linking. Heck, XML at a language level supports document linking. HTTP + JSON != REST.
  - logn 12y ago
    Interesting. I looked into HATEOAS more and found this link to a PayPal API: https://developer.paypal.com/docs/integration/direct/paypal-rest-payment-hateoas-links/ https://developer.paypal.com/docs/integration/direct/paypal-... I think I'll try to follow Rest better. Anyhow, I still prefer JSON. To me it's a more concise way to explain data, with nice syntax for arrays and maps. I'd rather jump through a few hoops to implement true Rest than try to craft verbose XML schemas for simple things like arrays and maps. ATOM and RSS I think are good for their main use cases but aren't as generic of syntax as JSON.
  - jalfresi 12y ago
    Whilst I agree with you, there is nothing stopping someone defining link structures in JSON documents, coining a new media type e.g. JSON+Link and boom, problem solved.
    - virmundi 12y ago
      Well, I'm responding to you and to your siblings. You're right. You can create a new MIME type and the problem is solved. Fortunately, as the sibling comments pointed out, there is an extension. What I want to see happen is that the JSON + Links becomes a standard. Ideally a W3C. Otherwise we're into the old XKCD comic https://xkcd.com/927/ https://xkcd.com/927/
rooted 12y ago
I think a distributed system is better defined as a system where timing becomes an issue to the coordination of components in the system.
dreamfactory2 12y ago
Hmm the article seems based on some false assumptions. I'd argue that the whole point of REST as an architectural style is to be stateless and async. Of course you would use an ESB of some kind rather than point-to-point if you want to protect yourself from failure of a solution component - REST lends itself well to that or to building error-handling in the client. And isn't 'turning operations into data' what we are doing by switching from a verb-based model to a noun-based one?
- cemerick 12y ago
  Hi, author here. REST has its set of semantics, but (a) I don't think they're particularly useful for building computational services with, and (b) it's for all practical purposes predicated on HTTP, which carries a lot of baggage. Each _request_ is stateless (barring things like sessions^H^H^H^H^H hack workarounds), but clients and servers certainly are not; and, how one maintains that state and orchestrates further REST interactions based on an intermediate response is entirely on the implementer, _every single time_ a service or client is built/used. I'd personally much prefer communication and computational primitives that can just as easily be used for a point-to-point interaction as they can be used to _build_ an ESB (enterprise service bus, I believe you mean?) if that's what I want. I don't think nouns vs. verbs are a useful distinction. Turning operations into data is a first step, but all data is not equivalent. Some representations lend themselves to composition such that you can represent essentially arbitrary structures (sets, graphs, trees, multimaps, etc), but most (including the common ones of JSON and XML) do not. Likewise, some data representations allow for commutative operations so as to reconcile concurrent actors' activity, but most (again including JSON and XML) do not.
ChuckMcM 12y ago
Fun stuff, amusing that the definition of a distributed system used; "Where a computer that you never heard of can bring your system down." is actually one of Leslie Lamport's more famous quotes. When I joined Sun in '86 I thought it was the pinnacle of technological excellence to be a kernel programmer, and I joined the Systems Group, the notional center of the Sun universe, in 1987. However I discovered that the primary reason you had to be picky about kernel programmers what that their bogus pointer references crashed the machine (as they occurred in kernel mode with full privileges) but discovered that network programmers could crash the whole world with their bugs. So clearly they must be in a pantheon above kernel programmers. :-) The author has come to discover that in the network world things can die anywhere, and this makes reasoning about such systems very complicated. Having been a part of the RPC and CORBA evolution I keenly felt the challenges of making APIs that "looked" like function calls to a programmer but took place across a network fabric and thus introduced error conditions that couldn't exist in locally called routines. (like the inability to return from the function due to a network partition for a simple example). Lamport's work in this space is brilliant and inspired. Network systems can be analysed and reasoned about as physical systems when they exhibit discontinuities when considered as simple algorithms. The value here is to realize that a large number of physical systems tolerate a tremendous amount of randomness and continue to work as intended (windmills for example) while many algorithms only work consistently given a set of key invariants. I gave a talk that was inspired by Dr. Lamports work titled 'Java as Newtonian Physics' which was a call to action to create a set of invariants, in the spirit of physical laws, that would govern the behavior and capabilities of distributed systems. It was way early for its time (AOL dialup connections were still a thing) but much of the same inspiration (presumably from Lamport) made it into the Google Spanner project. As with many things, at a surface level many people learn an API which does something under the covers across the network but having come up through their education thinking of everything as an API they don't fundamentally grasp the notion of distributed computation. Then at some point in their experience there will be that 'ah ha' moment when suddenly everything they know is wrong, which really means they suddenly see a bigger picture of things. It makes distributed systems questions in interviews an excellent litmus test for understanding where people are in their journey.
- jacquesm 12y ago
dgreensp 12y ago
As the author of EtherPad I'm familiar with CRDT, which is a cousin of OT. They don't really replace APIs, unless you are using an API to synchronize data, which is only one of many things you might be trying to do. In other words, if you're building EtherPad or Wave, use a fancy data structure for the collaborative document. Otherwise, don't. Meteor's DDP provides a nice model, where the results of RPCs stream in asynchronously.
- cemerick 12y ago
  Hi, author here. I'm not sure you read the whole piece. :-) (Modern) APIs are a very limited mechanism of state transfer that happens to be paired with often side-effecting operations. Thus, a "synchronization" (I don't think that word is particularly useful because reasons) mechanism paired with reactive computational services _does_ replace APIs, and offers the ability to do much, much more. OTs (operational transforms) _are_ a related precursor to CRDTs only in that they are both ways to reconcile concurrent changes, but that's really the limit of the connection. Unfortunately, the substrate for OTs (text, integer-indexed sequences of characters) is fundamentally not amenable to commutative operations. This makes implementing OTs _very_ difficult and error-prone, and certain combinations of concurrent operations are completely unreconcilable (a result that came out of a Korean group's study, can't find the cite for it right now).
  - jorangreef 12y ago
    I think the paper you are referencing might be [1]? It's one of my favorite papers on CRDTs and provides practical pseudocode for learning how to implement CRDTs yourself. The structures they present are simple to understand and have good performance characteristics compared to similar CRDTs [2]. A key insight from the second paper is to write CRDTs that optimize for applying remote operations over applying local operations, as the ratio of remote operations to local operations will be greater. i.e. 100 clients making 1 change to a CRDT will require all 100 clients to each apply 99 remote operations and 1 local operation. [1] Replicated abstract data types: Building blocks for collaborative applications - http://dl.acm.org/citation.cfm?id=1931272 http://dl.acm.org/citation.cfm?id=1931272 [2] Evaluating CRDTs for Real-time Document Editing - http://hal.archives-ouvertes.fr/docs/00/62/95/03/PDF/doce63-ahmednacer.pdf http://hal.archives-ouvertes.fr/docs/00/62/95/03/PDF/doce63-...
ryanobjc 12y ago
CRDTs are absolutely fascinating, but sometimes I really wonder. It seems like you throw words like 'semi-lattice' around ... Also there is one particular element to the eventual consistency that bothers me, it's that all these eventually consistent algorithms aren't how high powered neural nets will work. Our brain is highly eventually consistent, but it computes without ever needing these algorithms.
- bm1362 12y ago
  I think you're taking the neural net _model_ of the brain too strictly.
anuraj 12y ago
This is a network programmers view. System, Network programmers concern themselves about systems and topologies. For an application programmer, both needs to be abstracted and only the business logic is important. Money is in the top of the pyramid now - hence the proliferation of APIs.
- cemerick 12y ago
  The point of the piece, in large part, was to emphasize that we're all network programmers. If you're whacking away with APIs and couldn't care less about the broader system its topology, please stay away from me. ;-) APIs are proliferating because their coupling around client/server process and data representation makes for high switching costs and thus sweet vendor lock-in.
sagargv 12y ago
Joel Spolsky had written along similar lines and argued that it's important to know what is happening beneath abstractions. http://www.joelonsoftware.com/articles/LeakyAbstractions.html http://www.joelonsoftware.com/articles/LeakyAbstractions.htm...

209 points by mcms 12y ago

I stopped at the point where he claimed that APIs were always synchronous, this wasn't even true in the 80s. For example XLib is a rather well used API and is asynchronous (there are many others).

koide 12y ago
two paragraphs later he addresses that point, calling that support limited in current api designs
- richm44 12y ago
  Not really, he talks about HTTP which wasn't really designed for that purpose. There are plenty of protocols that were. Does this actually have anything to add that isn't covered in http://en.wikipedia.org/wiki/Fallacies_of_Distributed_Computing http://en.wikipedia.org/wiki/Fallacies_of_Distributed_Comput... , if so then I'll read further.
  - pjscott 12y ago
    It has some very interesting stuff about CRDTs, which is definitely worth a look.
koide 12y ago
two paragraphs later he addresses that point, calling that support limited in current api designs
koide 12y ago
two paragraphs later he addresses that point, calling that support limited in current api designs
dllthomas 12y ago
My recollection was that Xlib is an unfortunately synchronous library written for the the wonderfully asynchronous X protocol.
- bitwize 12y ago
  My understanding was that the Xlib protocol's asynchrony was hardly wonderful, and made syncing with vblank and pixel-perfect frames difficult, which motivates its abandonment in favor of the synchronous, local-host-only Wayland protocol.
  - dllthomas 12y ago
    Certainly, the X protocol's asynchronous nature isn't without some downsides (though I think you could address the vblank sync without discarding it). However, I maintain that Xlib itself was a synchronous interface in front of an asynchronous protocol - which gives us the worst of both worlds and motivated Xcb.

baldeagle 12y ago

TL;DR: APIs have issues with concurrency and latency, amongst others. Use Consistency As Logical Monotonicity (CALM) or Conflict-free Replicated Data Types (CRDTs) instead. Here is a little about how CRDTs work. btw: speech in NY on the 15th of May.

lstamour 12y ago
Thanks for the summary. Via Google, ended up at http://www.slideshare.net/jboner/the-road-to-akka-cluster-and-beyond http://www.slideshare.net/jboner/the-road-to-akka-cluster-an... which I've bookmarked to watch at a future date, since I'm new to these concepts.

cmeiklejohn 12y ago

I also covered a variety of similar issues when discussing that offline rich-web applications are perfect for CRDTs, because you are effectively building a distributed system, in my EmberConf 2014 talk [1][2] called Convergent/Divergent. [1] http://confreaks.com/videos/3311-emberconf2014-convergent-divergent http://confreaks.com/videos/3311-emberconf2014-convergent-di... [2] https://speakerdeck.com/cmeiklejohn/divergent https://speakerdeck.com/cmeiklejohn/divergent * edited to reformat list.

cmeiklejohn 12y ago
Also, very relevant: "A Note on Distributed Computing" 1994, Sun Microsystems Technical Report * http://lambda-the-ultimate.org/node/1450 http://lambda-the-ultimate.org/node/1450 * http://dl.acm.org/citation.cfm?id=974938 http://dl.acm.org/citation.cfm?id=974938

iadapter 12y ago

Of course APIs that serve as synchronous endpoints to distributed systems are a leaky abstraction. But its not the only one of its kind, there's also Guaranteed Message Delivery [1]. I find the philosophy behind Akka in this context a better fit - embrace that networks are unreliable and build your app around this limitation accordingly [2]. The cost is that it results in more work for the developer just like with the usage of CRDTs. [1] http://www.infoq.com/articles/no-reliable-messaging http://www.infoq.com/articles/no-reliable-messaging [2] http://doc.akka.io/docs/akka/2.1.0/general/message-delivery-guarantees.html http://doc.akka.io/docs/akka/2.1.0/general/message-delivery-...

deleted 12y ago
[deleted]

josephschmoe 12y ago

Many API users are not knowledgeable to the intricacies of network programming. This could definitely use an executive summary at the top with the following info: "What will these new libraries that replace APIs offer to single point API users?"

cemerick 12y ago
(Author here.) As a start, hopefully removing the blinders that makes statements like "many API users are not knowledgeable to the intricacies of network programming" oh so true. That that statement can be made without popular incredulity only reinforces the point that modern network API technologies have largely been built to sustain the illusion that there is no network, and you're just making a method call somewhere. Insert this-plt-life GIF here. :-P Building systems with things like CRDTs and tools and languages that support CALM will allow people using point-to-point APIs to continue to do the things they do now, but remove much of the incidental complexity from the equation. An example would be that, when you are relying upon N replication mechanisms to move CRDT state or operations from _here_ to _there_, you don't need complex timeout, retry, and backoff mechanisms to compensate for the realities of what's connecting the two parties. The message will arrive when it arrives...exactly the only guarantee that you can make in the general case of someone talking to external services.

jwingy 12y ago

CALM and CRDTs are interesting stuff. That being said, I feel like the author is confusing a bit the specific implementations of modern APIs vs the concept of an API which I see as simply some (somewhat standardized) interface to a system which you don't own. Those seem like two different problem domains to me, but perhaps I'm arguing over a different definition of APIs than from what the author is talking about....

mantrax5 12y ago
I think you got it exactly right. It feels like the author got a little too excited about CRDT-s and forgot all the other principles of good system design (it's about clear stable interface, low coupling, single responsibility and so on).
cemerick 12y ago
Hi, author here. (Somewhat standardized) interfaces are _fine_. My contention is that you can have an interface shared by disparate actors without the problematical bits of "APIs" (both in spirit and in their particular current best materializations), which provide no useful data model constraints, do not acknowledge the realities of the network, and inherently couple client and server. The point is that you can have a shared "interface" over _data_, in exactly the same way as producers and consumers share shapes/types of messages routed via queues — except that there are ways (CRDTs being one) to extend that dynamic so that data can be replicated along any topology, and shared and reacted to by N actors, not just a consumer downstream of your producer. I hope that clarifies. :-)

ddp 12y ago

I believe it was Leslie Lamport who said, "A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable."

cmeiklejohn 12y ago
The reference is here: http://research.microsoft.com/en-us/um/people/lamport/pubs/distributed-system.txt http://research.microsoft.com/en-us/um/people/lamport/pubs/d...
- cemerick 12y ago
  Dammit, thank you. I've even read that exact message before. :-/ Post updated!
- deleted 12y ago
  [deleted]

programminggeek 12y ago

At least the author is wise enough to see that REST/RPC are not so different from each other. I actually find it interesting that as I was learning about earlier networked "objects" type systems, programmers ran into problems where they were treating the networked objects as if they were local and that the network always works. Now, when we build REST api's they always ship with client libraries that feel like local objects and completely abstract away most notions of network failure, etc. I'm not saying we've made an unreasonable tradeoff, it's just interesting that we seem to be making more refined versions of the same solutions with the same fundamental problems. I guess the author was making a similar point.

deleted 12y ago
[deleted]
steveklabnik 12y ago
"Layman's REST" is very much RPC, yes. Fielding's REST is very much not.
- mantrax5 12y ago
  Fielding's REST is pretty much CRUD in HTTP disguise. Don't get me wrong, this can be great for "hypermedia applications" as Fielding's paper argues. But "hypermedia applications" just doesn't fit what many distributed services do these days. Services are naturally centered arounds verbs (commands and queries) and not nouns (resources), so like with any other CRUD system, at some point a REST API that shoehorns everything into the four standard verbs HTTP commonly gives us, no longer adequately describes the business requirements of your app. You can definitely force things to be RESTful, but it's typically not the natural way to build an API. Feels akin to the ORM kind of impedance mismatch in some ways.
  - icedchai 12y ago
    Exactly this. I've seen people doing crazy, non-nonsensical stuff to make their APIs "RESTful". So RESTful that they make no sense.

mantrax5 12y ago

So he favors exposing a standard set of distributed data models instead of having APIs. What a horrible idea. Exposing implementations is bad because implementations change. Exposing implementations is bad because as you expose the intricacies of your data model to your client (which he claims is a benefit) you in turn obscure and hide the intricacies of your business domain, which will surely not allow you to patch a service's distributed data tree in an arbitrary fashion. It's in essence like having SQL as your underlying data model, and replacing your API with an open read/write/delete access to your SQL server to the entire world, and hoping everyone will run the right queries and all will be all right. It won't be all right. APIs will become more asynchronous and eventually all APIs will be seen as protocols, that don't necessarily follow a simple request/response pattern. But they'll remain in the form of abstract commands and queries modeled after the business domain of the application, and not the underlying data model of it.

derefr 12y ago
> It's in essence like having SQL as your underlying data model, and replacing your API with an open read/write/delete access to your SQL server to the entire world, and hoping everyone will run the right queries and all will be all right. I find it kind of amusing that this was the original purpose of having an "SQL server": letting people (e.g. auditors) submit arbitrary queries, so you won't have to anticipate what exactly they'll want to do with your data. (Write-access was intended to be segregated to particular database users writing to particular tables, though--basically parallel to using WebDAV with HTTP Basic Auth.)
- mantrax5 12y ago
  It was, yes, and to this day read-only SQL access to certain tables is not that bad of a practice to allow for report-generating apps within a company. However the idea of exposing SQL databases publicly as an approach never took hold for many reasons we're today aware of. And the idea of public write access is ridiculous right from its premise. The anti-API rant of this author shows us that those who don't know their history are doomed to repeat it.
Shorel 12y ago

kylebrown 12y ago

Distributed API's are a big part of Ethereum. I think the Merkle tree of the bitcoin blockchain (and the Patricia tree of the Ethereum blockchain) might even qualify as a semilattice. In fact, its by the physics of information theory that a cryptographic blockchain solves the consensus problem. Specifically, information theory emerges from the laws of thermodynamics: Maxwell's demon is essentially what secures one's private keys from brute-force cracking attempts. I'd like to see a comparison of how the blockchain solves the CAP problem, alongside CRDT's. Are they not both solutions to the same problem?

marktangotango 12y ago
FYI information theory entropy and physics entropy really aren't the same thing: http://physics.ucsd.edu/do-the-math/2013/05/elusive-entropy/ http://physics.ucsd.edu/do-the-math/2013/05/elusive-entropy/
- kylebrown 12y ago
  Well that seemed excessively pedantic IMHO. It actually didn't touch much on information theory, and where it did, many of the comments disagree. I'll cite the Landauer limit[1] as what (yes, arguably) connects the entropy of information theory to the entropy of physics.[2] Also, I only mentioned physics because the article did, quoting Lamport "Most people view concurrency as a programming problem or a language problem. I regard it as a physics problem." Unfortunately the article didn't elaborate any more on the precise type of physics problem in question (Maybe Lamport does elsewhere), whether the physics of computational complexity or the physics of information theory, or something else. But even those two sub-fields have many connections and similarities (as does pretty much everything in physics and math. such connections are the bread-and-butter of theoreticians). 1. http://en.wikipedia.org/wiki/Von_Neumann-Landauer_limit http://en.wikipedia.org/wiki/Von_Neumann-Landauer_limit 2. http://en.wikipedia.org/wiki/Entropy_in_thermodynamics_and_information_theory http://en.wikipedia.org/wiki/Entropy_in_thermodynamics_and_i...
  - kylebrown 12y ago
    Here's an article which discusses Lamport's view: "The physics of distributed information systems"[1]. The first sentence: "This paper aims to present distributed systems as a new (interesting) area of applications of statistical physics, and to make the the case that statistical physics can be quite useful in understanding such systems." It has several mentions of statistical physics, but (curiously) no mentions of entropy. It does however discuss the Byzantine Generals problem, which of course is the problem the bitcoin blockchain solves. 1. http://iopscience.iop.org/1742-6596/473/1/012017/pdf/1742-6596_473_1_012017.pdf http://iopscience.iop.org/1742-6596/473/1/012017/pdf/1742-65...

logn 12y ago

Rest is less platform dependent than SOAP/RPC. I see that as the main benefit. JSON is easier to work with than XML. The whole idea of service oriented architectures is that users don't need to care about the tech stack details of your service. Rest and JSON do a better job of realizing that vision than SOAP/XML. I don't think anyone's claimed that Rest is a design pattern to end all woes. Maybe we haven't given due thought to what new design patterns (or data structures, architectures, etc) are emerging these days, and in that light, the article presents a lot of interesting pointers.

virmundi 12y ago
Actually JSON makes it damn near impossible to uniformally implement REST. You need HATEOS. This means that there has to be a semantic of following links in resources. JSON lacks this ability. ATOM or RSS, both XML, have linking. Heck, XML at a language level supports document linking. HTTP + JSON != REST.
- logn 12y ago
  Interesting. I looked into HATEOAS more and found this link to a PayPal API: https://developer.paypal.com/docs/integration/direct/paypal-rest-payment-hateoas-links/ https://developer.paypal.com/docs/integration/direct/paypal-... I think I'll try to follow Rest better. Anyhow, I still prefer JSON. To me it's a more concise way to explain data, with nice syntax for arrays and maps. I'd rather jump through a few hoops to implement true Rest than try to craft verbose XML schemas for simple things like arrays and maps. ATOM and RSS I think are good for their main use cases but aren't as generic of syntax as JSON.
- jalfresi 12y ago
  Whilst I agree with you, there is nothing stopping someone defining link structures in JSON documents, coining a new media type e.g. JSON+Link and boom, problem solved.
  - virmundi 12y ago
    Well, I'm responding to you and to your siblings. You're right. You can create a new MIME type and the problem is solved. Fortunately, as the sibling comments pointed out, there is an extension. What I want to see happen is that the JSON + Links becomes a standard. Ideally a W3C. Otherwise we're into the old XKCD comic https://xkcd.com/927/ https://xkcd.com/927/

rooted 12y ago

I think a distributed system is better defined as a system where timing becomes an issue to the coordination of components in the system.

dreamfactory2 12y ago

Hmm the article seems based on some false assumptions. I'd argue that the whole point of REST as an architectural style is to be stateless and async. Of course you would use an ESB of some kind rather than point-to-point if you want to protect yourself from failure of a solution component - REST lends itself well to that or to building error-handling in the client. And isn't 'turning operations into data' what we are doing by switching from a verb-based model to a noun-based one?

cemerick 12y ago
Hi, author here. REST has its set of semantics, but (a) I don't think they're particularly useful for building computational services with, and (b) it's for all practical purposes predicated on HTTP, which carries a lot of baggage. Each _request_ is stateless (barring things like sessions^H^H^H^H^H hack workarounds), but clients and servers certainly are not; and, how one maintains that state and orchestrates further REST interactions based on an intermediate response is entirely on the implementer, _every single time_ a service or client is built/used. I'd personally much prefer communication and computational primitives that can just as easily be used for a point-to-point interaction as they can be used to _build_ an ESB (enterprise service bus, I believe you mean?) if that's what I want. I don't think nouns vs. verbs are a useful distinction. Turning operations into data is a first step, but all data is not equivalent. Some representations lend themselves to composition such that you can represent essentially arbitrary structures (sets, graphs, trees, multimaps, etc), but most (including the common ones of JSON and XML) do not. Likewise, some data representations allow for commutative operations so as to reconcile concurrent actors' activity, but most (again including JSON and XML) do not.

ChuckMcM 12y ago

Fun stuff, amusing that the definition of a distributed system used; "Where a computer that you never heard of can bring your system down." is actually one of Leslie Lamport's more famous quotes. When I joined Sun in '86 I thought it was the pinnacle of technological excellence to be a kernel programmer, and I joined the Systems Group, the notional center of the Sun universe, in 1987. However I discovered that the primary reason you had to be picky about kernel programmers what that their bogus pointer references crashed the machine (as they occurred in kernel mode with full privileges) but discovered that network programmers could crash the whole world with their bugs. So clearly they must be in a pantheon above kernel programmers. :-) The author has come to discover that in the network world things can die anywhere, and this makes reasoning about such systems very complicated. Having been a part of the RPC and CORBA evolution I keenly felt the challenges of making APIs that "looked" like function calls to a programmer but took place across a network fabric and thus introduced error conditions that couldn't exist in locally called routines. (like the inability to return from the function due to a network partition for a simple example). Lamport's work in this space is brilliant and inspired. Network systems can be analysed and reasoned about as physical systems when they exhibit discontinuities when considered as simple algorithms. The value here is to realize that a large number of physical systems tolerate a tremendous amount of randomness and continue to work as intended (windmills for example) while many algorithms only work consistently given a set of key invariants. I gave a talk that was inspired by Dr. Lamports work titled 'Java as Newtonian Physics' which was a call to action to create a set of invariants, in the spirit of physical laws, that would govern the behavior and capabilities of distributed systems. It was way early for its time (AOL dialup connections were still a thing) but much of the same inspiration (presumably from Lamport) made it into the Google Spanner project. As with many things, at a surface level many people learn an API which does something under the covers across the network but having come up through their education thinking of everything as an API they don't fundamentally grasp the notion of distributed computation. Then at some point in their experience there will be that 'ah ha' moment when suddenly everything they know is wrong, which really means they suddenly see a bigger picture of things. It makes distributed systems questions in interviews an excellent litmus test for understanding where people are in their journey.

jacquesm 12y ago

dgreensp 12y ago

As the author of EtherPad I'm familiar with CRDT, which is a cousin of OT. They don't really replace APIs, unless you are using an API to synchronize data, which is only one of many things you might be trying to do. In other words, if you're building EtherPad or Wave, use a fancy data structure for the collaborative document. Otherwise, don't. Meteor's DDP provides a nice model, where the results of RPCs stream in asynchronously.

cemerick 12y ago
Hi, author here. I'm not sure you read the whole piece. :-) (Modern) APIs are a very limited mechanism of state transfer that happens to be paired with often side-effecting operations. Thus, a "synchronization" (I don't think that word is particularly useful because reasons) mechanism paired with reactive computational services _does_ replace APIs, and offers the ability to do much, much more. OTs (operational transforms) _are_ a related precursor to CRDTs only in that they are both ways to reconcile concurrent changes, but that's really the limit of the connection. Unfortunately, the substrate for OTs (text, integer-indexed sequences of characters) is fundamentally not amenable to commutative operations. This makes implementing OTs _very_ difficult and error-prone, and certain combinations of concurrent operations are completely unreconcilable (a result that came out of a Korean group's study, can't find the cite for it right now).
- jorangreef 12y ago
  I think the paper you are referencing might be [1]? It's one of my favorite papers on CRDTs and provides practical pseudocode for learning how to implement CRDTs yourself. The structures they present are simple to understand and have good performance characteristics compared to similar CRDTs [2]. A key insight from the second paper is to write CRDTs that optimize for applying remote operations over applying local operations, as the ratio of remote operations to local operations will be greater. i.e. 100 clients making 1 change to a CRDT will require all 100 clients to each apply 99 remote operations and 1 local operation. [1] Replicated abstract data types: Building blocks for collaborative applications - http://dl.acm.org/citation.cfm?id=1931272 http://dl.acm.org/citation.cfm?id=1931272 [2] Evaluating CRDTs for Real-time Document Editing - http://hal.archives-ouvertes.fr/docs/00/62/95/03/PDF/doce63-ahmednacer.pdf http://hal.archives-ouvertes.fr/docs/00/62/95/03/PDF/doce63-...

ryanobjc 12y ago

CRDTs are absolutely fascinating, but sometimes I really wonder. It seems like you throw words like 'semi-lattice' around ... Also there is one particular element to the eventual consistency that bothers me, it's that all these eventually consistent algorithms aren't how high powered neural nets will work. Our brain is highly eventually consistent, but it computes without ever needing these algorithms.

bm1362 12y ago
I think you're taking the neural net _model_ of the brain too strictly.

anuraj 12y ago

This is a network programmers view. System, Network programmers concern themselves about systems and topologies. For an application programmer, both needs to be abstracted and only the business logic is important. Money is in the top of the pyramid now - hence the proliferation of APIs.

cemerick 12y ago
The point of the piece, in large part, was to emphasize that we're all network programmers. If you're whacking away with APIs and couldn't care less about the broader system its topology, please stay away from me. ;-) APIs are proliferating because their coupling around client/server process and data representation makes for high switching costs and thus sweet vendor lock-in.

sagargv 12y ago

Joel Spolsky had written along similar lines and argued that it's important to know what is happening beneath abstractions. http://www.joelonsoftware.com/articles/LeakyAbstractions.html http://www.joelonsoftware.com/articles/LeakyAbstractions.htm...