5 ms·

Understanding Practical API Design, Static Typing, and Functional Programming

48 points by jimminy 16y ago

kbutler 16y ago
This is an exercise in "How much can I enforce through the type system" rather than "How usable can I make the API." It involves a great deal more type system calisthenics than I find "practical" in API design.
- thesz 16y ago
  Let me introduce HOOPL to you: http://research.microsoft.com/en-us/um/people/simonpj/papers/c--/dfopt.pdf http://research.microsoft.com/en-us/um/people/simonpj/papers... "Hoopl: A Modular, Reusable Library for Dataflow Analysis and Transformation" They use invariants like those in article for practical API that happens to be in the greatest need of high dependability.
  - kbutler 16y ago
    Let me introduce Erlang to you: http://ll2.ai.mit.edu/talks/armstrong.pdf http://ll2.ai.mit.edu/talks/armstrong.pdf Slide 27 describes an Erlang system from the late 90's/early 2000's that contains 1.7M lines of Erlang, with "99.9999999% reliability" This suggests that complicated compiler type checking is not "the greatest need of high dependability".
thurn 16y ago
Enforcing things at the type level certainly has its place, but I have to think that wading through that TicTacToe API would probably introduce a lot more cognitive load than a nice simple dynamically typed one that raised errors when you did the wrong thing. You can still have correctness guarantees if you validate actions at runtime.
- justincormack 16y ago
  I am not sure that it has its place in distributed systems. That sort of tight service coupling was the SOAP model. A REST model will tell you the valid actions at runtime ie the valid moves or actions as URLs, a looser coupling and also safe.
  - Peaker 16y ago
    How does static typing increase coupling?
    - justincormack 16y ago
      You have to recompile both sides of the API if you change the types and redeploy together.
      - Peaker 16y ago
        A change of types would almost always accompany a change in semantics. Wouldn't you need to redeploy both anyway when you have a change of semantics? If you retain the old API with the old semantics, you could of course retain the old type, too.
        justincormack 16y ago
        No, you can add a field on a server that some clients do not yet use without changing the semantics for the clients that do not know about it, for example. The loosely coupled model is more or less Duck typing for interfaces.
skybrian 16y ago
He wouldn't get any support calls because it would be very hard to get a paying customer to use a Java API like this. In particular, TakenBack.fold() isn't idiomatic Java. In other words, he's basically defined the question so that only people who think like him can answer it. And yet, this isn't the only way to write a tic tac toe solver.
carsongross 16y ago
Worse is better. http://www.jwz.org/doc/worse-is-better.html http://www.jwz.org/doc/worse-is-better.html From the point of view of practical developer productivity, static typing is about tool support. Especially the all-mighty "list of stuff actually relevant after I hit the '.' key," all other tools being mere footnotes to the weeping tiger blood tears of this tools win. Catching type errors at compile time is great as long as it doesn't fk up your API too much, but it's a far second to tools when it comes to getting sh*t done.
nightski 16y ago
Great article, wish more developers thought like this. Our industry is so incredibly frustrating with its low barrier to entry. Most programmers won't even come close to appreciating the techniques discussed here.
Silhouette 16y ago
> I was really tempted to title this post, "What to Solve Before Expecting Me to Take Your Opinions of Static Typing Seriously", however, I figured this would be a bit too controversial and might detract from the points I wish to make. Nevertheless, I just want to make note, I think it is a very appropriate title. Then kindly allow me to reciprocate. Here are a few things you should do if you want me, as an experienced professional software developer, to take your article and/or training seriously: 1. Don't lead with a transparent appeal to your own authority. 2. Don't put a transparent ad hominem attack near the end. 3. Don't specify the functional requirements to be supported by your API with little more than a vague one-liner, but then specify in intricate detail those implementation details where you have prejudged the only acceptable way to meet the requirements. 4. Don't dismiss real world processes that, for all their problems, are clearly driven more by producing working software than dogma. 5. Don't write a whole post about how smart you are, include some superficial documentation that shows you are relying on unusual concepts implemented using functionality that is not in either the standard library or your own documentation, fail to include any actual code, and then pretend you're just doing us a favour. 6. Don't call the type system in a language that powers vast amounts of real world software "impractical". Inelegant? Perhaps. Theoretically weak? Sure. But Java's type system is, on the evidence to date, vastly more practical than the kind of system you apparently prefer as an academic. 7. Don't suggest that a good set of types is a substitute for documentation. That's about as dumb as the XP guys saying that a comprehensive suite of unit tests is a substitute for documentation, and for much the same reasons. 8. Most of all, don't pretend that the techniques you advocate are somehow justified on the basis of some hypothetical real world support problem, when you would have significant problems shipping in the first place if you actually adopted them in a large-scale real world project today. Your whole Java example is based on anti-idiomatic "clever" code, and that sort of "clever" code is usually a bad idea in real world projects. If the ideas have sufficient merit, use tools that support them idiomatically. If the cost of switching tools is prohibitive for some reason, then IME it is nearly always better to stick with idiomatic practices that fit with the tools you are using than to try to get a round peg into a square hole that might deform just enough if you push really hard and none of your colleagues will ever need to move the peg again later. 9. Oh, and don't make sweeping claims about the practical relevance of ideas just because they work in a very simple scenario like your example, unless you also have a good argument for how those ideas scale up to more realistic projects as well. But apart from those things, I was completely convinced, so I repent my evil ways and will henceforth develop all of the code for which my clients pay me using your proposed policies. Oh, hang on, like most real projects we are working in an infinite problem space with many continuous variables, and not a finite, small, discrete, problem space, so that pretty much invalidates your entire argument. Sorry.
gfodor 16y ago
The clever tricks used may make the code easier to write, but come at the cost of readability. Given the choice between smarter compiler errors but types that fail to model the domain intuitively, and relying on a unit test suite but getting more readable code, I know which choice I'd make.
- nightski 16y ago
  These are not clever tricks, they are the mathematical foundation of our profession. Why do you assume the types fail to model the domain intuitively? Have you ever heard of denotational semantics?
  - zoul 16y ago
    They could be considered clever tricks because the Java type system does not seem very well suited to expressing such higher-level invariants. These excercises are interesting, but smarter solutions are not magically guaranteed to perform better in the real world. As for denotational semantics, you should either explain your point fully or this is nothing more than a trivial condescendig remark. [edited for grammar]
  - Silhouette 16y ago
    > These are not clever tricks, they are the mathematical foundation of our profession. Perhaps, but many of the best real world programming tools have a theoretically sound base but sufficient abstraction that the programmer does not need to concern himself with it. > Why do you assume the types fail to model the domain intuitively? Well, using six classes/interfaces and an enumeration just to describe a 3x3 array is one clue. The horrific complexity of the proposed BoardLike interface is another. And I would argue that the very existence of the Position enumeration is a third, betraying an underlying complexity that isn't shown in the interface here but is almost certainly present throughout the underlying implementation. > Have you ever heard of denotational semantics? I have! I have! Can I be clever too now, please?
    - nightski 16y ago
joe_the_user 16y ago
If nothing else, this article shows how extraordinarily difficult it is to make a good api and also how hard it is to decide what a good api is. For example, his scheme involves making it extremely hard to do a lot when your program could do something wrong - if your code could take back a move at the opening, you can't even compile it. This might be very good but it might very bad, depending on the complexity of undos. Personally, I'm still working on determining what a good API is. I like the QT guidelines on the subject - they feel more down to earth than a lot of comments. http://qt.gitorious.org/qt/pages/ApiDesignPrinciples http://qt.gitorious.org/qt/pages/ApiDesignPrinciples
davidmathers 16y ago
How about: load all 63,792 valid games into a relational database and automagically get pure functions along with an algebra for function composition. Simple, clean, straightforward. Denotational semantics. Is that cheating?
warrenwilkinson 16y ago
I think this API is over-engineered here is what I would do: 1 global integer called board (9*2 bits, enough to represent every square on the board as 0=vacant, 1=X, 2=O), and a list of agent callbacks. The program loops through the agents calling each in turn. The agent does whatever it wants, but generally computes the next state from the current state (either via AI, GUI, or network) and sets the global board variable. It is the agents responsibily to operate correctly. If the GUI or AI wants to cheat, undo moves, call in a pinch-hitter, or change their opponents agent mid-game, let them (all they need do is set the state and callbacks appropriately). Both the GUI and AI should be designed to be stateless in this regard: display or compute off the current state, whatever it is. There's no need for the API to detect when the game is over, nor enforce rules. The AI plays it correctly or is rewritten. The GUI echos the users desire, so let it do whatever it wants. If you want to prevent cheating, just add a cheat-detector callback between each agent. If you want to know when the game ends, have your GUI detect it and declare the winner. The AI doesn't even need to know when the game ends because you'd obviously have a GUI to display the battle -- let the gui determine who has won and when. (Before you laugh, consider that this design would reduce the API size from [assumably] ~150 loc (plus comments) to about 5 simple, flexible lines of code: 1 global variable, 1 global array of callbacks, and 1 function to unendingly iterate over the callbacks.)

48 points by jimminy 16y ago

This is an exercise in "How much can I enforce through the type system" rather than "How usable can I make the API." It involves a great deal more type system calisthenics than I find "practical" in API design.

thesz 16y ago
Let me introduce HOOPL to you: http://research.microsoft.com/en-us/um/people/simonpj/papers/c--/dfopt.pdf http://research.microsoft.com/en-us/um/people/simonpj/papers... "Hoopl: A Modular, Reusable Library for Dataflow Analysis and Transformation" They use invariants like those in article for practical API that happens to be in the greatest need of high dependability.
- kbutler 16y ago
  Let me introduce Erlang to you: http://ll2.ai.mit.edu/talks/armstrong.pdf http://ll2.ai.mit.edu/talks/armstrong.pdf Slide 27 describes an Erlang system from the late 90's/early 2000's that contains 1.7M lines of Erlang, with "99.9999999% reliability" This suggests that complicated compiler type checking is not "the greatest need of high dependability".

thurn 16y ago

Enforcing things at the type level certainly has its place, but I have to think that wading through that TicTacToe API would probably introduce a lot more cognitive load than a nice simple dynamically typed one that raised errors when you did the wrong thing. You can still have correctness guarantees if you validate actions at runtime.

justincormack 16y ago
I am not sure that it has its place in distributed systems. That sort of tight service coupling was the SOAP model. A REST model will tell you the valid actions at runtime ie the valid moves or actions as URLs, a looser coupling and also safe.
- Peaker 16y ago
  How does static typing increase coupling?
  - justincormack 16y ago
    You have to recompile both sides of the API if you change the types and redeploy together.
    - Peaker 16y ago
      A change of types would almost always accompany a change in semantics. Wouldn't you need to redeploy both anyway when you have a change of semantics? If you retain the old API with the old semantics, you could of course retain the old type, too.
      - justincormack 16y ago
        No, you can add a field on a server that some clients do not yet use without changing the semantics for the clients that do not know about it, for example. The loosely coupled model is more or less Duck typing for interfaces.

skybrian 16y ago

He wouldn't get any support calls because it would be very hard to get a paying customer to use a Java API like this. In particular, TakenBack.fold() isn't idiomatic Java. In other words, he's basically defined the question so that only people who think like him can answer it. And yet, this isn't the only way to write a tic tac toe solver.

carsongross 16y ago

Worse is better. http://www.jwz.org/doc/worse-is-better.html http://www.jwz.org/doc/worse-is-better.html From the point of view of practical developer productivity, static typing is about tool support. Especially the all-mighty "list of stuff actually relevant after I hit the '.' key," all other tools being mere footnotes to the weeping tiger blood tears of this tools win. Catching type errors at compile time is great as long as it doesn't fk up your API too much, but it's a far second to tools when it comes to getting sh*t done.

nightski 16y ago

Great article, wish more developers thought like this. Our industry is so incredibly frustrating with its low barrier to entry. Most programmers won't even come close to appreciating the techniques discussed here.

Silhouette 16y ago

> I was really tempted to title this post, "What to Solve Before Expecting Me to Take Your Opinions of Static Typing Seriously", however, I figured this would be a bit too controversial and might detract from the points I wish to make. Nevertheless, I just want to make note, I think it is a very appropriate title. Then kindly allow me to reciprocate. Here are a few things you should do if you want me, as an experienced professional software developer, to take your article and/or training seriously: 1. Don't lead with a transparent appeal to your own authority. 2. Don't put a transparent ad hominem attack near the end. 3. Don't specify the functional requirements to be supported by your API with little more than a vague one-liner, but then specify in intricate detail those implementation details where you have prejudged the only acceptable way to meet the requirements. 4. Don't dismiss real world processes that, for all their problems, are clearly driven more by producing working software than dogma. 5. Don't write a whole post about how smart you are, include some superficial documentation that shows you are relying on unusual concepts implemented using functionality that is not in either the standard library or your own documentation, fail to include any actual code, and then pretend you're just doing us a favour. 6. Don't call the type system in a language that powers vast amounts of real world software "impractical". Inelegant? Perhaps. Theoretically weak? Sure. But Java's type system is, on the evidence to date, vastly more practical than the kind of system you apparently prefer as an academic. 7. Don't suggest that a good set of types is a substitute for documentation. That's about as dumb as the XP guys saying that a comprehensive suite of unit tests is a substitute for documentation, and for much the same reasons. 8. Most of all, don't pretend that the techniques you advocate are somehow justified on the basis of some hypothetical real world support problem, when you would have significant problems shipping in the first place if you actually adopted them in a large-scale real world project today. Your whole Java example is based on anti-idiomatic "clever" code, and that sort of "clever" code is usually a bad idea in real world projects. If the ideas have sufficient merit, use tools that support them idiomatically. If the cost of switching tools is prohibitive for some reason, then IME it is nearly always better to stick with idiomatic practices that fit with the tools you are using than to try to get a round peg into a square hole that might deform just enough if you push really hard and none of your colleagues will ever need to move the peg again later. 9. Oh, and don't make sweeping claims about the practical relevance of ideas just because they work in a very simple scenario like your example, unless you also have a good argument for how those ideas scale up to more realistic projects as well. But apart from those things, I was completely convinced, so I repent my evil ways and will henceforth develop all of the code for which my clients pay me using your proposed policies. Oh, hang on, like most real projects we are working in an infinite problem space with many continuous variables, and not a finite, small, discrete, problem space, so that pretty much invalidates your entire argument. Sorry.

gfodor 16y ago

The clever tricks used may make the code easier to write, but come at the cost of readability. Given the choice between smarter compiler errors but types that fail to model the domain intuitively, and relying on a unit test suite but getting more readable code, I know which choice I'd make.

nightski 16y ago
These are not clever tricks, they are the mathematical foundation of our profession. Why do you assume the types fail to model the domain intuitively? Have you ever heard of denotational semantics?
- zoul 16y ago
  They could be considered clever tricks because the Java type system does not seem very well suited to expressing such higher-level invariants. These excercises are interesting, but smarter solutions are not magically guaranteed to perform better in the real world. As for denotational semantics, you should either explain your point fully or this is nothing more than a trivial condescendig remark. [edited for grammar]
- Silhouette 16y ago
  > These are not clever tricks, they are the mathematical foundation of our profession. Perhaps, but many of the best real world programming tools have a theoretically sound base but sufficient abstraction that the programmer does not need to concern himself with it. > Why do you assume the types fail to model the domain intuitively? Well, using six classes/interfaces and an enumeration just to describe a 3x3 array is one clue. The horrific complexity of the proposed BoardLike interface is another. And I would argue that the very existence of the Position enumeration is a third, betraying an underlying complexity that isn't shown in the interface here but is almost certainly present throughout the underlying implementation. > Have you ever heard of denotational semantics? I have! I have! Can I be clever too now, please?
  - nightski 16y ago

joe_the_user 16y ago

If nothing else, this article shows how extraordinarily difficult it is to make a good api and also how hard it is to decide what a good api is. For example, his scheme involves making it extremely hard to do a lot when your program could do something wrong - if your code could take back a move at the opening, you can't even compile it. This might be very good but it might very bad, depending on the complexity of undos. Personally, I'm still working on determining what a good API is. I like the QT guidelines on the subject - they feel more down to earth than a lot of comments. http://qt.gitorious.org/qt/pages/ApiDesignPrinciples http://qt.gitorious.org/qt/pages/ApiDesignPrinciples

davidmathers 16y ago

How about: load all 63,792 valid games into a relational database and automagically get pure functions along with an algebra for function composition. Simple, clean, straightforward. Denotational semantics. Is that cheating?

warrenwilkinson 16y ago

I think this API is over-engineered here is what I would do: 1 global integer called board (9*2 bits, enough to represent every square on the board as 0=vacant, 1=X, 2=O), and a list of agent callbacks. The program loops through the agents calling each in turn. The agent does whatever it wants, but generally computes the next state from the current state (either via AI, GUI, or network) and sets the global board variable. It is the agents responsibily to operate correctly. If the GUI or AI wants to cheat, undo moves, call in a pinch-hitter, or change their opponents agent mid-game, let them (all they need do is set the state and callbacks appropriately). Both the GUI and AI should be designed to be stateless in this regard: display or compute off the current state, whatever it is. There's no need for the API to detect when the game is over, nor enforce rules. The AI plays it correctly or is rewritten. The GUI echos the users desire, so let it do whatever it wants. If you want to prevent cheating, just add a cheat-detector callback between each agent. If you want to know when the game ends, have your GUI detect it and declare the winner. The AI doesn't even need to know when the game ends because you'd obviously have a GUI to display the battle -- let the gui determine who has won and when. (Before you laugh, consider that this design would reduce the API size from [assumably] ~150 loc (plus comments) to about 5 simple, flexible lines of code: 1 global variable, 1 global array of callbacks, and 1 function to unendingly iterate over the callbacks.)