5 ms·

> If you’re building an abstraction-heavy API, be prepared to think hard before adding new features. If you’re building an abstraction-light API, commit to it a

> If you’re building an abstraction-heavy API, be prepared to think hard before adding new features. If you’re building an abstraction-light API, commit to it and resist the temptation to add abstractions when it comes along. You could always do both. Provide a low-level abstraction-light API that allows fine control but requires deep expertise, and write a higher-level abstraction-rich API on top of it that maps to fewer simple operations for the most common use cases - which some of your clients might be implementing their own half-baked versions of anyway. If you maintain a clean separation between the two, having both in place might mean there is less pressure to add abstractions to the low-level API, or to add warts and special-cases to the high-level API. If a client wants one of those things, it already exists - in the other API. Bonus points for providing materials to help your clients learn how to move from one to the other. You can attract clients who do not yet have deep knowledge of payment network internals, but are looking to improve in that direction.

paulddraper 2y ago
Git is an example of this. [1] There are high-level "porcelain" commands like branch and checkout. And then there are low-level "plumbing" commands like commit-tree and update-ref. [1] https://git-scm.com/book/en/v2/Git-Internals-Plumbing-and-Porcelain https://git-scm.com/book/en/v2/Git-Internals-Plumbing-and-Po...
- mbork_pl 2y ago
  Came here to say that. Also, to some extent, Emacs. There are thousands of functions (actually, a bit less than 10k in stock Emacs without packages, and over 46k in my Emacs) performing various low-level tasks, and much fewer commands (~3k in stock Emacs, almost 12k in my config), i.e., interactive functions, often higher-level, designed for the user.
deleted 2y ago
[deleted]
yen223 2y ago
It does double your API surface area, so that's the tradeoff you'll have to consider. It can be the correct decision in a lot of cases.
- Pxtl 2y ago
  It doesn't double the security surface area if the abstracted API goes through the low-level API. The outer one is just chrome, and so the risks of screwing something up there is far lower. Unless you're using a trash language where even simple wrappers could buffer underrun or something.
  - lmz 2y ago
    Isn't there the issue of modifying the state enough through the low-level API such that it breaks the assumptions of the high-level one?
    - eru 2y ago
      Just say that you don't support mixing both APIs.
      - lazyasciiart 2y ago
        Now you’re telling users to check the source code of any tools they use with your stuff? “We recommend using ToolOne only on content you do not also manage with ToolTwo”
        eru 2y ago
        See how libfuse handles it with their low level and high level APIs.
      - actionfromafar 2y ago
        That’ll fix it. ;-)
    - withinboredom 2y ago
jampekka 2y ago
This. There should be a low level API to be able to do rarer more complicated cases, and a higher level simple API for common cases built on the lower-level API. Just today I was working with the Web File System API, and e.g. just writing a string to a file requires seven function calls, most async. And this doesn't even handle errors. And has to be done in a worker, setting up of which is similar faff in itself. Similar horrors can be seen in e.g. IndexedDB, WebRTC and even plain old vanilla DOM manipulation. And things like Vulkan and DirectX and ffmpeg are even way worse. The complexity can be largely justified to be able to handle all sorts of exotic cases, but vast majority of cases aren't these exotic ones. API design should start first by sketching out how using the API looks for common cases, and those should be as simple as possible. E.g. the fetch API does this quite well. XMLHttpRequest definitely did not. https://developer.mozilla.org/en-US/docs/Web/API/FileSystemSyncAccessHandle https://developer.mozilla.org/en-US/docs/Web/API/FileSystemS... Edit: I've thought many a time that there should be some unified "porcelain" API for all the Web APIs. It would wrap all the (awesome) features in one coherent "standard library" wrapper, supporting at least the most common use cases. Modern browsers are very powerful and capable but a lot of this power and capability is largely unknown or underused because each API tends to have quite an idiosyncratic design and they are needlessly hard to learn and/or use. Something like what jQuery did for DOM (but with less magic and no extra bells and whistles). E.g. node.js has somewhat coherent, but a bit outdated APIs (e.g. Promise support is spotty and inconsistent). Something like how Python strives for "pythonic" APIs (with varying success).
- metalspoon 2y ago
  Tbf Vulkan is not intended for an endprogrammer. It is a deliberately low level standardization to allow directly control GPU hardware. The high-level approach (OpenGL) failed. The endprogrammer is supposed to use a third party middleware, not Vulkan itself.
  - withinboredom 2y ago
    Someone has to write that middleware…
  - f1shy 2y ago
InvisibleUp 2y ago
.NET also does this a lot. Here's a recent devblog post looking at file I/O: https://devblogs.microsoft.com/dotnet/the-convenience-of-system-io/ https://devblogs.microsoft.com/dotnet/the-convenience-of-sys...
- eterm 2y ago
  That's a fantastic blog post, worthy of it's own HN submission.
Groxx 2y ago
I'm particularly fond of this pattern when you can implement the high level API that you want outside the library, which ensures that your low level API is sufficiently flexible and means you're dogfooding your own stuff as a user. It's far too easy to get used to the internal side of a tool you're building, and forget how people actually use it.
- HelloNurse 2y ago
  It is also important to guarantee that the two API designs are coherent and interoperable, and this kind of strict layering is the best strategy to avoid mistakes.
andenacitelli 2y ago
Another example of this is AWS CDK. There are a few “levels” of constructs - high level ones that are simpler and apply “presets” that are enough for 80% of users, but the core low level ones still exist if you have a nonstandard use case.
zellyn 2y ago
Problems can sneak in when you use the low-level API to do something to an object that can't be cleanly represented in the higher-level API. You need some kind of escape hatch, like a list of links to or ids of low-level details or a blob (Map<String,arbitrary-JSON> of miscellaneous data that can hold the low-level additions. Hopefully the top-level important concepts like "amount_due" will still reflect the correct numbers!
- withinboredom 2y ago
  Those problems usually present themselves by people overthinking the high level api and trying to be smart. As an example, you can use chattr to make a file in Linux immutable. ls still shows that you have permission to write to the file, even though it will fail. When people try to overthink the api and have it determine if you really can write to a file, people will try using the high level api first (chmod) and it won’t work because it has nothing to do with permissions. KISS is really needed for high level APIs.
eru 2y ago
Compare also exokernels, and how they delegate abstraction to libraries, not the OS.
lwhi 2y ago
I like this idea a lot. One level of API for implementation model. And second level for mental model.
devjab 2y ago
I’m genuinely curious as to what an abstraction-rich api would look like and why it would be useful. I’ve mainly worked in enterprise organisations or in startups transitioning into enterprise which is sort of where my expertise lies. I’ve never seen an API that wasn’t similar to the examples in this case. I mean… I have… but they wouldn’t be labelled as high-abstraction api’s. If they needed a label it would be terrible APIs. Like sending table headers, column types in another full length array with a line for each column, and then the actual data/content in a third array. Even sending the html style that was used in what ever frontend, so that some data is represented as “some data” and other is represented as [“some data”, [“text-aligned”, “center”…],… . Yes, that is an actual example. Anyway I’ve never seen a high abstraction api and I feel like I’m missing out.
- Karellen 2y ago
  Compare GTK/Qt to raw xlib/wayland/win32/cocoa primitives - including the way that those toolkits abstract away the differences. Or (as others have pointed out) the various `git` "porcelain" commands (checkout/add/commit/branch) compared to the primitive operations for dealing with various object types. Even just `git pull` as a combination of `git fetch` and `git merge`. Or how a filesystem API of open/read/write/close/unlink is a very simple abstraction over block allocation and (in the old days) moving a disk head and waiting for the platter to spin under it in order to access the right sector. Not to mention the "directory" and "subdirectory" abstraction, instead of just having one giant table of inode numbers. Edit: Or compare HTML with abstractions like "headings" and "paragraphs" to a raw typesetting language like troff or TeX.
ChrisMarshallNY 2y ago
That's pretty much what Apple does, and, I suspect, Google.
neeleshs 2y ago
Agreed. We have a rich API to build complex pipelines, but a lot of times, users do simple mappings. So there is an API for that common use case, which has a much simpler syntax, but internally uses the rich API
- _a_a_a_ 2y ago
  Please email this to Microsoft
jimbob45 2y ago
That sounds like the exact philosophy the Vulkan devs took versus OpenGL.
yixu34 2y ago
+1, I like to refer to this post on this topic: https://blog.sbensu.com/posts/apis-as-ladders/ https://blog.sbensu.com/posts/apis-as-ladders/
geophph 2y ago
matplotlib seems to have implemented this approach
sameoldtune 2y ago
“Make the easy things easy and the hard things possible”

paulddraper 2y ago

Git is an example of this. [1] There are high-level "porcelain" commands like branch and checkout. And then there are low-level "plumbing" commands like commit-tree and update-ref. [1] https://git-scm.com/book/en/v2/Git-Internals-Plumbing-and-Porcelain https://git-scm.com/book/en/v2/Git-Internals-Plumbing-and-Po...

mbork_pl 2y ago
Came here to say that. Also, to some extent, Emacs. There are thousands of functions (actually, a bit less than 10k in stock Emacs without packages, and over 46k in my Emacs) performing various low-level tasks, and much fewer commands (~3k in stock Emacs, almost 12k in my config), i.e., interactive functions, often higher-level, designed for the user.

deleted 2y ago

[deleted]

yen223 2y ago

It does double your API surface area, so that's the tradeoff you'll have to consider. It can be the correct decision in a lot of cases.

Pxtl 2y ago
It doesn't double the security surface area if the abstracted API goes through the low-level API. The outer one is just chrome, and so the risks of screwing something up there is far lower. Unless you're using a trash language where even simple wrappers could buffer underrun or something.
- lmz 2y ago
  Isn't there the issue of modifying the state enough through the low-level API such that it breaks the assumptions of the high-level one?
  - eru 2y ago
    Just say that you don't support mixing both APIs.
    - lazyasciiart 2y ago
      Now you’re telling users to check the source code of any tools they use with your stuff? “We recommend using ToolOne only on content you do not also manage with ToolTwo”
      - eru 2y ago
        See how libfuse handles it with their low level and high level APIs.
    - actionfromafar 2y ago
      That’ll fix it. ;-)
  - withinboredom 2y ago

jampekka 2y ago

This. There should be a low level API to be able to do rarer more complicated cases, and a higher level simple API for common cases built on the lower-level API. Just today I was working with the Web File System API, and e.g. just writing a string to a file requires seven function calls, most async. And this doesn't even handle errors. And has to be done in a worker, setting up of which is similar faff in itself. Similar horrors can be seen in e.g. IndexedDB, WebRTC and even plain old vanilla DOM manipulation. And things like Vulkan and DirectX and ffmpeg are even way worse. The complexity can be largely justified to be able to handle all sorts of exotic cases, but vast majority of cases aren't these exotic ones. API design should start first by sketching out how using the API looks for common cases, and those should be as simple as possible. E.g. the fetch API does this quite well. XMLHttpRequest definitely did not. https://developer.mozilla.org/en-US/docs/Web/API/FileSystemSyncAccessHandle https://developer.mozilla.org/en-US/docs/Web/API/FileSystemS... Edit: I've thought many a time that there should be some unified "porcelain" API for all the Web APIs. It would wrap all the (awesome) features in one coherent "standard library" wrapper, supporting at least the most common use cases. Modern browsers are very powerful and capable but a lot of this power and capability is largely unknown or underused because each API tends to have quite an idiosyncratic design and they are needlessly hard to learn and/or use. Something like what jQuery did for DOM (but with less magic and no extra bells and whistles). E.g. node.js has somewhat coherent, but a bit outdated APIs (e.g. Promise support is spotty and inconsistent). Something like how Python strives for "pythonic" APIs (with varying success).

metalspoon 2y ago
Tbf Vulkan is not intended for an endprogrammer. It is a deliberately low level standardization to allow directly control GPU hardware. The high-level approach (OpenGL) failed. The endprogrammer is supposed to use a third party middleware, not Vulkan itself.
- withinboredom 2y ago
  Someone has to write that middleware…
- f1shy 2y ago

InvisibleUp 2y ago

.NET also does this a lot. Here's a recent devblog post looking at file I/O: https://devblogs.microsoft.com/dotnet/the-convenience-of-system-io/ https://devblogs.microsoft.com/dotnet/the-convenience-of-sys...

eterm 2y ago
That's a fantastic blog post, worthy of it's own HN submission.

Groxx 2y ago

I'm particularly fond of this pattern when you can implement the high level API that you want outside the library, which ensures that your low level API is sufficiently flexible and means you're dogfooding your own stuff as a user. It's far too easy to get used to the internal side of a tool you're building, and forget how people actually use it.

HelloNurse 2y ago
It is also important to guarantee that the two API designs are coherent and interoperable, and this kind of strict layering is the best strategy to avoid mistakes.

andenacitelli 2y ago

Another example of this is AWS CDK. There are a few “levels” of constructs - high level ones that are simpler and apply “presets” that are enough for 80% of users, but the core low level ones still exist if you have a nonstandard use case.

zellyn 2y ago

Problems can sneak in when you use the low-level API to do something to an object that can't be cleanly represented in the higher-level API. You need some kind of escape hatch, like a list of links to or ids of low-level details or a blob (Map<String,arbitrary-JSON> of miscellaneous data that can hold the low-level additions. Hopefully the top-level important concepts like "amount_due" will still reflect the correct numbers!

withinboredom 2y ago
Those problems usually present themselves by people overthinking the high level api and trying to be smart. As an example, you can use chattr to make a file in Linux immutable. ls still shows that you have permission to write to the file, even though it will fail. When people try to overthink the api and have it determine if you really can write to a file, people will try using the high level api first (chmod) and it won’t work because it has nothing to do with permissions. KISS is really needed for high level APIs.

eru 2y ago

Compare also exokernels, and how they delegate abstraction to libraries, not the OS.

lwhi 2y ago

I like this idea a lot. One level of API for implementation model. And second level for mental model.

devjab 2y ago

I’m genuinely curious as to what an abstraction-rich api would look like and why it would be useful. I’ve mainly worked in enterprise organisations or in startups transitioning into enterprise which is sort of where my expertise lies. I’ve never seen an API that wasn’t similar to the examples in this case. I mean… I have… but they wouldn’t be labelled as high-abstraction api’s. If they needed a label it would be terrible APIs. Like sending table headers, column types in another full length array with a line for each column, and then the actual data/content in a third array. Even sending the html style that was used in what ever frontend, so that some data is represented as “some data” and other is represented as [“some data”, [“text-aligned”, “center”…],… . Yes, that is an actual example. Anyway I’ve never seen a high abstraction api and I feel like I’m missing out.

Karellen 2y ago
Compare GTK/Qt to raw xlib/wayland/win32/cocoa primitives - including the way that those toolkits abstract away the differences. Or (as others have pointed out) the various `git` "porcelain" commands (checkout/add/commit/branch) compared to the primitive operations for dealing with various object types. Even just `git pull` as a combination of `git fetch` and `git merge`. Or how a filesystem API of open/read/write/close/unlink is a very simple abstraction over block allocation and (in the old days) moving a disk head and waiting for the platter to spin under it in order to access the right sector. Not to mention the "directory" and "subdirectory" abstraction, instead of just having one giant table of inode numbers. Edit: Or compare HTML with abstractions like "headings" and "paragraphs" to a raw typesetting language like troff or TeX.

ChrisMarshallNY 2y ago

That's pretty much what Apple does, and, I suspect, Google.

neeleshs 2y ago

Agreed. We have a rich API to build complex pipelines, but a lot of times, users do simple mappings. So there is an API for that common use case, which has a much simpler syntax, but internally uses the rich API

_a_a_a_ 2y ago
Please email this to Microsoft

jimbob45 2y ago

That sounds like the exact philosophy the Vulkan devs took versus OpenGL.

yixu34 2y ago

+1, I like to refer to this post on this topic: https://blog.sbensu.com/posts/apis-as-ladders/ https://blog.sbensu.com/posts/apis-as-ladders/

geophph 2y ago

matplotlib seems to have implemented this approach

sameoldtune 2y ago

“Make the easy things easy and the hard things possible”