5 ms·

I've long been having a hunch that we're currently in the "wild west of abstraction". I think we're missing an essential constraint on the way we do abstractio

I've long been having a hunch that we're currently in the "wild west of abstraction". I think we're missing an essential constraint on the way we do abstraction. My hunch is that this constraint should be that abstractions must be reversible. Here's an example: When you use a compiler, you can work at a higher layer of abstraction (the higher-level language). But, this means you're now locked into that layer of abstraction. By that I mean, you can no longer work at the lower layer (assembly), even if you wanted to. You could in theory of course modify the compiler output after it's been generated, but then you'd have to somehow manually keep that work in sync whenever you want to re-generate. Using an abstraction kinda locks you into that layer. I see this problem appearing everywhere: - Use framework <--> Write from scratch - Use an ORM <--> Write raw SQL - Garbage collection <--> Manual memory management - Using a DSL <--> Writing raw language code - Cross platform UI framework <--> Native UI code - ... I think we're missing a fundamental primitive of abstraction that allows us to work on each layer of abstraction without being locked in. If you have any thoughts at all on this, please share them here!

ihumanable 3y ago
Lots of abstractions have an escape hatch down to the lower level, you can put assembly in your C code, most ORMs have some way to just run a query, etc. I think the question I have is, what benefit does this provide? Let's say we could wave a magic wand and you can operate at any layer of abstraction. Is this beneficial in some way? The article is about leaky abstractions and states > One reason the law of leaky abstractions is problematic is that it means that abstractions do not really simplify our lives as much as they were meant to. I think I'm just struggling to understand how this would help with that.
- cloogshicer 3y ago
  It would help because you could tackle the problem at hand always at the right layer of abstraction. If a certain aspect of the problem can be solved easily in a higher layer of abstraction, great! Let's solve it at that layer, because it's usually easier and allows for more expressiveness. But whenever we need more control, we can seamlessly drop down to the lower layer and work there. I think we need to find a fundamental principle that allows this. But I see barely anyone working on this - instead we keep trying to find higher and higher layers of abstractions (LLMs being the most recent addition) in the hopes they will get rid of the need of dealing with the lower layers. Which is a false hope, I feel.
titzer 3y ago
I work on programming languages and systems (virtual machines). A key thing with a systems programming language is that you need to be able to do things at the machine level. Here's a talk I gave a year ago about it: https://www.youtube.com/watch?v=jNcEBXqt9pU https://www.youtube.com/watch?v=jNcEBXqt9pU
eschneider 3y ago
That's not really true of, at least C compilers. Because compilers have ABI's and fixed calling conventions, it's straightforward, documented, and not uncommon (depending on your application area/deployment target) to drop down to the ASM layer if you need to do that. It's definitely one of those things that makes C nice for bare metal programming.
- cloogshicer 3y ago
  Interesting, I'm curious though, once you do drop down to the ASM layer, how do you ensure that this code doesn't get overwritten by new compiler output? Or is this something you somehow include in the compile step?
  - zro 3y ago
    In my experience (admittedly limited) you include the assembly in the compile step. Your linker hopefully puts everything together so you can talk to yourself
  - ufo 3y ago
    Typically you'd have some assembly functions in a separate file. Compile to an object file and link it, as you'd do with a separate C file. If you want to insert assembly snippets inside a C function, many C compilers have an inline assembly feature. For simple snippets, the C compiler can figure it out. For more complex things, you there are ways to tell the C compilers which registers you're using, so it does not step on your toes.
    - cloogshicer 3y ago
      Thank you for explaining!
  - eschneider 3y ago
    Assembly can either be inline with the C code or in a separate file. It's just source. Of course, if you really want to have fun with breaking abstractions, check out linker script files. That's for when you absolutely need different bits of code and data in specific parts of the application. Fun!
Veserv 3y ago
No, reversible abstractions are just one kind of abstraction. For instance, a machine code sequence to a linear sequence of assembly instructions is a reversible abstraction. Not every machine code sequence is expressible as a linear sequence of assembly instructions, but every linear sequence of assembly instructions has a trivial correspondence to a machine code sequence. However, consider the jump to a C-like language. The key abstraction provided there is the abstraction of infinite local variables. The compiler manages this through a stack, register allocation, and stack spilling to provide the abstraction and consumes your ability to control the registers directly to provide this abstraction. To interface at both levels simultaneously requires the leakage of the implementation details of the abstraction and careful interaction. What you can do easily is what I call a separable abstraction, a abstraction that can be restricted to just the places it is needed/removed where unneeded. In certain cases in C code you need to do some specific assembly instruction, sequence, or even function. This can be easily done by writing a assembly function that interfaces with the C code via the C ABI. What is happening there is that the C code defines a interface allowing you to drop down or even exit the abstraction hierarchy for the duration of that function. The ease of doing so makes C highly separable and is part of the reason why it is so easy to call out to C, but you hardly ever see anybody calling out to say Java or Haskell. Of course, that is just one of the many properties of abstractions that can make them easier to use, simpler, and more robust.
jerf 3y ago
Abstractions work by restricting the domain of what you can do, then building on those restrictions. For example, raw hardware can jump anywhere, but structured programming constrains you to jump only to certain locations in order to implement if, for, functions, etc. It is precisely those restrictions that bring the benefits of structured programming; if you still frequently dipped into jumping around directly structured programming would fail to provide the guarantees it is supposed to provide. CRUD frameworks provide their power by restricting you to CRUD operations, then building on that. Immutable data is accomplished by forbidding you from updating values even though the hardware will happily do it. And so on. Escape hatches under the abstractions are generally there precisely to break the abstractions, and break them they do. Abstractions necessarily involve being irreversible, or, to forestall a tedious discussion of the definition of "irreversible", necessarily involve making it an uphill journey to violate and go under the abstraction. There's no way around it. Careful thought can make using an escape hatch less pain than it might otherwise be (such as the ORM that makes it virtually impossible to use SQL by successfully hiding everything about the SQL tables from you so you're basically typing table and column names by dead reckoning), but that's all that can be done. One thing to do about this is that just as in the past few years the programming community has started to grapple with the fact that libraries aren't free but come with a certain cost that really adds up once you're pulling in a few thousand libraries for a framework's "hello world", abstractions that look really useful but whose restrictions don't match your needs need to be looked at a lot more closely. I had something like that happen to me just this week. I needed a simple byte ring buffer. I looked in my language's repos for an existing one. I found them. But they were all super complicated, offering tons of features I didn't need, like being a writethrough buffer (which involved taking restrictions I didn't want), or where the simple task of trying to understand the API was quite literally on par with implementing one myself. So I just wrote the simple thing. (Aiding this decision is that broadly speaking if this buffer does fail or have a bug it's not terribly consequential, in my situation it's only for logging output and only effectively at a very high DEBUG level.) It wasn't worth the restrictions to build up stuff I didn't even want.
- cloogshicer 3y ago
  > It is precisely those restrictions that bring the benefits [...] Wouldn't it be possible to say "ok, I'll take those restrictions as long as they benefit me, but once I notice that they no longer do, I'll break them and drop down to the lower layer. But only for those parts that actually require it"? > Abstractions necessarily involve being irreversible, or, to forestall a tedious discussion of the definition of "irreversible", necessarily involve making it an uphill journey to violate and go under the abstraction. Why? Not being snarky, I'm genuinely trying to understand this better.
samatman 3y ago
> Here's an example: When you use a compiler, you can work at a higher layer of abstraction (the higher-level language). But, this means you're now locked into that layer of abstraction. By that I mean, you can no longer work at the lower layer (assembly), even if you wanted to. Native-code compilers commonly allow emitting assembly directly, but now your source code isn't portable between CPUs. Many interpreted languages, even most, allow FFI code to be imported, modifying the runtime accordingly, but now your program isn't portable between implementations of that language, and you have to be careful to make sure the behavior you've introduced doesn't mess with other parts of the system in unexpected ways. Generalizing, it's often possible to drill down beneath the abstraction layer, but there's often an inherent price to be paid, whether it be taking pains to preserve the invariants of the abstraction, losing some of the benefits of it, or both. There are better and worse versions of this layer, I would point to Lua as a language which is explicitly designed to cross the C/Lua boundary in both directions, and which did a good job of it. But nothing can change the fact that pure-Lua code simply won't segfault, but bring in userdata and it very easily can; the problems posed are inherent.
wvenable 3y ago
Most ORMs give a way to integrate nicely with SQL if you need to reach down to that layer and still use the rest of the ORM features. There is no silver bullet; everything is a trade off. Almost all of the time, the trade off is entirely worth it even if that gets you locked into that solution.
- cloogshicer 3y ago
  > Most ORMs give a way to integrate nicely with SQL if you need to reach down to that layer and still use the rest of the ORM features. Agreed, that's a good thing, in my experience. > Almost all of the time, the trade off is entirely worth it even if that gets you locked into that solution. I wish this would match my experience.
  - wvenable 3y ago
    What's funny is that ORMs giving you directly access to SQL is a leaky part of the abstraction but in this case that's good! I think being locked into a some abstraction is so common place that you don't even consider it being a thing until you have a problem with it. Look at your examples: compilers, libraries, frameworks, etc. As an example, is anyone truly upset that coding in Python locks you into the Python ecosystem? Yet, I've used libraries/frameworks that didn't win the war of popularity and I'm still unfortunately committed. I think there's a bias at play in how we look at these things.
yen223 3y ago
There's a well-written article by Bret Victor on climbing the ladder of abstraction. It makes the same argument you made, in that climbing "down" the ladder is just as important as going "up" https://worrydream.com/LadderOfAbstraction/ https://worrydream.com/LadderOfAbstraction/
- cloogshicer 3y ago
  Thank you for posting, I love that article. Bret Victor is a genius in my opinion and his writings and talks have inspired many of the thoughts I've written above.
lmm 3y ago
The best paradigm for understanding abstractions is not the theory-and-model style (which requires hiding details irreversibly), but the equivalence style. A good abstraction is e.g. summing a list whose elements are a monoid - summing the list is equivalent to adding up all the elements in a loop. Crucially, this doesn't require you to "forget" the specific type of element that your list has - a bad version of this library would say that your list elements have to be subtypes of some "number" type and the sum of your list came back as a "number", permanently destroying the details of the specific type that it actually is. But with the monoid model your "sum" is still whatever complex type you wanted it to be - you've just summed it up in the way appropriate to that type.
- cloogshicer 3y ago
  Interesting! Can you point me to some further reading or resources on these different paradigms?
saghm 3y ago
> My hunch is that this constraint should be that abstractions must be reversible. > Here's an example: When you use a compiler, you can work at a higher layer of abstraction (the higher-level language). But, this means you're now locked into that layer of abstraction. By that I mean, you can no longer work at the lower layer (assembly), even if you wanted to. You could in theory of course modify the compiler output after it's been generated, but then you'd have to somehow manually keep that work in sync whenever you want to re-generate. Using an abstraction kinda locks you into that layer. Just to make sure I understand, you're proposing a constraint that would rule out every compiler in existence today? I feel like overall I think compilers have worked out well, but if I'm not misunderstanding and this is how you actually feel, I guess I at least should comment your audacity, because I don't think I'd be willing to seriously propose something that radical.
- yogorenapan 3y ago
  A lot of languages allow in-line assembly
  - saghm 3y ago
    I'm repeating the claim that the parent comment made; the assertion that compilers lock you into a level of abstraction is theirs, not mine.
- cloogshicer 3y ago
  Yes, you understood correctly. What I'm saying is extremely radical and would require rethinking and rebuilding practically everything we have.
  - saghm 3y ago
    Thanks for the clarification! I'm probably too far into the orthodoxy to seriously consider something like this, but I admire your willingness to question things that most of us probably take for granted.
__s 3y ago
I think this is a good way to frame abstraction vs macro
dhdjksosja 3y ago
Babel towers of macro edsls, aka learn lisp. https://github.com/combinatorylogic/mbase https://github.com/combinatorylogic/mbase
drewm1980 3y ago
You can probably find an IDE plugin that inlines the assembly for a c function. Most ids can show the assembly side by side with your c code so it wouldn't be that much of a step. To fulfill your vision you would also need a decompiler (and an inliner) to convert a block of assembly back into C if a corresponding C routine exists.
samsquire 3y ago
This is something I think a lot about. I spend a lot of time trying to think of something that composes. Monads are one answer. I think we need advanced term rewriting systems that also optimize and equivalise. I really enjoy Joel on Software blog posts from this era.

ihumanable 3y ago

Lots of abstractions have an escape hatch down to the lower level, you can put assembly in your C code, most ORMs have some way to just run a query, etc. I think the question I have is, what benefit does this provide? Let's say we could wave a magic wand and you can operate at any layer of abstraction. Is this beneficial in some way? The article is about leaky abstractions and states > One reason the law of leaky abstractions is problematic is that it means that abstractions do not really simplify our lives as much as they were meant to. I think I'm just struggling to understand how this would help with that.

cloogshicer 3y ago
It would help because you could tackle the problem at hand always at the right layer of abstraction. If a certain aspect of the problem can be solved easily in a higher layer of abstraction, great! Let's solve it at that layer, because it's usually easier and allows for more expressiveness. But whenever we need more control, we can seamlessly drop down to the lower layer and work there. I think we need to find a fundamental principle that allows this. But I see barely anyone working on this - instead we keep trying to find higher and higher layers of abstractions (LLMs being the most recent addition) in the hopes they will get rid of the need of dealing with the lower layers. Which is a false hope, I feel.

titzer 3y ago

I work on programming languages and systems (virtual machines). A key thing with a systems programming language is that you need to be able to do things at the machine level. Here's a talk I gave a year ago about it: https://www.youtube.com/watch?v=jNcEBXqt9pU https://www.youtube.com/watch?v=jNcEBXqt9pU

eschneider 3y ago

That's not really true of, at least C compilers. Because compilers have ABI's and fixed calling conventions, it's straightforward, documented, and not uncommon (depending on your application area/deployment target) to drop down to the ASM layer if you need to do that. It's definitely one of those things that makes C nice for bare metal programming.

cloogshicer 3y ago
Interesting, I'm curious though, once you do drop down to the ASM layer, how do you ensure that this code doesn't get overwritten by new compiler output? Or is this something you somehow include in the compile step?
- zro 3y ago
  In my experience (admittedly limited) you include the assembly in the compile step. Your linker hopefully puts everything together so you can talk to yourself
- ufo 3y ago
  Typically you'd have some assembly functions in a separate file. Compile to an object file and link it, as you'd do with a separate C file. If you want to insert assembly snippets inside a C function, many C compilers have an inline assembly feature. For simple snippets, the C compiler can figure it out. For more complex things, you there are ways to tell the C compilers which registers you're using, so it does not step on your toes.
  - cloogshicer 3y ago
    Thank you for explaining!
- eschneider 3y ago
  Assembly can either be inline with the C code or in a separate file. It's just source. Of course, if you really want to have fun with breaking abstractions, check out linker script files. That's for when you absolutely need different bits of code and data in specific parts of the application. Fun!

Veserv 3y ago

No, reversible abstractions are just one kind of abstraction. For instance, a machine code sequence to a linear sequence of assembly instructions is a reversible abstraction. Not every machine code sequence is expressible as a linear sequence of assembly instructions, but every linear sequence of assembly instructions has a trivial correspondence to a machine code sequence. However, consider the jump to a C-like language. The key abstraction provided there is the abstraction of infinite local variables. The compiler manages this through a stack, register allocation, and stack spilling to provide the abstraction and consumes your ability to control the registers directly to provide this abstraction. To interface at both levels simultaneously requires the leakage of the implementation details of the abstraction and careful interaction. What you can do easily is what I call a separable abstraction, a abstraction that can be restricted to just the places it is needed/removed where unneeded. In certain cases in C code you need to do some specific assembly instruction, sequence, or even function. This can be easily done by writing a assembly function that interfaces with the C code via the C ABI. What is happening there is that the C code defines a interface allowing you to drop down or even exit the abstraction hierarchy for the duration of that function. The ease of doing so makes C highly separable and is part of the reason why it is so easy to call out to C, but you hardly ever see anybody calling out to say Java or Haskell. Of course, that is just one of the many properties of abstractions that can make them easier to use, simpler, and more robust.

jerf 3y ago

Abstractions work by restricting the domain of what you can do, then building on those restrictions. For example, raw hardware can jump anywhere, but structured programming constrains you to jump only to certain locations in order to implement if, for, functions, etc. It is precisely those restrictions that bring the benefits of structured programming; if you still frequently dipped into jumping around directly structured programming would fail to provide the guarantees it is supposed to provide. CRUD frameworks provide their power by restricting you to CRUD operations, then building on that. Immutable data is accomplished by forbidding you from updating values even though the hardware will happily do it. And so on. Escape hatches under the abstractions are generally there precisely to break the abstractions, and break them they do. Abstractions necessarily involve being irreversible, or, to forestall a tedious discussion of the definition of "irreversible", necessarily involve making it an uphill journey to violate and go under the abstraction. There's no way around it. Careful thought can make using an escape hatch less pain than it might otherwise be (such as the ORM that makes it virtually impossible to use SQL by successfully hiding everything about the SQL tables from you so you're basically typing table and column names by dead reckoning), but that's all that can be done. One thing to do about this is that just as in the past few years the programming community has started to grapple with the fact that libraries aren't free but come with a certain cost that really adds up once you're pulling in a few thousand libraries for a framework's "hello world", abstractions that look really useful but whose restrictions don't match your needs need to be looked at a lot more closely. I had something like that happen to me just this week. I needed a simple byte ring buffer. I looked in my language's repos for an existing one. I found them. But they were all super complicated, offering tons of features I didn't need, like being a writethrough buffer (which involved taking restrictions I didn't want), or where the simple task of trying to understand the API was quite literally on par with implementing one myself. So I just wrote the simple thing. (Aiding this decision is that broadly speaking if this buffer does fail or have a bug it's not terribly consequential, in my situation it's only for logging output and only effectively at a very high DEBUG level.) It wasn't worth the restrictions to build up stuff I didn't even want.

cloogshicer 3y ago
> It is precisely those restrictions that bring the benefits [...] Wouldn't it be possible to say "ok, I'll take those restrictions as long as they benefit me, but once I notice that they no longer do, I'll break them and drop down to the lower layer. But only for those parts that actually require it"? > Abstractions necessarily involve being irreversible, or, to forestall a tedious discussion of the definition of "irreversible", necessarily involve making it an uphill journey to violate and go under the abstraction. Why? Not being snarky, I'm genuinely trying to understand this better.

samatman 3y ago

> Here's an example: When you use a compiler, you can work at a higher layer of abstraction (the higher-level language). But, this means you're now locked into that layer of abstraction. By that I mean, you can no longer work at the lower layer (assembly), even if you wanted to. Native-code compilers commonly allow emitting assembly directly, but now your source code isn't portable between CPUs. Many interpreted languages, even most, allow FFI code to be imported, modifying the runtime accordingly, but now your program isn't portable between implementations of that language, and you have to be careful to make sure the behavior you've introduced doesn't mess with other parts of the system in unexpected ways. Generalizing, it's often possible to drill down beneath the abstraction layer, but there's often an inherent price to be paid, whether it be taking pains to preserve the invariants of the abstraction, losing some of the benefits of it, or both. There are better and worse versions of this layer, I would point to Lua as a language which is explicitly designed to cross the C/Lua boundary in both directions, and which did a good job of it. But nothing can change the fact that pure-Lua code simply won't segfault, but bring in userdata and it very easily can; the problems posed are inherent.

wvenable 3y ago

Most ORMs give a way to integrate nicely with SQL if you need to reach down to that layer and still use the rest of the ORM features. There is no silver bullet; everything is a trade off. Almost all of the time, the trade off is entirely worth it even if that gets you locked into that solution.

cloogshicer 3y ago
> Most ORMs give a way to integrate nicely with SQL if you need to reach down to that layer and still use the rest of the ORM features. Agreed, that's a good thing, in my experience. > Almost all of the time, the trade off is entirely worth it even if that gets you locked into that solution. I wish this would match my experience.
- wvenable 3y ago
  What's funny is that ORMs giving you directly access to SQL is a leaky part of the abstraction but in this case that's good! I think being locked into a some abstraction is so common place that you don't even consider it being a thing until you have a problem with it. Look at your examples: compilers, libraries, frameworks, etc. As an example, is anyone truly upset that coding in Python locks you into the Python ecosystem? Yet, I've used libraries/frameworks that didn't win the war of popularity and I'm still unfortunately committed. I think there's a bias at play in how we look at these things.

yen223 3y ago

There's a well-written article by Bret Victor on climbing the ladder of abstraction. It makes the same argument you made, in that climbing "down" the ladder is just as important as going "up" https://worrydream.com/LadderOfAbstraction/ https://worrydream.com/LadderOfAbstraction/

cloogshicer 3y ago
Thank you for posting, I love that article. Bret Victor is a genius in my opinion and his writings and talks have inspired many of the thoughts I've written above.

lmm 3y ago

The best paradigm for understanding abstractions is not the theory-and-model style (which requires hiding details irreversibly), but the equivalence style. A good abstraction is e.g. summing a list whose elements are a monoid - summing the list is equivalent to adding up all the elements in a loop. Crucially, this doesn't require you to "forget" the specific type of element that your list has - a bad version of this library would say that your list elements have to be subtypes of some "number" type and the sum of your list came back as a "number", permanently destroying the details of the specific type that it actually is. But with the monoid model your "sum" is still whatever complex type you wanted it to be - you've just summed it up in the way appropriate to that type.

cloogshicer 3y ago
Interesting! Can you point me to some further reading or resources on these different paradigms?

saghm 3y ago

> My hunch is that this constraint should be that abstractions must be reversible. > Here's an example: When you use a compiler, you can work at a higher layer of abstraction (the higher-level language). But, this means you're now locked into that layer of abstraction. By that I mean, you can no longer work at the lower layer (assembly), even if you wanted to. You could in theory of course modify the compiler output after it's been generated, but then you'd have to somehow manually keep that work in sync whenever you want to re-generate. Using an abstraction kinda locks you into that layer. Just to make sure I understand, you're proposing a constraint that would rule out every compiler in existence today? I feel like overall I think compilers have worked out well, but if I'm not misunderstanding and this is how you actually feel, I guess I at least should comment your audacity, because I don't think I'd be willing to seriously propose something that radical.

yogorenapan 3y ago
A lot of languages allow in-line assembly
- saghm 3y ago
  I'm repeating the claim that the parent comment made; the assertion that compilers lock you into a level of abstraction is theirs, not mine.
cloogshicer 3y ago
Yes, you understood correctly. What I'm saying is extremely radical and would require rethinking and rebuilding practically everything we have.
- saghm 3y ago
  Thanks for the clarification! I'm probably too far into the orthodoxy to seriously consider something like this, but I admire your willingness to question things that most of us probably take for granted.

__s 3y ago

I think this is a good way to frame abstraction vs macro

dhdjksosja 3y ago

Babel towers of macro edsls, aka learn lisp. https://github.com/combinatorylogic/mbase https://github.com/combinatorylogic/mbase

drewm1980 3y ago

You can probably find an IDE plugin that inlines the assembly for a c function. Most ids can show the assembly side by side with your c code so it wouldn't be that much of a step. To fulfill your vision you would also need a decompiler (and an inliner) to convert a block of assembly back into C if a corresponding C routine exists.

samsquire 3y ago

This is something I think a lot about. I spend a lot of time trying to think of something that composes. Monads are one answer. I think we need advanced term rewriting systems that also optimize and equivalise. I really enjoy Joel on Software blog posts from this era.