7 ms·

Thinking About Glue Code

92 points by gHeadphone 5y ago

ipnon 5y ago
Does functional programming eliminate glue code? Is the sidewinder motion of the Turing machine the font of quadratic coding jobs?
- dhosek 5y ago
  Nope, The author's view of glue code is that most of it is to make the data output by function A a usable input for function B. This varies from the trivial (I'm moving an integer from A to B. Piece of cake as long as A & B both have the same sized integer with the same endianness and we've defined what to do if A is outputting a 64-bit value while B is inputting a 32-bit value and A sends something that doesn't fit in 32 bits) to the nightmarish (a large, poorly documented binary blob with inconsistent protocols) to lots of stuff in-between. Programming is easy. Data is hard.
  - cratermoon 5y ago
    > as long as A & B both have the same sized integer with the same endianness and we've defined what to do if A is outputting a 64-bit value while B is inputting a 32-bit value and A sends something that doesn't fit in 32 bits Well put. I still remember the time I helped a junior colleague debug an error (in C) simply by suggesting he look at the size of the types he was computing. Lightbulbs went off.
- xedrac 5y ago
  In Haskell, monads are all about composition of types. While the glue code doesn't go away, it stays neatly in the bind operator's definition for a given monad, which makes the rest of the code compose very easily. So no, it doesn't eliminate it, but I think it makes it a lot more manageable.
- cerved 5y ago
  No but it tends to enforce patterns that makes it a lot more manageable.
Nevermark 5y ago
I also view glue code as going up with the square (n^2) of the number of connections (n) grows between two conceptually connected but implementation-disconnected systems. But that is hardly the worst of it. The two system model generalizes to much worse complexity (n^m) relative to the number of connections (n) that involve tight linkage of a multiple number of systems (m >= 2). In practice, we all avoid m>2 wherever we can, and simply slog along less and less efficiently with m=2 as we try to add new value (or maintain value in the presence of necessary changes). So the general law is mostly apparent in the broad absence of conceptually simple m>2 solutions, as apposed to their existence as quagmires. Simple thought experiment to illustrate m>2 solutions: “Captain! If we just reprogram the nanobot solar system explorer cannon to spread the bots on a tight bezier trajectory matching our enemies detected firing pattern, adjust their transmission frequencies to pass through Romulan dark hull steel and resonate with their traditional diptholamite crystal bathroom fixtures, we can apply a combination frequency comb filter and inverse fractal hash to completely uncloak their vessels in real time!” “But Chief Engineer, there is but one problem!! While conceptually simple, your elegant solution will take one hundred years to implement! None of those systems were designed together!” Kaboom. Hissss… Silence I call the problem of rapidly integrating foreign systems into conceptually simple novel solutions “The Star Trek Rapid Integration Problem.” Apparently this problem has been solved by the Federation of Planets in our future.
- ipnon 5y ago
  It seems like there most be some group theory formulation of this problem, where each "gluing" can be seen as different symmetries of a software system.
- einpoklum 5y ago
  In the future, Star Trek equips its starships with casks of Phlebotium, which is then applied generously to both systems in such situations, allowing them to interact seamlessly: https://tvtropes.org/pmwiki/pmwiki.php/Main/AppliedPhlebotinum https://tvtropes.org/pmwiki/pmwiki.php/Main/AppliedPhlebotin...
armchairhacker 5y ago
> Glue isn’t some kind of computational waste; it’s what holds our systems together. Glue development is software development. My take is: glue code is just “obvious” code, as opposed to the less obvious code. The problem is that it’s not really obvious. That’s how you get “write-only” code. What’s obvious to one person isn’t to another, or even to that same person later on. This is why Java, a very verbose (“full of glue”) language, is so popular. People say “an ideal programming language would have no glue code, it would all be inferred by the compiler”. And it’s true that a good language and libraries can remove a lot of trivial boilerplate. But not all of it, because some of that “boilerplate” isn’t so trivial. Like, casting floats to integers. The cast is glue code. C and other languages truncate the float, so it gets floored if positive and ceiled if negative. But do you want that, or do you want to round the float or ceil it always? Or is casting the float to an integer an indicator that you’re using the wrong variable? —- I think in the future, we will create AI to write a lot of ”glue” code fast, like GitHub copilot. Also, proven-equivalent (or heuristically 99% predicted equivalent) large-scale refactors. But even in the ideal scenario where we eliminate low-level glue code, we’ll just consider higher-level code “glue code”. Because, all code is glue code, to convert “what I want” to “what computer does”.
Wowfunhappy 5y ago
The last few sentences of the article capture how I feel about the whole thing: > Programming is ultimately about gluing things together, whether they’re microservices or programming libraries. Glue isn’t some kind of computational waste; it’s what holds our systems together. Glue development is software development. Unless you're programming in assembly, aren't you always essentially writing glue code? Your language has some number of predefined assembly instructions which you can call in different ways and connect to fulfill your needs.
- dnautics 5y ago
  I think there's a real meaningful difference between glue code and business logic, even if your business logic is created by encapsulating low level operations that are hidden by your PL, the total reductiviness to "glue" is not helpful
  - anton-107 5y ago
    Someone in this thread gave an example of casting floats to integers being glue code. But how exactly you want to cast - using round, floor or ceil is a business logic. Where is the meaningful difference in this example?
    - dnautics 5y ago
      I'm not sure about that. But business logic is what you have to do de Novo to make your product. I work for an invoice processing company. Connecting to a/many Saas is glue code (this is most of our codebase). The code that decides the rules for how an invoice is "approved" by reviewers is business logic.
    - ratww 5y ago
      It depends on the intention. If you're intentionally rounding a number because it's in some algorithm, or maybe your tax law, or to give discounts to customers, then it's not glue code. Using a cast to round down will obscure the intention, but I'd argue it's not really glue code. However, if you cast the float because you have two systems and the second one only accepts integers, then you are using glue code to make the two incompatible things work together. The lines a bit blurry sometimes. But the distinction is still helpful, because less code can potentially (but not necessarily) mean less bugs, less time needed to understand the system, less cost, more performance, etc. And glue code is code that you can potentially (but not necessarily) remove.
cratermoon 5y ago
> the real problems stem more from schemas than data formats Oh, how the turntables turn. Ontologies are hard, and always fail in some way[1]. There are strategies like entity resolution[2] that attempt to take items from dataset A and items from dataset B and figure out if they refer to the same thing, but they are expensive and time-consuming and even when they work, only approximate. Anyone has ever done a data migration project of any complexity knows that there must be provisions for dealing with data quality issues, especially if effort includes reconciling data from multiple sources. 1 https://oc.ac.ge/file.php/16/_1_Shirky_2005_Ontology_is_Overrated.pdf https://oc.ac.ge/file.php/16/_1_Shirky_2005_Ontology_is_Over... 2 https://doi.org/10.1145/3418896 https://doi.org/10.1145/3418896
- skohan 5y ago
  > There are strategies like entity resolution[2] that attempt to take items from dataset A and items from dataset B and figure out if they refer to the same thing The more code I have written, the more convinced I become that "general" solutions almost never pay for themselves when you compare the implementation and/or performance complexity required to make them work. It's almost always better to "just write the damn code" and solve the exact problem you actually have.
  - cratermoon 5y ago
    If every time you do it is a one-off, yes. If your business is essentially nothing but aggregating datasets (which may themselves be aggregates of still more datasets), then by the third time you've solved the exact problem, you're going to see commonalities. You abstract those out so that the fourth, fifth, and sixth problems don't have to re-write that part. After a while, you've built out enough abstractions that you're programming the migration process itself, not each migration. Add in the fact that your business depends on doing it at scale, ingesting live data as it is generated, and developing an entity resolution system is almost a no-brainer.
  - arthur_sav 5y ago
    Yes and no. They don't pay if you try to generilize before you have a problem. If you have a frequent use case that follows the same patterns, then you probably can benefit from an abstraction layer. But trying to build an abstraction layer before even having a use case then yes, it usually doesn't pay off.
gitgud 5y ago
Programs become messy, abstractions become leaky, glue code helps fix these problems
SeriousM 5y ago
One thing that evolved over the years is swagger. It is a defined way to document APIs - the endpoint, return type, returning data structure, arguments, behavior, etc. Microsoft took this as a way to connect different APIs in their logic app (think of workflow definition like an advanced version of ifttt, n8n, ...). It certainly doesn't remove the need for glue code but it reduces the amount of it! On the flip side you have up the control of the program flow by using their framework that handles all the connections between APIs.
- skohan 5y ago
  All swagger really is is a tool for formalization of HTTP/JSON interfaces. And that's great! Formalized interfaces are really the best way to reduce the need for glue-code and post-processing, since the parties on both sides can agree on the exact target their shooting for.
Kinrany 5y ago
The simplest solution to having N data types and N^2 conversions between them is to have a canonical data type and 2N conversions between that one type and the rest. The real problem is that all that glue code is proprietary, and companies cannot cooperate on a large scale to pool their resources and share their software.
tomxor 5y ago
> as systems become more all-encompassing, the need to integrate with different systems increases. This is the underlying problem: the perception that programs need to become more all encompassing. It's the ultimate feature creep.
- justsomeuser 5y ago
  I thought that line was a mistake too. Your software is just a tiny amount of code (relative to the whole system) making up the control plane between all the pieces. I think the author probably meant that to personally write less code you need to leverage more external code (libraries and network API's). Write less code -> Leverage other peoples code -> More integration with different systems
Chris_Newton 5y ago
I’m not sure I buy the N² argument being made here. If we’re talking about communications between components within a software system then yes, for N components, there are roughly N² possible connections between them. But isn’t managing that complexity so we don’t connect everything to everything Software Design 101? If you design your system using structure like layers and pipelines then you’re effectively reducing that O(N²) connectedness to something more like O(N). If we’re talking about having multiple choices for how to implement each component, such as when we have several libraries available plus the option of writing something from scratch, then given N choices for the first component and M choices for the second, there are NM possible connections we might have to implement. However, we probably only need one of those connections at any given time. If we need to modify or replace either implementation later, the most important thing for flexibility and future-proofing is that the data models and communications protocols are well specified, ideally following established standards that are likely to be followed by the other alternatives as well. Again, though, separating interface from implementation is Software Design 101. So it seems to me that if either of those interpretations was intended, we’re talking about a problem that has largely been solved for a long time. Perhaps part of the reason it doesn’t always feel that way today is that there is so much emphasis on doing everything in “agile” fashion now. Sometimes that agility comes at the expense of establishing a well-considered software architecture and long-lived standards for interoperability. The “glue code” in software systems is where we pay any interest on those forms of technical debt, so while there will always be some need to marshal data around a large system with many components, there will be much, much more code needed if those forms of complexity aren’t well managed.
- keithnz 5y ago
  came in here to say the much the same. I also see much more that we get "pre glued" things that we then glue a few more things to. After 40ish years of coding, I think the amount of glue we use is about the same. It goes up and down a bit but there seems to be a natural limit to the complexity we can deal with.
  - Chris_Newton 5y ago
    I also see much more that we get "pre glued" things that we then glue a few more things to. Do you mean other trends like increasing use of frameworks, scaffolding tools and “convention over configuration”? If so then I agree. One of the trade-offs being made there is immediate convenience against long-term flexibility. You get more functionality out-of-the-box with these kinds of tools, but the price is that they often don’t compose well.
mark_l_watson 5y ago
About forty years ago, I did a weekend consulting gig at a lab in San Diego where they had a massive N^2 connection problem of data processing code vs. data formats. I wrote a FORTRAN program to enumerate all combinations and generate a lot of very baroque code. My customer was not happy with the solution until I showed them how simple it was to modify the generating code and regenerate the glue code. Then they liked the approach. If I had written this a few years later, I would have used Lisp, and things would have been even simpler.
jesseschalken 5y ago
The industry already has solutions for this in the form of schema and interface definitions that generate glue code for you: - gRPC, protocol bufffers - OpenAPI, Swagger, JSON Schema - Apache Thrift - Apache Avro - Cap'n Proto - IDL, Microsoft IDL, Web IDL - etc

92 points by gHeadphone 5y ago

Does functional programming eliminate glue code? Is the sidewinder motion of the Turing machine the font of quadratic coding jobs?

dhosek 5y ago
Nope, The author's view of glue code is that most of it is to make the data output by function A a usable input for function B. This varies from the trivial (I'm moving an integer from A to B. Piece of cake as long as A & B both have the same sized integer with the same endianness and we've defined what to do if A is outputting a 64-bit value while B is inputting a 32-bit value and A sends something that doesn't fit in 32 bits) to the nightmarish (a large, poorly documented binary blob with inconsistent protocols) to lots of stuff in-between. Programming is easy. Data is hard.
- cratermoon 5y ago
  > as long as A & B both have the same sized integer with the same endianness and we've defined what to do if A is outputting a 64-bit value while B is inputting a 32-bit value and A sends something that doesn't fit in 32 bits Well put. I still remember the time I helped a junior colleague debug an error (in C) simply by suggesting he look at the size of the types he was computing. Lightbulbs went off.
xedrac 5y ago
In Haskell, monads are all about composition of types. While the glue code doesn't go away, it stays neatly in the bind operator's definition for a given monad, which makes the rest of the code compose very easily. So no, it doesn't eliminate it, but I think it makes it a lot more manageable.
cerved 5y ago
No but it tends to enforce patterns that makes it a lot more manageable.

Nevermark 5y ago

I also view glue code as going up with the square (n^2) of the number of connections (n) grows between two conceptually connected but implementation-disconnected systems. But that is hardly the worst of it. The two system model generalizes to much worse complexity (n^m) relative to the number of connections (n) that involve tight linkage of a multiple number of systems (m >= 2). In practice, we all avoid m>2 wherever we can, and simply slog along less and less efficiently with m=2 as we try to add new value (or maintain value in the presence of necessary changes). So the general law is mostly apparent in the broad absence of conceptually simple m>2 solutions, as apposed to their existence as quagmires. Simple thought experiment to illustrate m>2 solutions: “Captain! If we just reprogram the nanobot solar system explorer cannon to spread the bots on a tight bezier trajectory matching our enemies detected firing pattern, adjust their transmission frequencies to pass through Romulan dark hull steel and resonate with their traditional diptholamite crystal bathroom fixtures, we can apply a combination frequency comb filter and inverse fractal hash to completely uncloak their vessels in real time!” “But Chief Engineer, there is but one problem!! While conceptually simple, your elegant solution will take one hundred years to implement! None of those systems were designed together!” Kaboom. Hissss… Silence I call the problem of rapidly integrating foreign systems into conceptually simple novel solutions “The Star Trek Rapid Integration Problem.” Apparently this problem has been solved by the Federation of Planets in our future.

ipnon 5y ago
It seems like there most be some group theory formulation of this problem, where each "gluing" can be seen as different symmetries of a software system.
einpoklum 5y ago
In the future, Star Trek equips its starships with casks of Phlebotium, which is then applied generously to both systems in such situations, allowing them to interact seamlessly: https://tvtropes.org/pmwiki/pmwiki.php/Main/AppliedPhlebotinum https://tvtropes.org/pmwiki/pmwiki.php/Main/AppliedPhlebotin...

armchairhacker 5y ago

> Glue isn’t some kind of computational waste; it’s what holds our systems together. Glue development is software development. My take is: glue code is just “obvious” code, as opposed to the less obvious code. The problem is that it’s not really obvious. That’s how you get “write-only” code. What’s obvious to one person isn’t to another, or even to that same person later on. This is why Java, a very verbose (“full of glue”) language, is so popular. People say “an ideal programming language would have no glue code, it would all be inferred by the compiler”. And it’s true that a good language and libraries can remove a lot of trivial boilerplate. But not all of it, because some of that “boilerplate” isn’t so trivial. Like, casting floats to integers. The cast is glue code. C and other languages truncate the float, so it gets floored if positive and ceiled if negative. But do you want that, or do you want to round the float or ceil it always? Or is casting the float to an integer an indicator that you’re using the wrong variable? —- I think in the future, we will create AI to write a lot of ”glue” code fast, like GitHub copilot. Also, proven-equivalent (or heuristically 99% predicted equivalent) large-scale refactors. But even in the ideal scenario where we eliminate low-level glue code, we’ll just consider higher-level code “glue code”. Because, all code is glue code, to convert “what I want” to “what computer does”.

Wowfunhappy 5y ago

The last few sentences of the article capture how I feel about the whole thing: > Programming is ultimately about gluing things together, whether they’re microservices or programming libraries. Glue isn’t some kind of computational waste; it’s what holds our systems together. Glue development is software development. Unless you're programming in assembly, aren't you always essentially writing glue code? Your language has some number of predefined assembly instructions which you can call in different ways and connect to fulfill your needs.

dnautics 5y ago
I think there's a real meaningful difference between glue code and business logic, even if your business logic is created by encapsulating low level operations that are hidden by your PL, the total reductiviness to "glue" is not helpful
- anton-107 5y ago
  Someone in this thread gave an example of casting floats to integers being glue code. But how exactly you want to cast - using round, floor or ceil is a business logic. Where is the meaningful difference in this example?
  - dnautics 5y ago
    I'm not sure about that. But business logic is what you have to do de Novo to make your product. I work for an invoice processing company. Connecting to a/many Saas is glue code (this is most of our codebase). The code that decides the rules for how an invoice is "approved" by reviewers is business logic.
  - ratww 5y ago
    It depends on the intention. If you're intentionally rounding a number because it's in some algorithm, or maybe your tax law, or to give discounts to customers, then it's not glue code. Using a cast to round down will obscure the intention, but I'd argue it's not really glue code. However, if you cast the float because you have two systems and the second one only accepts integers, then you are using glue code to make the two incompatible things work together. The lines a bit blurry sometimes. But the distinction is still helpful, because less code can potentially (but not necessarily) mean less bugs, less time needed to understand the system, less cost, more performance, etc. And glue code is code that you can potentially (but not necessarily) remove.

cratermoon 5y ago

> the real problems stem more from schemas than data formats Oh, how the turntables turn. Ontologies are hard, and always fail in some way[1]. There are strategies like entity resolution[2] that attempt to take items from dataset A and items from dataset B and figure out if they refer to the same thing, but they are expensive and time-consuming and even when they work, only approximate. Anyone has ever done a data migration project of any complexity knows that there must be provisions for dealing with data quality issues, especially if effort includes reconciling data from multiple sources. 1 https://oc.ac.ge/file.php/16/_1_Shirky_2005_Ontology_is_Overrated.pdf https://oc.ac.ge/file.php/16/_1_Shirky_2005_Ontology_is_Over... 2 https://doi.org/10.1145/3418896 https://doi.org/10.1145/3418896

skohan 5y ago
> There are strategies like entity resolution[2] that attempt to take items from dataset A and items from dataset B and figure out if they refer to the same thing The more code I have written, the more convinced I become that "general" solutions almost never pay for themselves when you compare the implementation and/or performance complexity required to make them work. It's almost always better to "just write the damn code" and solve the exact problem you actually have.
- cratermoon 5y ago
  If every time you do it is a one-off, yes. If your business is essentially nothing but aggregating datasets (which may themselves be aggregates of still more datasets), then by the third time you've solved the exact problem, you're going to see commonalities. You abstract those out so that the fourth, fifth, and sixth problems don't have to re-write that part. After a while, you've built out enough abstractions that you're programming the migration process itself, not each migration. Add in the fact that your business depends on doing it at scale, ingesting live data as it is generated, and developing an entity resolution system is almost a no-brainer.
- arthur_sav 5y ago
  Yes and no. They don't pay if you try to generilize before you have a problem. If you have a frequent use case that follows the same patterns, then you probably can benefit from an abstraction layer. But trying to build an abstraction layer before even having a use case then yes, it usually doesn't pay off.

gitgud 5y ago

Programs become messy, abstractions become leaky, glue code helps fix these problems

SeriousM 5y ago

One thing that evolved over the years is swagger. It is a defined way to document APIs - the endpoint, return type, returning data structure, arguments, behavior, etc. Microsoft took this as a way to connect different APIs in their logic app (think of workflow definition like an advanced version of ifttt, n8n, ...). It certainly doesn't remove the need for glue code but it reduces the amount of it! On the flip side you have up the control of the program flow by using their framework that handles all the connections between APIs.

skohan 5y ago
All swagger really is is a tool for formalization of HTTP/JSON interfaces. And that's great! Formalized interfaces are really the best way to reduce the need for glue-code and post-processing, since the parties on both sides can agree on the exact target their shooting for.

Kinrany 5y ago

The simplest solution to having N data types and N^2 conversions between them is to have a canonical data type and 2N conversions between that one type and the rest. The real problem is that all that glue code is proprietary, and companies cannot cooperate on a large scale to pool their resources and share their software.

tomxor 5y ago

> as systems become more all-encompassing, the need to integrate with different systems increases. This is the underlying problem: the perception that programs need to become more all encompassing. It's the ultimate feature creep.

justsomeuser 5y ago
I thought that line was a mistake too. Your software is just a tiny amount of code (relative to the whole system) making up the control plane between all the pieces. I think the author probably meant that to personally write less code you need to leverage more external code (libraries and network API's). Write less code -> Leverage other peoples code -> More integration with different systems

Chris_Newton 5y ago

I’m not sure I buy the N² argument being made here. If we’re talking about communications between components within a software system then yes, for N components, there are roughly N² possible connections between them. But isn’t managing that complexity so we don’t connect everything to everything Software Design 101? If you design your system using structure like layers and pipelines then you’re effectively reducing that O(N²) connectedness to something more like O(N). If we’re talking about having multiple choices for how to implement each component, such as when we have several libraries available plus the option of writing something from scratch, then given N choices for the first component and M choices for the second, there are NM possible connections we might have to implement. However, we probably only need one of those connections at any given time. If we need to modify or replace either implementation later, the most important thing for flexibility and future-proofing is that the data models and communications protocols are well specified, ideally following established standards that are likely to be followed by the other alternatives as well. Again, though, separating interface from implementation is Software Design 101. So it seems to me that if either of those interpretations was intended, we’re talking about a problem that has largely been solved for a long time. Perhaps part of the reason it doesn’t always feel that way today is that there is so much emphasis on doing everything in “agile” fashion now. Sometimes that agility comes at the expense of establishing a well-considered software architecture and long-lived standards for interoperability. The “glue code” in software systems is where we pay any interest on those forms of technical debt, so while there will always be some need to marshal data around a large system with many components, there will be much, much more code needed if those forms of complexity aren’t well managed.

keithnz 5y ago
came in here to say the much the same. I also see much more that we get "pre glued" things that we then glue a few more things to. After 40ish years of coding, I think the amount of glue we use is about the same. It goes up and down a bit but there seems to be a natural limit to the complexity we can deal with.
- Chris_Newton 5y ago
  I also see much more that we get "pre glued" things that we then glue a few more things to. Do you mean other trends like increasing use of frameworks, scaffolding tools and “convention over configuration”? If so then I agree. One of the trade-offs being made there is immediate convenience against long-term flexibility. You get more functionality out-of-the-box with these kinds of tools, but the price is that they often don’t compose well.

mark_l_watson 5y ago

About forty years ago, I did a weekend consulting gig at a lab in San Diego where they had a massive N^2 connection problem of data processing code vs. data formats. I wrote a FORTRAN program to enumerate all combinations and generate a lot of very baroque code. My customer was not happy with the solution until I showed them how simple it was to modify the generating code and regenerate the glue code. Then they liked the approach. If I had written this a few years later, I would have used Lisp, and things would have been even simpler.

jesseschalken 5y ago

The industry already has solutions for this in the form of schema and interface definitions that generate glue code for you: - gRPC, protocol bufffers - OpenAPI, Swagger, JSON Schema - Apache Thrift - Apache Avro - Cap'n Proto - IDL, Microsoft IDL, Web IDL - etc