13 ms·

B-threads: programming in a way that allows for easier changes

167 points by sktrdie 7y ago

brianberns 7y ago
I was recently tasked with making some small incremental changes to a large code base that I was not familiar with. I was successful specifically because I could see the existing implementation and use it as a template for making the necessary changes. For example, if I had to add a widget to a form, I could copy the implementation of an existing, similar widget and then make the necessary changes to it. Similarly, I had to change the datatype of a field from string to integer all the way from the UI back to the database. This was fairly easy because I could make the change at one level, and then run the compiler to see what my change broke one level up. Fix, rinse, and repeat. If I had to do any of this without access to the existing code, it would've been 100x harder.
akkartik 7y ago
Interestingly, my article from a few years ago has a very similar motivation and initial figure. And it too ends up at append-only programming. But it does so in a very different way: http://akkartik.name/post/wart-layers http://akkartik.name/post/wart-layers (I still program all my side projects in this way.)
asperous 7y ago
Video game engines often have plugin systems like this that listen for hooks or events and can act on them. It does make extremely complex things a bit more simple. But the chart in graph 1 still applies, often when adding new functionality you realize you need to add or remove modularity and extensions.
- Chris2048 7y ago
  Eclipse has lots of plugins, but that turned out not so great as too many is known to cause issues.
daenz 7y ago
I'm going to make a conscious effort not to come off sounding like an asshole, but please excuse me if I slip up. I find many of the ideas in post to be fundamentally at odds with the direction that "good software development" should be travelling. The core of my feeling is best captured by the following quote from the post: >As a system grows in complexity we don’t necessarily care about how old b-threads have been written, hence we don’t care about maintaining them. This post is essentially formalizing the process of creating a Big Ball of Mud[0] that is so complex and convoluted that it is impossible to understand. The motivation for formalizing this process seems sane and with good intentions: to add functionality quickly to code you don't really understand. Normally, doing something like is considered cutting corners and incurring explicit technical debt, and must be used sparingly and responsibly. However, the process of "append-only development" is embracing the corner cutting and technical debt as a legitimate development process. I can't get on board with this. To be more specific, with an example (and maybe I am wrong in understanding the post, this would be the time to point that out to me), let's suppose you have a massive complex software system that was built over the years with this "append-only" style of development. One day you find a nasty bug in one of the lower layers, and to correct it, you have to change some functionality, which moves/removes some events that subsequent layers are depending on for their own functionality. Suddenly, you are faced with rewriting all of those layers in order to adapt to your bugfix. What you're left with is a nightmare of changes that disturb many layers of functionality, because they're all based on this append-only diffing concept: the next layer is dependent on the functionality of the previous layer. This is what programming APIs are for: to change functionality in lower layers with minimal influence subsequent layers. This post and process seems to be imagining a world without APIs. 0. http://www.laputan.org/mud/ http://www.laputan.org/mud/
- loopbit 7y ago
  I completely agree with you. The thing is, I might be able to see this "working" in very simple and limited processes[0]... But going back to your example of the nasty bug in complex software, I believe the post's idea is that you shouldn't fix the bug in the layer it's happening, but, instead, write a new b-thread that corrects that behavior. Which might sound nice in theory but I feel it's much more likely that you won't be able to fix it there (the info you need might be lost in a previous b-thread) or become a piece of code so complex that negates any supposedly benefit that this system had in the first place. [0] It might be just me and completely off-topic, but this reminds me a bit of rule-based expert systems, where the rules get activated by certain conditions and produce effects (that can activate other rules). The idea was always that you could model very complex (and emerging) behaviors based on very simple, human readable rules. The thing is that you could definitely add/tweak/remove the rules as needed.
munificent 7y ago
I feel like Babbage when he said, "On two occasions I have been asked, 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question." Maybe I'm missing some key insight, but this process seems like madness to me. > Then we wouldn’t have to read and understand where to squeeze our changes. We’d simply add stuff based on the new requirements and things would somehow magically work. But, at some point, someone has to understand if those requirements clash with the old ones that are currently implemented. The hot/cold water example is, I think, telling. At no point in the article does the author actually state what the requirements (new or old) are. You have some program that, whenever the water level gets low, adds three hot waters. I guess that was some original requirement. Then another b-thread is added that also adds three cold waters. Now whenever the water gets low, twice as much total water is added. Presumably that's OK? When the second b-thread was added, was the requirement to fill the container with cold water instead of hot, or was it to fill the container with twice as much lukewarm water? Finally, the third b-thread is added that interacts with the two previous ones to cause the hot and cold water adds to be interleaved. What requirement does that meet? The stated purpose of all of this is that you don't need to understand the original program to modify it, but the entire purpose of this third thread is to interact with the previous two. You have to know there are already two b-threads going, which events they listen to, and what they yield in order to create the third thread that blocks them. So I don't see how this is any better than needing to read the code in a normal program. I feel like I'm taking crazy pills.
- Twisol 7y ago
  On top of the requirements engineering that's left implicit here, there's still the matter of understanding the event trace that you're supposed to be modifying. If every patch is a transformer from traces to traces, then it stands to reason that you must understand the event trace at least as well as the program you would otherwise be modifying. And program traces are an inherently dynamic thing -- how do you gain an understanding about the potentially very complicated, interwoven traces you're developing with if you can only see one trace per execution? A great deal of the problem of concurrent programming is just coming to grips with these traces, and decomposing them in a comprehensible way within the structure of your codebase.
hzhou321 7y ago
It starts with an interesting description of problem, but I felt that it took a sharp slope and reached into a bizzare conclusion.
- hinkley 7y ago
  We are still wrestling mightily with the notion that solving a problem badly can be worse than not solving it at all.
kazinator 7y ago
Yikes; this nonsense will turn into an incomprehensible ball of hair in the implementation of anything real. You don't know if anything will work or continue to work due to being randomly blocked or otherwise interfered with by code that either already exists or will be added tomorrow. Basically this is just flow-based programming (FBP) in a new form. https://en.wikipedia.org/wiki/Flow-based_programming https://en.wikipedia.org/wiki/Flow-based_programming Every new generation of coders reaches a puberty characterized by uncontrollable excitement with multiple threads of control doing little things independently.
- crimsonalucard 7y ago
  Flow based programming is basically functional programming. It feels like a redefinition of a style of programming that's already heavily established.
  - yellowapple 7y ago
    I'd say it's more of a potential application of functional programming - that is, you could* certainly use functional programming techniques/languages to implement flow-based programming, but functional languages are not automatically flow languages by merely existing. * But this is not strictly necessary. See also: Unix pipes, which fit the definition of flow-based programming while being possible - and in fact frequently done - with very-much-not-functional languages both implementing the individual steps and connecting them together.
    - crimsonalucard 7y ago
      It looks like flow based programming is exactly the same thing as functional composition. Can you illustrate to me how these two concepts are disparate? I ask because if they are one in the same, then because the only way to link functions in functional programming is through function composition than functional programming is flow based programming.
      - kazinator 7y ago
CodeSucks 7y ago
You might like FlowNote, as it implements many of these ideas at a high level coding language. https://emblem21-opensource.github.io/flownote/docs/06-flow.html https://emblem21-opensource.github.io/flownote/docs/06-flow....
hacker_9 7y ago
Append style programming is very interesting, especially when thinking of modelling future languages closer to natural language. This foray though seems to end up generating more problems than it's worth. For one, its not clear how this aids easier development - hasn't it just shifted understanding of a complex program to understanding a complex overlapping sequence of b threads instead? Secondly, and I think the nail in the coffin, the compiler would have to be brilliant at optimisation to make the final program performant. The idea of 'show ads' later being disabled by another overlapping b thread - does this mean the network request is never performed? Or is it performed and the result thrown away, wasting resources. How would an 'offline mode' requirement be appended as a b thread in such a case?
- codebje 7y ago
  The events are a scheduling mechanism, not the work done. Performance probably relates more to the ratio of time spent selecting the next event to time spent executing code between yields than anything else: a thread waiting on an event joins a queue, a thread requesting an event joins the same queue but also flags it as ready, and a thread blocking an event flags the queue as not ready. When a queue is selected, it's drained completely before a new queue is selected. The run cost of the scheduler would depend mostly on the structure used to maintain the ready set, but a basic double ended linked list would provide linear time operations for ready/blocked management. Whole program compilation could provide lookups or perfect hashes for event queueing, or partial compilation could rely on imperfect hashing. Optimising during compilation might work best by trying to fuse events such that the scheduler is invoked less often. Things that are not apparent from a lightweight article like this include state transfer (how do I know details of the card inserted?) and the likelihood that I'd keep the code to each b-thread and hack on those to make new ones...
deleted 7y ago
[deleted]
yowlingcat 7y ago
Nothing about this approach seems easier to reason about nor maintain over the long term. Transactional integrity, migrating data structures, and modifying business logic/control flow are key building blocks required for many production applications, certainly line of business ones. I'm reminded of the golden hammer fascination with event sourcing, and I continue to see event sourcing applied inappropriately. Is it useful for some specialized applications? Yes, and it's a great tool there! But should it be the default tool you reach for to solve most problems over a relational DB and your most trusted, high level, stable programming language with a mature library ecosystem? No to that question, and no to this.
msteffen 7y ago
In a prior team, we accomplished the goal of "a newcomer should be able to figure out the history of the code they're looking at, and thereby understand why it is this way" in a simpler, lower-tech way: 1. One commit per PR (so the commit history wasn't polluted with rough drafts and debug logs and such) 2. Every PR links to a bug 3. Design discussions happen in bugs Then when you see some weird code, you can "blame" back to the commit, and from there look at the bug to see why the commit was needed. We also really encouraged people to separate refactoring from new features (refactor first, in a descriptive PR, then add the feature), so that you avoided the problem of seeing a complex refactoring in a PR called "add foo" I agree with other commenters that encoding the design history of your project in its operation doesn't seem helpful, and in my opinion that's because it's solving the wrong problem. Finding the places in a complicated project that need to change is only hard because knowing, to a fairly complete degree, how the system already works is hard, and knowing how a system already works is mostly the product of understanding the problem that it solves deeply. Experienced engineers on a project usually have no problem figuring out which parts of a system need to be changed, because they already know the system well, and they don't know they system well by virtue of having all the code memorized, which would be impossible, but by having participated in all of the design discussions. Therefore, the problem that needs to be solved is making the learning process as fast and easy as possible—thus the strict process around documenting changes and their motivation. Allowing people to quickly discover the existing code's motivation gives them the information that will actually need be committed to memory.
- 0xdeadbeefbabe 7y ago
  Is all your code in one repo? Just curious, because I see this learning problem made worse by multiple repos with multiple PRs.
  - msteffen 7y ago
    It was all in one repo on that team, yes. Offhand, it seems like linking all PRs in all repos back to the bug, and linking the bug to all of the disparate PRs might help. A version (possibly) of the "many PRs" problem that we sort of had was that it often took many PRs to close a bug. Our goal wasn't to minimize the number of PRs, and in fact we encouraged small PRs. Rather, the goal was to get engineers past the commit and PR to the underlying motivation and design.
jonstaab 7y ago
Just to add some contrast to the mostly negative comments here (which have merit), this is interesting to me, not because it aims to hide the past, but because it makes time a first-class concept in the software development model, which most programming styles fail to do (e.g. most RDBMS frameworks add migrations on as an afterthought). I like this, and hope something like it catches on. The problems with this approach seem solvable to me, albeit with more experimental magic that could explode: - The resulting big ball of mud (and subsequent performance problems of a long pipeline of relations) could be compiled away, resulting in a single artifact. That is, you'd develop in append-only style, but when you "commit" your release, you'd end up with a single, optimized artifact for deployment, which would also be readable. This seems really nice to me, since your changes, while being based on the behavior of the system, would create a diff in the implementation of the system, potentially reaching way back into upstream events (in the example, the behaviors that block hot water and substitute cold water would just completely eliminate the first pane and simplify to adding cold water). This would let you see your changes from multiple perspectives. This approach also seems really friendly to fuzz-testing, which would give you a third look into the behavior of the system, and you could write tests based on the final state of the system after a number of given events. - Migrating data structures actually seems easier to me for an event sourced approach, since you'd just re-project your domain models based on the new flow of events. b-threads would allow you to re-compile your event stream just like you re-compiled the source artifact (having parameterized events remains a problem, since your historical data could end up being incomplete and invalid based on new policy, you'd have to adopt a permissive schema to keep stuff that validation would otherwise reject). I'll agree that b-trees don't really solve anything, but they do bring up some interesting questions that I think are worth asking. Datomic and Darklang I think are much more practical, and seem to dabble in the same sort of areas.
- hacker_9 7y ago
  >> when you "commit" your release, you'd end up with a single, optimized artifact for deployment I agree this would be nice, but think of this problem more deeply and you quickly run into major issues. The biggest one being that your compiler now has a large scale optimisation step to do completely automatically, better than a human. In order to make this work, you'd effectively have to build everything out of composable components, which you then can switch on/off with some sort of layering system. The problem here is how do you come to a decision on what those composable units look like? You don't know what you don't know - i.e. what those future layers you append are going to need to turn off or on. And coming back to optimisation - once you know the features you are combining together in an algorithm, only then do you know the data structures that fit the problem. Optimisation isn't a forward process when you turn parts off and end up with an efficient system. Optimisation requires a high level view, a low level view, an understanding of context, of memory requirements, of language capabilities, even leaps of logic. For the last part when I mean is that whilst a compiler may be able to reorganize something in the form a*b=c (and even then knowing when to do this is a book on it's own), it can't know that if I, for example, disable a feature that adds nested nodes to a tree structure, that it could rewrite functions to no longer require recursion and instead treat them like serial lists. You could rewrite the methods yourself and add them to your new layer, but you also will inevitably end up not fitting the previous composable structure. I think the deeper problem is related more to the tying of syntax to structure, as structure is ultimately your application's performance profile. Untying these two though could lead to a way forward for the append style paradigm.
crimsonalucard 7y ago
The solution to incremental changes is to develop your app in a way where every unit of computation is compose-able under a strict rule. Meaning I can write a universal function compose(A,B) = C and that function can compose any primitive. Then you build complicated logic just like how you build a wall from a set of bricks... form complicated logic as a composition of more primitive logic all the way down. If your program is constructed this way at every layer of logic, not only will your program be amenable to append only styles of programming. But it will be amenable to decomposition. Often you find that your primitives are too big and you need to split your primitives... well if your primitives are itself made out of the compositions of lower level primitives then breaking apart that function is trivial. There is only one primitive in all of programming that follows this composition rule. Procedures cannot compose, Objects cannot compose. What's left?
- yellowapple 7y ago
  > if your primitives are itself made out of the compositions of lower level primitives I'd argue those ain't "primitives" in that case; what differentiates a "primitive" from some other kind of computation is that it can't be broken down further. > Procedures cannot compose Depends on how you define "procedure". In the sense used in procedural programming, you certainly can compose them (i.e. by jumping into a subroutine that in turn jumps into other subroutines, pushing and popping things to/from the stack in the process). This is a pretty fundamental concept for threaded interpreters, and Forth in particular (as well as other stack-driven concatenative languages descended from it) exemplifies this as the very basis of the language itself. > Objects cannot compose Object composition (both by including objects within other objects and by using interfaces and implementations thereof) has been a thing for multiple decades now.
  - crimsonalucard 7y ago
    > I'd argue those ain't "primitives" in that case; what differentiates a "primitive" from some other kind of computation is that it can't be broken down further. Usually when you program you don't create a library made out of the lowest level primitives. You start out with higher level primitives and hopefully if your design is correct, all upper layers are different compositions of this lowest level set of primitives. I am talking about this layer of primitives. If you chose your primitives incorrectly and find out that you need to break apart your primitives, it is far easier to do this if your primitives themselves were also made out of compositions of lower level primitives. If your primitives were already a tangle of objects and procedures this would be very hard. >Depends on how you define "procedure". In the sense used in procedural programming, you certainly can compose them (i.e. by jumping into a subroutine that in turn jumps into other subroutines, pushing and popping things to/from the stack in the process). This is a pretty fundamental concept for threaded interpreters, and Forth in particular (as well as other stack-driven concatenative languages descended from it) exemplifies this as the very basis of the language itself. I defined what I mean by composition. By procedure I mean a list of procedures of instructions. What does it mean to compose steps One through five with steps six through ten? Is steps six through ten compose-able with another set of steps from a whole different sub routine? Most likely no. You cannot define a singular function compose(step1, step6) = step4 like I described in my definition. >Object composition (both by including objects within other objects and by using interfaces and implementations thereof) has been a thing for multiple decades now. Does this fit with what I defined as composition? No it does not. Object composition is a horrible word. It is not true composition. Dependency injection is a more fitting word for what is actually happening. You are making one object depend on another object, you are not composing two objects to form a new object. When objects "compose" you create dependencies and custom glue code to make everything fit together. When I refer to composition I am talking about how bricks compose to form a wall. There is only a singular form of composition to compose bricks into a wall and each brick can exist without a dependency to another brick. "Object composition" is using as much concrete as possible to compose bricks of every geometric shape into a mishmash ball of solids. Again objects and procedures do not compose like bricks. Another programming primitive is more suited for this. Maybe someone knows what this primitive is.
CodingSucks 7y ago
You might like FlowNote, as it implements many of these ideas at a high level coding language. https://emblem21-opensource.github.io/flownote/docs/06-flow.html https://emblem21-opensource.github.io/flownote/docs/06-flow....
kadendogthing 7y ago
Anyone who's had to analyze complex systems at a high level will recognize that this is a pattern that often emerges in some form. The over all pattern happens in game engines, plugin systems, UI frameworks, in service buses, all over the place. It's just now at the code level. It's a good observation, though it seems the author has arrived at the correct conclusion/analysis and tried to shoe horn it in to fundamentally lacking systems (i.e., other languages/stacks that are inherently not functional, have side effects, and do not work off "events"/sync points). My thoughts on this are: append only programming is less effectual than having an over all append only system, where many programs take input (including the original source "message"), then hand off the out put to the next program in line. Which programs get run depends entirely on that run context's configuration. When you have disparate programs working towards a final goal, and these programs are defined in a readable configuration it gives you quite a few benefits. Good clean logging structures, easy to reason about, easy to change (feature toggling, data migration points, etc), and easy to clean up when required. I call it unix'ing your systems. There's a lot of comments calling this silly but please attempt to give it a few read overs and apply it to systems you've worked with. Hopefully they've been large enough to draw similarities to what he's talking about in a very clean, academic manner.
HenryDavis65 7y ago
paywall
punnerud 7y ago
Gives me the feeling of business logic with only SQL and PL/SQL
dan-robertson 7y ago
I think the issue with the argument of the article (just look at the event stream and write programs to tweak it into the stream you want) is threefold: 1. You can’t see the blocking in the event stream behaviour so your change might mysteriously fail to work. I wasn’t super clear on what the semantics were but I think the last change to not show ads to enterprise customers accidentally breaks the normal case and causes the program to lock up once it hits isValid. Normal program changes also suffer from accidentally breaking existing behaviour. Especially eg a change to a superclass breaking subclasses or accidentally mutating some global state or throwing an exception in the wrong place. 2. There are lots of possible event streams and it is hard to predict how well your change will behave under all of them. This is also a problem for any sufficiently large and modified program. 3. You can’t block a block, so it is hard to undo modified behaviour in some cases (this is what causes the atm to lock up after the last change (if I’m right in thinking it does)) However I don’t think this means that this is a pointless area of research. It may be that good ways of dealing with these things are eventually devised. And the idea of lots of small processes doing simple things coming together to make something which holistically behaves in a smart reliable and resilient way seems popular and reasonable (eg see copycat). One good thing about this method would be testability. A test can be just another bthread (or more) running in the system followed by printing a trace of the events. That way one might easily know if some existing behaviour is broken because the printed trace would change. Another similar idea (without necessarily requiring append-only programming) is one from eve and a talk I don’t remember the name of where one writes prolog-like rules to manipulate a set of known facts over time. This also suffers from difficulties with negation. I suppose a general idea is that with these “bag of interacting rules” systems (which seem a good start for building complex systems which can be modified and tweaked reliably and resiliently) it is hard to have negatives (thus blocking) and in particular very hard to have reliable double negatives (blocking a block). So what is the solution? Well I don’t know and it seems that any answer could be unsatisfactory. One possibility could be to allow for more probabilistic or otherwise weighted behaviour but this seems bad for reliability of results. Another idea is to disallow negation but this seems to make everything hard. Another idea might be to somehow make synchronisation points better.
- btown 7y ago
reason-mr 7y ago
Author really needs to read up on the actor model : https://en.wikipedia.org/wiki/Actor_model https://en.wikipedia.org/wiki/Actor_model
- Chris2048 7y ago
  Could you expand on your meaning, and relate B-threads to Actors, instead of dumping a massive article? TLDR of why time is well spend reading it.
taowen 7y ago
We have made a system with similar ideas to re-organize the business logic in a way that is more continuous instead of scattered around ( https://medium.com/software-engineering-problems/out-of-the-tar-pit-another-approach-b3e701e089ee https://medium.com/software-engineering-problems/out-of-the-... ). However, we do not believe event sourcing is the answer to state management problem. It requires too dramatic change on existing infrastructure and ways of human thinking.
magicalhippo 7y ago
This article was way too short to make any clear case for how b-threads would make changes easier to make. In the end, our code is complex because what we want to achieve is complex. Sure sometimes programmers add some extra complexity on top for no good reason, but there's a certain level you just can't get away from. Moving the complexity to understanding event streams does not seem to be any improvement at all, at least as far as explained in the article. The toy examples were entirely too simple to give any real sense of how this could work in practice.
sktrdie 7y ago
Author here: sorry if the "append-only" analogy went a bit upstream in the comments, it was certainly meant to be taken with a pinch of salt. All of regular programming methodologies apply with b-threads. And most of the time changing requirements would require one to change the b-threads involved in that requirement. The append-only analogy was really more to showcase the feature of incrementality. Technically one can develop a complicated system just via append-only but again you'll likely better off having clean b-threads that deal with specific behavior organized in specific and clean ways the same way we organize any other software. From the comments it seems I wasn't successful at highlighting the key insights of this paradigm. One read that I suggested in this article that does a much better job at explaining these concepts is "The quest for runware: on compositional, executable and intuitive models" [1] where the authors essentially describe the need for something like Behavioral Programming (authors are the creators of BP). Some quotes from the paper: > As mentioned earlier, it will be possible to represent new requirements, or changes thereof, in a new behavioral component or module, with minimal change to the previously specified parts of the model, and without sacrificing executability and manageability. Such modules will be simply added to the existing model, virtually ‘piled-atop’ it, with no component specific interface, connectivity, or ordering requirements. > In our vision, the units of the specification and models are not assembled in detail like resistors or chips on a computer board, or methods and fields in an OO-programming object class. The interweaving of behavioral modules will be facilitated by their reference to common aspects of system behavior described using shared vocabularies (for example, common events), and not via mutual awareness and direct communication between components. From the point of view of such a module, the other modules could be transparently replaced by new ones. In fact, the implementation of any part of the specification or a model should be replaceable by some kind of ‘invisible box’, whose implementation remains a mystery, and the effectiveness of the remaining modules and the integrity of the overall system behavior will be preserved. And from the conclusion: > What will happen if and when the human way of expressing requirements for systems will become almost indistinguishable from the way this is done with computer programs? And what will be the result of a level of compositionality that would allow humans to add capabilities to a system with much less dependence on that system than is possible today? > When a collection of specification units grows over time, accumulating an unmanageable collection of patches, exceptions, and enhancements, it is likely developers will call for merging or refactoring them into much more concise artifacts. The new modules will replace the existing collection in all its uses (final executing system, official record of the specification, and means for communication between humans) without changing other parts of the specification. Such refactoring will be acceptable, even if it ends up being done manually, as it will focus on capturing the human’s revised perception of the affected behavior. 1. http://www.wisdom.weizmann.ac.il/~harel/papers/Runware.pdf http://www.wisdom.weizmann.ac.il/~harel/papers/Runware.pdf
patientplatypus 7y ago
Honestly, just don't wind your abstractions too tightly. It's the ultimate "rookie mistake". Just today, I had to write a simple navigation menu. Suppose that I have pages to navigate to that have /a/ through /n/. I abstracted away the button, but I didn't write a loop to loop over each route entirely. Why? Well, based on user interactions I might want to have the buttons change shape or color and I don't want to have to go back and unwind that abstraction. Some people will look at that code and say its not DRY. Well, I like to make code WET - write everything twice. There is some ideal amount of boilerplate as well just so someone can read and understand the darn thing - if they have a few examples to play with it'll be much easier for them to understand. Also, it amuses me how much stake people put in nonsense acronyms, as if some sort of borrowed linguistic authority will cover ineptitude and not thinking for oneself. Ultimately this looks like an over engineered technical fix for what amounts to a social problem. Coding is ultimately a social issue (unless you're out there grinding out 1s and 0s on cuneiform tablets) and requires social solutions.
- quickthrower2 7y ago
  The /a/ to /n/ thing - well of course it depends. It depends on the meaning of those. For example, if I write 1,2,3 can you guess the next number? It could be a sequence, so answer=4, or just three numbers that happen to be 1,2,3 and the next number I need to add is 8, and later I will remove the 2 (perhaps they are hard coded user ids). I am guessing your /a/ to /n/ structure is quite flexible. More like the latter of my example. If you are saying WET event if it is a sequence then I'd argue that's OK if it is unlikely to change and it depends on other factors. Is there other metadata that depends on the letter for example that would benefit from the loop.
sadness2 7y ago
Experience has led me to conclude that code maintainability is a Team. Dedicated staff are needed to recommend cross-cutting solutions, roll them out company wide, perform refactors and clean up technical debt. Making it a "culture" is too hard. It means getting buy-in with regards to how crucial the function is for the sustainability of product development, but once it's a job description, it will actually happen.
hallihax 7y ago
Maybe I'm just too set in my ways - but nothing in this article looks like a solution to anything remotely relevant to most software development. Difficulty in updating software isn't a new problem - but we already have much, much better solutions than this. I don't think there's anything wrong with changing existing code - and if changing existing code is causing you so much pain that you're led down this route, then you probably have bigger issues. Maybe this methodology just isn't for me - but I really don't see how this improves anything from a development perspective. I only care about the current state of the code - and if I need to care about the previous state of the code, then I just check the history. The idea of having to understand the entire history of a codebase just to grok the current state seems insane.
deleted 7y ago
[deleted]
mannykannot 7y ago
I am having trouble reconciling the examples with the description of how B-threads work. Take the three-thread version of the 'water level' example: we start with the first two B-threads waiting on 'waterLevelLow', but the third thread is not waiting on anything, and presumably could advance to the point where it is waiting on 'addCold' - but it has blocked 'addCold' from occurring (a fact used later in the example), so if it did advance before 'waterLevelLow' occurred, it would become deadlocked. I think there must be more complexity here than is being explained.
- mannykannot 7y ago
  When I slowed down the animation, I saw what was happening: the third thread is waiting on 'addHot', and while (and only while) it is waiting, 'addCold' is blocked - so there is no deadlock here, but is deadlock ruled out in general? More generally, does the claimed ease of modification come at the cost of expanding the amount of reasoning about concurrency that you have to do?

167 points by sktrdie 7y ago

I was recently tasked with making some small incremental changes to a large code base that I was not familiar with. I was successful specifically because I could see the existing implementation and use it as a template for making the necessary changes. For example, if I had to add a widget to a form, I could copy the implementation of an existing, similar widget and then make the necessary changes to it. Similarly, I had to change the datatype of a field from string to integer all the way from the UI back to the database. This was fairly easy because I could make the change at one level, and then run the compiler to see what my change broke one level up. Fix, rinse, and repeat. If I had to do any of this without access to the existing code, it would've been 100x harder.

akkartik 7y ago

Interestingly, my article from a few years ago has a very similar motivation and initial figure. And it too ends up at append-only programming. But it does so in a very different way: http://akkartik.name/post/wart-layers http://akkartik.name/post/wart-layers (I still program all my side projects in this way.)

asperous 7y ago

Video game engines often have plugin systems like this that listen for hooks or events and can act on them. It does make extremely complex things a bit more simple. But the chart in graph 1 still applies, often when adding new functionality you realize you need to add or remove modularity and extensions.

Chris2048 7y ago
Eclipse has lots of plugins, but that turned out not so great as too many is known to cause issues.

daenz 7y ago

I'm going to make a conscious effort not to come off sounding like an asshole, but please excuse me if I slip up. I find many of the ideas in post to be fundamentally at odds with the direction that "good software development" should be travelling. The core of my feeling is best captured by the following quote from the post: >As a system grows in complexity we don’t necessarily care about how old b-threads have been written, hence we don’t care about maintaining them. This post is essentially formalizing the process of creating a Big Ball of Mud[0] that is so complex and convoluted that it is impossible to understand. The motivation for formalizing this process seems sane and with good intentions: to add functionality quickly to code you don't really understand. Normally, doing something like is considered cutting corners and incurring explicit technical debt, and must be used sparingly and responsibly. However, the process of "append-only development" is embracing the corner cutting and technical debt as a legitimate development process. I can't get on board with this. To be more specific, with an example (and maybe I am wrong in understanding the post, this would be the time to point that out to me), let's suppose you have a massive complex software system that was built over the years with this "append-only" style of development. One day you find a nasty bug in one of the lower layers, and to correct it, you have to change some functionality, which moves/removes some events that subsequent layers are depending on for their own functionality. Suddenly, you are faced with rewriting all of those layers in order to adapt to your bugfix. What you're left with is a nightmare of changes that disturb many layers of functionality, because they're all based on this append-only diffing concept: the next layer is dependent on the functionality of the previous layer. This is what programming APIs are for: to change functionality in lower layers with minimal influence subsequent layers. This post and process seems to be imagining a world without APIs. 0. http://www.laputan.org/mud/ http://www.laputan.org/mud/

loopbit 7y ago
I completely agree with you. The thing is, I might be able to see this "working" in very simple and limited processes[0]... But going back to your example of the nasty bug in complex software, I believe the post's idea is that you shouldn't fix the bug in the layer it's happening, but, instead, write a new b-thread that corrects that behavior. Which might sound nice in theory but I feel it's much more likely that you won't be able to fix it there (the info you need might be lost in a previous b-thread) or become a piece of code so complex that negates any supposedly benefit that this system had in the first place. [0] It might be just me and completely off-topic, but this reminds me a bit of rule-based expert systems, where the rules get activated by certain conditions and produce effects (that can activate other rules). The idea was always that you could model very complex (and emerging) behaviors based on very simple, human readable rules. The thing is that you could definitely add/tweak/remove the rules as needed.

munificent 7y ago

I feel like Babbage when he said, "On two occasions I have been asked, 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question." Maybe I'm missing some key insight, but this process seems like madness to me. > Then we wouldn’t have to read and understand where to squeeze our changes. We’d simply add stuff based on the new requirements and things would somehow magically work. But, at some point, someone has to understand if those requirements clash with the old ones that are currently implemented. The hot/cold water example is, I think, telling. At no point in the article does the author actually state what the requirements (new or old) are. You have some program that, whenever the water level gets low, adds three hot waters. I guess that was some original requirement. Then another b-thread is added that also adds three cold waters. Now whenever the water gets low, twice as much total water is added. Presumably that's OK? When the second b-thread was added, was the requirement to fill the container with cold water instead of hot, or was it to fill the container with twice as much lukewarm water? Finally, the third b-thread is added that interacts with the two previous ones to cause the hot and cold water adds to be interleaved. What requirement does that meet? The stated purpose of all of this is that you don't need to understand the original program to modify it, but the entire purpose of this third thread is to interact with the previous two. You have to know there are already two b-threads going, which events they listen to, and what they yield in order to create the third thread that blocks them. So I don't see how this is any better than needing to read the code in a normal program. I feel like I'm taking crazy pills.

Twisol 7y ago
On top of the requirements engineering that's left implicit here, there's still the matter of understanding the event trace that you're supposed to be modifying. If every patch is a transformer from traces to traces, then it stands to reason that you must understand the event trace at least as well as the program you would otherwise be modifying. And program traces are an inherently dynamic thing -- how do you gain an understanding about the potentially very complicated, interwoven traces you're developing with if you can only see one trace per execution? A great deal of the problem of concurrent programming is just coming to grips with these traces, and decomposing them in a comprehensible way within the structure of your codebase.

hzhou321 7y ago

It starts with an interesting description of problem, but I felt that it took a sharp slope and reached into a bizzare conclusion.

hinkley 7y ago
We are still wrestling mightily with the notion that solving a problem badly can be worse than not solving it at all.

kazinator 7y ago

Yikes; this nonsense will turn into an incomprehensible ball of hair in the implementation of anything real. You don't know if anything will work or continue to work due to being randomly blocked or otherwise interfered with by code that either already exists or will be added tomorrow. Basically this is just flow-based programming (FBP) in a new form. https://en.wikipedia.org/wiki/Flow-based_programming https://en.wikipedia.org/wiki/Flow-based_programming Every new generation of coders reaches a puberty characterized by uncontrollable excitement with multiple threads of control doing little things independently.

crimsonalucard 7y ago
Flow based programming is basically functional programming. It feels like a redefinition of a style of programming that's already heavily established.
- yellowapple 7y ago
  I'd say it's more of a potential application of functional programming - that is, you could* certainly use functional programming techniques/languages to implement flow-based programming, but functional languages are not automatically flow languages by merely existing. * But this is not strictly necessary. See also: Unix pipes, which fit the definition of flow-based programming while being possible - and in fact frequently done - with very-much-not-functional languages both implementing the individual steps and connecting them together.
  - crimsonalucard 7y ago
    It looks like flow based programming is exactly the same thing as functional composition. Can you illustrate to me how these two concepts are disparate? I ask because if they are one in the same, then because the only way to link functions in functional programming is through function composition than functional programming is flow based programming.
    - kazinator 7y ago

CodeSucks 7y ago

You might like FlowNote, as it implements many of these ideas at a high level coding language. https://emblem21-opensource.github.io/flownote/docs/06-flow.html https://emblem21-opensource.github.io/flownote/docs/06-flow....

hacker_9 7y ago

Append style programming is very interesting, especially when thinking of modelling future languages closer to natural language. This foray though seems to end up generating more problems than it's worth. For one, its not clear how this aids easier development - hasn't it just shifted understanding of a complex program to understanding a complex overlapping sequence of b threads instead? Secondly, and I think the nail in the coffin, the compiler would have to be brilliant at optimisation to make the final program performant. The idea of 'show ads' later being disabled by another overlapping b thread - does this mean the network request is never performed? Or is it performed and the result thrown away, wasting resources. How would an 'offline mode' requirement be appended as a b thread in such a case?

codebje 7y ago
The events are a scheduling mechanism, not the work done. Performance probably relates more to the ratio of time spent selecting the next event to time spent executing code between yields than anything else: a thread waiting on an event joins a queue, a thread requesting an event joins the same queue but also flags it as ready, and a thread blocking an event flags the queue as not ready. When a queue is selected, it's drained completely before a new queue is selected. The run cost of the scheduler would depend mostly on the structure used to maintain the ready set, but a basic double ended linked list would provide linear time operations for ready/blocked management. Whole program compilation could provide lookups or perfect hashes for event queueing, or partial compilation could rely on imperfect hashing. Optimising during compilation might work best by trying to fuse events such that the scheduler is invoked less often. Things that are not apparent from a lightweight article like this include state transfer (how do I know details of the card inserted?) and the likelihood that I'd keep the code to each b-thread and hack on those to make new ones...

deleted 7y ago

[deleted]

yowlingcat 7y ago

Nothing about this approach seems easier to reason about nor maintain over the long term. Transactional integrity, migrating data structures, and modifying business logic/control flow are key building blocks required for many production applications, certainly line of business ones. I'm reminded of the golden hammer fascination with event sourcing, and I continue to see event sourcing applied inappropriately. Is it useful for some specialized applications? Yes, and it's a great tool there! But should it be the default tool you reach for to solve most problems over a relational DB and your most trusted, high level, stable programming language with a mature library ecosystem? No to that question, and no to this.

msteffen 7y ago

In a prior team, we accomplished the goal of "a newcomer should be able to figure out the history of the code they're looking at, and thereby understand why it is this way" in a simpler, lower-tech way: 1. One commit per PR (so the commit history wasn't polluted with rough drafts and debug logs and such) 2. Every PR links to a bug 3. Design discussions happen in bugs Then when you see some weird code, you can "blame" back to the commit, and from there look at the bug to see why the commit was needed. We also really encouraged people to separate refactoring from new features (refactor first, in a descriptive PR, then add the feature), so that you avoided the problem of seeing a complex refactoring in a PR called "add foo" I agree with other commenters that encoding the design history of your project in its operation doesn't seem helpful, and in my opinion that's because it's solving the wrong problem. Finding the places in a complicated project that need to change is only hard because knowing, to a fairly complete degree, how the system already works is hard, and knowing how a system already works is mostly the product of understanding the problem that it solves deeply. Experienced engineers on a project usually have no problem figuring out which parts of a system need to be changed, because they already know the system well, and they don't know they system well by virtue of having all the code memorized, which would be impossible, but by having participated in all of the design discussions. Therefore, the problem that needs to be solved is making the learning process as fast and easy as possible—thus the strict process around documenting changes and their motivation. Allowing people to quickly discover the existing code's motivation gives them the information that will actually need be committed to memory.

0xdeadbeefbabe 7y ago
Is all your code in one repo? Just curious, because I see this learning problem made worse by multiple repos with multiple PRs.
- msteffen 7y ago
  It was all in one repo on that team, yes. Offhand, it seems like linking all PRs in all repos back to the bug, and linking the bug to all of the disparate PRs might help. A version (possibly) of the "many PRs" problem that we sort of had was that it often took many PRs to close a bug. Our goal wasn't to minimize the number of PRs, and in fact we encouraged small PRs. Rather, the goal was to get engineers past the commit and PR to the underlying motivation and design.

jonstaab 7y ago

Just to add some contrast to the mostly negative comments here (which have merit), this is interesting to me, not because it aims to hide the past, but because it makes time a first-class concept in the software development model, which most programming styles fail to do (e.g. most RDBMS frameworks add migrations on as an afterthought). I like this, and hope something like it catches on. The problems with this approach seem solvable to me, albeit with more experimental magic that could explode: - The resulting big ball of mud (and subsequent performance problems of a long pipeline of relations) could be compiled away, resulting in a single artifact. That is, you'd develop in append-only style, but when you "commit" your release, you'd end up with a single, optimized artifact for deployment, which would also be readable. This seems really nice to me, since your changes, while being based on the behavior of the system, would create a diff in the implementation of the system, potentially reaching way back into upstream events (in the example, the behaviors that block hot water and substitute cold water would just completely eliminate the first pane and simplify to adding cold water). This would let you see your changes from multiple perspectives. This approach also seems really friendly to fuzz-testing, which would give you a third look into the behavior of the system, and you could write tests based on the final state of the system after a number of given events. - Migrating data structures actually seems easier to me for an event sourced approach, since you'd just re-project your domain models based on the new flow of events. b-threads would allow you to re-compile your event stream just like you re-compiled the source artifact (having parameterized events remains a problem, since your historical data could end up being incomplete and invalid based on new policy, you'd have to adopt a permissive schema to keep stuff that validation would otherwise reject). I'll agree that b-trees don't really solve anything, but they do bring up some interesting questions that I think are worth asking. Datomic and Darklang I think are much more practical, and seem to dabble in the same sort of areas.

hacker_9 7y ago
>> when you "commit" your release, you'd end up with a single, optimized artifact for deployment I agree this would be nice, but think of this problem more deeply and you quickly run into major issues. The biggest one being that your compiler now has a large scale optimisation step to do completely automatically, better than a human. In order to make this work, you'd effectively have to build everything out of composable components, which you then can switch on/off with some sort of layering system. The problem here is how do you come to a decision on what those composable units look like? You don't know what you don't know - i.e. what those future layers you append are going to need to turn off or on. And coming back to optimisation - once you know the features you are combining together in an algorithm, only then do you know the data structures that fit the problem. Optimisation isn't a forward process when you turn parts off and end up with an efficient system. Optimisation requires a high level view, a low level view, an understanding of context, of memory requirements, of language capabilities, even leaps of logic. For the last part when I mean is that whilst a compiler may be able to reorganize something in the form a*b=c (and even then knowing when to do this is a book on it's own), it can't know that if I, for example, disable a feature that adds nested nodes to a tree structure, that it could rewrite functions to no longer require recursion and instead treat them like serial lists. You could rewrite the methods yourself and add them to your new layer, but you also will inevitably end up not fitting the previous composable structure. I think the deeper problem is related more to the tying of syntax to structure, as structure is ultimately your application's performance profile. Untying these two though could lead to a way forward for the append style paradigm.

crimsonalucard 7y ago

The solution to incremental changes is to develop your app in a way where every unit of computation is compose-able under a strict rule. Meaning I can write a universal function compose(A,B) = C and that function can compose any primitive. Then you build complicated logic just like how you build a wall from a set of bricks... form complicated logic as a composition of more primitive logic all the way down. If your program is constructed this way at every layer of logic, not only will your program be amenable to append only styles of programming. But it will be amenable to decomposition. Often you find that your primitives are too big and you need to split your primitives... well if your primitives are itself made out of the compositions of lower level primitives then breaking apart that function is trivial. There is only one primitive in all of programming that follows this composition rule. Procedures cannot compose, Objects cannot compose. What's left?

yellowapple 7y ago
> if your primitives are itself made out of the compositions of lower level primitives I'd argue those ain't "primitives" in that case; what differentiates a "primitive" from some other kind of computation is that it can't be broken down further. > Procedures cannot compose Depends on how you define "procedure". In the sense used in procedural programming, you certainly can compose them (i.e. by jumping into a subroutine that in turn jumps into other subroutines, pushing and popping things to/from the stack in the process). This is a pretty fundamental concept for threaded interpreters, and Forth in particular (as well as other stack-driven concatenative languages descended from it) exemplifies this as the very basis of the language itself. > Objects cannot compose Object composition (both by including objects within other objects and by using interfaces and implementations thereof) has been a thing for multiple decades now.
- crimsonalucard 7y ago
  > I'd argue those ain't "primitives" in that case; what differentiates a "primitive" from some other kind of computation is that it can't be broken down further. Usually when you program you don't create a library made out of the lowest level primitives. You start out with higher level primitives and hopefully if your design is correct, all upper layers are different compositions of this lowest level set of primitives. I am talking about this layer of primitives. If you chose your primitives incorrectly and find out that you need to break apart your primitives, it is far easier to do this if your primitives themselves were also made out of compositions of lower level primitives. If your primitives were already a tangle of objects and procedures this would be very hard. >Depends on how you define "procedure". In the sense used in procedural programming, you certainly can compose them (i.e. by jumping into a subroutine that in turn jumps into other subroutines, pushing and popping things to/from the stack in the process). This is a pretty fundamental concept for threaded interpreters, and Forth in particular (as well as other stack-driven concatenative languages descended from it) exemplifies this as the very basis of the language itself. I defined what I mean by composition. By procedure I mean a list of procedures of instructions. What does it mean to compose steps One through five with steps six through ten? Is steps six through ten compose-able with another set of steps from a whole different sub routine? Most likely no. You cannot define a singular function compose(step1, step6) = step4 like I described in my definition. >Object composition (both by including objects within other objects and by using interfaces and implementations thereof) has been a thing for multiple decades now. Does this fit with what I defined as composition? No it does not. Object composition is a horrible word. It is not true composition. Dependency injection is a more fitting word for what is actually happening. You are making one object depend on another object, you are not composing two objects to form a new object. When objects "compose" you create dependencies and custom glue code to make everything fit together. When I refer to composition I am talking about how bricks compose to form a wall. There is only a singular form of composition to compose bricks into a wall and each brick can exist without a dependency to another brick. "Object composition" is using as much concrete as possible to compose bricks of every geometric shape into a mishmash ball of solids. Again objects and procedures do not compose like bricks. Another programming primitive is more suited for this. Maybe someone knows what this primitive is.

CodingSucks 7y ago

kadendogthing 7y ago

Anyone who's had to analyze complex systems at a high level will recognize that this is a pattern that often emerges in some form. The over all pattern happens in game engines, plugin systems, UI frameworks, in service buses, all over the place. It's just now at the code level. It's a good observation, though it seems the author has arrived at the correct conclusion/analysis and tried to shoe horn it in to fundamentally lacking systems (i.e., other languages/stacks that are inherently not functional, have side effects, and do not work off "events"/sync points). My thoughts on this are: append only programming is less effectual than having an over all append only system, where many programs take input (including the original source "message"), then hand off the out put to the next program in line. Which programs get run depends entirely on that run context's configuration. When you have disparate programs working towards a final goal, and these programs are defined in a readable configuration it gives you quite a few benefits. Good clean logging structures, easy to reason about, easy to change (feature toggling, data migration points, etc), and easy to clean up when required. I call it unix'ing your systems. There's a lot of comments calling this silly but please attempt to give it a few read overs and apply it to systems you've worked with. Hopefully they've been large enough to draw similarities to what he's talking about in a very clean, academic manner.

HenryDavis65 7y ago

paywall

punnerud 7y ago

Gives me the feeling of business logic with only SQL and PL/SQL

dan-robertson 7y ago

I think the issue with the argument of the article (just look at the event stream and write programs to tweak it into the stream you want) is threefold: 1. You can’t see the blocking in the event stream behaviour so your change might mysteriously fail to work. I wasn’t super clear on what the semantics were but I think the last change to not show ads to enterprise customers accidentally breaks the normal case and causes the program to lock up once it hits isValid. Normal program changes also suffer from accidentally breaking existing behaviour. Especially eg a change to a superclass breaking subclasses or accidentally mutating some global state or throwing an exception in the wrong place. 2. There are lots of possible event streams and it is hard to predict how well your change will behave under all of them. This is also a problem for any sufficiently large and modified program. 3. You can’t block a block, so it is hard to undo modified behaviour in some cases (this is what causes the atm to lock up after the last change (if I’m right in thinking it does)) However I don’t think this means that this is a pointless area of research. It may be that good ways of dealing with these things are eventually devised. And the idea of lots of small processes doing simple things coming together to make something which holistically behaves in a smart reliable and resilient way seems popular and reasonable (eg see copycat). One good thing about this method would be testability. A test can be just another bthread (or more) running in the system followed by printing a trace of the events. That way one might easily know if some existing behaviour is broken because the printed trace would change. Another similar idea (without necessarily requiring append-only programming) is one from eve and a talk I don’t remember the name of where one writes prolog-like rules to manipulate a set of known facts over time. This also suffers from difficulties with negation. I suppose a general idea is that with these “bag of interacting rules” systems (which seem a good start for building complex systems which can be modified and tweaked reliably and resiliently) it is hard to have negatives (thus blocking) and in particular very hard to have reliable double negatives (blocking a block). So what is the solution? Well I don’t know and it seems that any answer could be unsatisfactory. One possibility could be to allow for more probabilistic or otherwise weighted behaviour but this seems bad for reliability of results. Another idea is to disallow negation but this seems to make everything hard. Another idea might be to somehow make synchronisation points better.

btown 7y ago

reason-mr 7y ago

Author really needs to read up on the actor model : https://en.wikipedia.org/wiki/Actor_model https://en.wikipedia.org/wiki/Actor_model

Chris2048 7y ago
Could you expand on your meaning, and relate B-threads to Actors, instead of dumping a massive article? TLDR of why time is well spend reading it.

taowen 7y ago

We have made a system with similar ideas to re-organize the business logic in a way that is more continuous instead of scattered around ( https://medium.com/software-engineering-problems/out-of-the-tar-pit-another-approach-b3e701e089ee https://medium.com/software-engineering-problems/out-of-the-... ). However, we do not believe event sourcing is the answer to state management problem. It requires too dramatic change on existing infrastructure and ways of human thinking.

magicalhippo 7y ago

This article was way too short to make any clear case for how b-threads would make changes easier to make. In the end, our code is complex because what we want to achieve is complex. Sure sometimes programmers add some extra complexity on top for no good reason, but there's a certain level you just can't get away from. Moving the complexity to understanding event streams does not seem to be any improvement at all, at least as far as explained in the article. The toy examples were entirely too simple to give any real sense of how this could work in practice.

sktrdie 7y ago

Author here: sorry if the "append-only" analogy went a bit upstream in the comments, it was certainly meant to be taken with a pinch of salt. All of regular programming methodologies apply with b-threads. And most of the time changing requirements would require one to change the b-threads involved in that requirement. The append-only analogy was really more to showcase the feature of incrementality. Technically one can develop a complicated system just via append-only but again you'll likely better off having clean b-threads that deal with specific behavior organized in specific and clean ways the same way we organize any other software. From the comments it seems I wasn't successful at highlighting the key insights of this paradigm. One read that I suggested in this article that does a much better job at explaining these concepts is "The quest for runware: on compositional, executable and intuitive models" [1] where the authors essentially describe the need for something like Behavioral Programming (authors are the creators of BP). Some quotes from the paper: > As mentioned earlier, it will be possible to represent new requirements, or changes thereof, in a new behavioral component or module, with minimal change to the previously specified parts of the model, and without sacrificing executability and manageability. Such modules will be simply added to the existing model, virtually ‘piled-atop’ it, with no component specific interface, connectivity, or ordering requirements. > In our vision, the units of the specification and models are not assembled in detail like resistors or chips on a computer board, or methods and fields in an OO-programming object class. The interweaving of behavioral modules will be facilitated by their reference to common aspects of system behavior described using shared vocabularies (for example, common events), and not via mutual awareness and direct communication between components. From the point of view of such a module, the other modules could be transparently replaced by new ones. In fact, the implementation of any part of the specification or a model should be replaceable by some kind of ‘invisible box’, whose implementation remains a mystery, and the effectiveness of the remaining modules and the integrity of the overall system behavior will be preserved. And from the conclusion: > What will happen if and when the human way of expressing requirements for systems will become almost indistinguishable from the way this is done with computer programs? And what will be the result of a level of compositionality that would allow humans to add capabilities to a system with much less dependence on that system than is possible today? > When a collection of specification units grows over time, accumulating an unmanageable collection of patches, exceptions, and enhancements, it is likely developers will call for merging or refactoring them into much more concise artifacts. The new modules will replace the existing collection in all its uses (final executing system, official record of the specification, and means for communication between humans) without changing other parts of the specification. Such refactoring will be acceptable, even if it ends up being done manually, as it will focus on capturing the human’s revised perception of the affected behavior. 1. http://www.wisdom.weizmann.ac.il/~harel/papers/Runware.pdf http://www.wisdom.weizmann.ac.il/~harel/papers/Runware.pdf

patientplatypus 7y ago

Honestly, just don't wind your abstractions too tightly. It's the ultimate "rookie mistake". Just today, I had to write a simple navigation menu. Suppose that I have pages to navigate to that have /a/ through /n/. I abstracted away the button, but I didn't write a loop to loop over each route entirely. Why? Well, based on user interactions I might want to have the buttons change shape or color and I don't want to have to go back and unwind that abstraction. Some people will look at that code and say its not DRY. Well, I like to make code WET - write everything twice. There is some ideal amount of boilerplate as well just so someone can read and understand the darn thing - if they have a few examples to play with it'll be much easier for them to understand. Also, it amuses me how much stake people put in nonsense acronyms, as if some sort of borrowed linguistic authority will cover ineptitude and not thinking for oneself. Ultimately this looks like an over engineered technical fix for what amounts to a social problem. Coding is ultimately a social issue (unless you're out there grinding out 1s and 0s on cuneiform tablets) and requires social solutions.

quickthrower2 7y ago
The /a/ to /n/ thing - well of course it depends. It depends on the meaning of those. For example, if I write 1,2,3 can you guess the next number? It could be a sequence, so answer=4, or just three numbers that happen to be 1,2,3 and the next number I need to add is 8, and later I will remove the 2 (perhaps they are hard coded user ids). I am guessing your /a/ to /n/ structure is quite flexible. More like the latter of my example. If you are saying WET event if it is a sequence then I'd argue that's OK if it is unlikely to change and it depends on other factors. Is there other metadata that depends on the letter for example that would benefit from the loop.

sadness2 7y ago

Experience has led me to conclude that code maintainability is a Team. Dedicated staff are needed to recommend cross-cutting solutions, roll them out company wide, perform refactors and clean up technical debt. Making it a "culture" is too hard. It means getting buy-in with regards to how crucial the function is for the sustainability of product development, but once it's a job description, it will actually happen.

hallihax 7y ago

Maybe I'm just too set in my ways - but nothing in this article looks like a solution to anything remotely relevant to most software development. Difficulty in updating software isn't a new problem - but we already have much, much better solutions than this. I don't think there's anything wrong with changing existing code - and if changing existing code is causing you so much pain that you're led down this route, then you probably have bigger issues. Maybe this methodology just isn't for me - but I really don't see how this improves anything from a development perspective. I only care about the current state of the code - and if I need to care about the previous state of the code, then I just check the history. The idea of having to understand the entire history of a codebase just to grok the current state seems insane.

deleted 7y ago

[deleted]

mannykannot 7y ago

I am having trouble reconciling the examples with the description of how B-threads work. Take the three-thread version of the 'water level' example: we start with the first two B-threads waiting on 'waterLevelLow', but the third thread is not waiting on anything, and presumably could advance to the point where it is waiting on 'addCold' - but it has blocked 'addCold' from occurring (a fact used later in the example), so if it did advance before 'waterLevelLow' occurred, it would become deadlocked. I think there must be more complexity here than is being explained.

mannykannot 7y ago
When I slowed down the animation, I saw what was happening: the third thread is waiting on 'addHot', and while (and only while) it is waiting, 'addCold' is blocked - so there is no deadlock here, but is deadlock ruled out in general? More generally, does the claimed ease of modification come at the cost of expanding the amount of reasoning about concurrency that you have to do?