9 ms·

It's my own weird little take, but I'll share it. In my experience, simple concurrency is easy. Confused concurrency is exceptionally hard. I think about this

• joubert 6y ago
  > Confused concurrency. I’m not sure I know what you mean by that. > If you take a certain view on concurrency, which is "do it as simply as possible, don't break certain rules, don't try to be too clever," Is there a list of those rules?
  • klodolph 6y ago
    > Is there a list of those rules? Several different lists. You have to pick a concurrency model. Within that concurrency model, you pick some set of rules to follow which has the guarantees you want.
    • chii 6y ago
      so how do you know which model to use? How do you know what rules guarentee what outcome? How do you compare them against each other? All of those things mentioned in the original reply is vague, and can only come from having experience in the subject matter. That's why concurrency is hard.
      • klodolph 6y ago
        > so how do you know which model to use? That's like asking, "How do you know what algorithm to use?" You can come up with a justification for why a particular algorithm may be suitable for your purposes, but you'd be hard-pressed to explain how you go about choosing an algorithm. It's the same way with rules for a codebase... you can justify why you are using a specific set of rules, but explaining the decision process that led you to choose those rules is much more difficult. That's why tech leads don't grow on trees. I'll add to what the original comment (about "confused concurrency") was saying, because it matches with my experiences. The good lead developers are the ones who are able to get the job done (design systems) and keep the concurrency model simple. It's often possible to keep concurrency mostly simple and mostly safe.
        • hedora 6y ago
          I worked on a large extremely high performance system (dozens of developers) that’s almost entirely lock free, and constantly mutates shared state. Concurrency issues weren’t really a big problem in that code base. Developers took a long time to ramp up, but that’s an OK tradeoff in my book. None of this would have been possible without lead developers that designed and continuously refined the guard rails that our internal “standard library” implemented. Oddly, I now find it easier to reason about lock free algorithms than mutex based code (and I wrote the latter for 15 years before working on the lock free system)
          • RcouF1uZ4gsC 6y ago
            >Oddly, I now find it easier to reason about lock free algorithms than mutex based code That is because lock free code composes. Mutex based code does not compose. With lock free code, I can freely call another function, knowing I cannot deadlock. With mutex based code, I have to be really careful about the state of my mutexes and think if the code I am calling will attempt to lock one of the mutexes I am holding, thereby causing a deadlock.
            • jbn 6y ago
• xfer 6y ago
  As always the problem is the specification. Once you sit down and write the formal specification, suddenly the implementation becomes much clearer.
  • jjav 6y ago
    I wish this was a more widely held understanding. Document first, code later. Makes nearly all problems easy. Jumping to writing code first is a future disaster in the making.
    • lmm 6y ago
      I find this is nonsense. Code is the best documentation; I've watched people spend hours debating a spec that becomes obviously broken as soon as you spend 5 minutes writing it up in something with a typechecker. Write the specification as code - make sure it's something that can actually be checked. Then you'll know whether it's actually any good.
      • blub 6y ago
        Documentation is written in a human language, can include other media like graphics and is addressed to humans. We write documentation for the same reason that we don't produce maps at a 1:1 scale: documentation offers a higher-level description that can be grasped without having to understand all the code. Not only is code not the best documentation, it's not documentation at all, outside of special cases where the entire code describes e.g. an algorithm and is encapsulated into one function.
        • mrbald 6y ago
          Code should have a hierarchical structure, so one could read it on the high level of abstraction and also zoom in gradually, as necessary — this is what they usually mean by “code is the documentation”, plan the reading process as you plan the functionality. It is a very important aspect of the readability, otherwise you will just sink in details when reading. But it is not easy to formally validate with tools, so naturally it is better to have both code and a picture and have discipline to keep them in sync.
• marta_morena_28 6y ago
  This is incredibly vague. There is no "set of rules" that just magically works without thought. There is just SOOOO much that can go wrong. But one general rule crystallizes itself pretty quickly: DO NOT EVER MUTATE STATE! I.e. when you write concurrent code that builds on state mutation, you are opening Pandorra's box. If you want concurrency that people can understand and maintain, write immutable, functionally pure code (same data in, same result out, no exceptions). That is relatively easy to police and enforce, while still flexible enough to solve most use cases. It may however, not be the most performant.
  • maxeonyx 6y ago
    I like to think of mutation as a performance optimisation. Correctness first, then optimisation. Unless performance is a problem, don't ever mutate state.
    • nyanpasu64 6y ago
      Mutation is not merely a performance optimization, but makes code more concise and expressive. Mutation makes things like `cfg.tabs[2].title = "Suspended"` possible to write in one line. Trying to achieve the same thing with only immutable data structures forces you to clone title, then clone cfg.tabs[2] with a new title, then clone cfg.tabs with a new [2], then clone cfg with a new tabs.
      • waynesonfire 6y ago
        You don't lose conciseness with mutable state. What you seem to think is lost can be regained through abstraction. Just think about the complexity behind the assignment operator, all the way to the logic gates in RAM.
      • lmm 6y ago
        You don't need mutation to write that expressively; you can achieve the same thing with lenses. With the advantage that if you ever get confused about what's happening you can break the lens down into what it's "really" doing and reason about that, in a way that you just can't with language-level mutation. Syntax shortcuts are good when they're built on a rigorous foundation. But if you build a shortcut directly into the language, you'll never be able to retrofit a reasonable model for how it works.
• corytheboyd 6y ago
  I recently wrote some of the type of code that you would be upset to read, so maybe someone can sympathize with this a bit. The task at hand wasn’t easy at all, it required both: 1. Making numerous network requests (some endpoints support batching, so do that as much as possible) 2. The entire main loop needs to be as fast as reasonably possible The only real way to satisfy both, given that serially executing the network activity would waste tons of time, was some fairly tricky concurrent logic. I ended up using a few queue structures to collect day before making batch network calls, and stitching together all of the results at the end (block on all work threads, collect, report, then start next iteration) Every step along the way made sense, every addition of complexity felt justified. And the first person to come to me asking how the fuck it got so complicated is frustrated, so my knee jerk reaction is to get frustrated back saying well how else could we satisfy complex requirements with a complex solution. We should always be challenging ourselves and others to keep solutions simple, but remember that the simplest solution may itself still be complex.
  • ccccc0 6y ago
    Could you not have used a simple thread pool model in your case? That would be a trivial parallelization of your network requests with limiting on the resources (number of parallel connections) Otherwise when the logic gets complex I try to model it as independent threads each having a well-defined role (e.g. exclusive write access to SQLite) communicating with channels. Although I tend to code these things in Rust, the language who taught me that philosophy is Go.
  • tleb_ 6y ago
    Maybe providing a diagram to explain the structure would help? Representing queues and workers as arrows between queues.
  • lmm 6y ago
    IMO the key is always to be compositional. E.g. here you should be able to separate the batching up requests from the performing network calls, and then you can test and reason about both pieces separately. I find the iteratee model is really good for this.
  • 6y ago
• jjav 6y ago
  Good perspective, I agree. One needs to pick the concurrency model and design up front. Document it before writing a single line of code. Then stick to the plan. It takes discipline, but it's not difficult. I mostly write in C with pthreads and as long as one is willing to apply discipline to the task, there's no reason to fear concurrency. If one starts adding a thread here and a mutex there and some shared structures over there without any prior plan then yes, absolutely, that leads to a world of hurt and a deep hole that is very difficult to get out of. So, don't do that.
  • mrbald 6y ago
    "If one starts adding a thread here and a mutex there..." -- precisely that!
• lloeki 6y ago
  > "do it as simply as possible, don't break certain rules, don't try to be too clever," This has to be a universal rule of programming. It's equally true for Ruby and Perl, which are extremely effective languages, but devolve quickly because of their uncanny ability to power cleverness. It's the whole premise of Go, by being boring by design. Complex is easy, simple is hard, but it seems mostly because we fail to be able to resist the allure of cleverness.
  • roryrjb 6y ago
    I will argue that Go's concurrency model is fundamentally flawed[0] precisely because its underlying mechanism is hidden from the user. If it wasn't for this model I would be using Go instead of C and probably instead of a lot of other alternatives. Yes Go the language is boring and I like this idea, but the runtime is a black box, that I don't like so much. [0]. For example: https://github.com/golang/go/issues/14592 https://github.com/golang/go/issues/14592.
    • ampdepolymerase 6y ago
      Managed concurrency like Node.js and Go are fundamentally different from Rust and C/C++'s minimal runtime concurrency.
• BiteCode_dev 6y ago
  You just sum up the entire raison d'être of reactjs and the stores ecosystem around it: force all the team to "do it as simply as possible, don't break certain rules, don't try to be too clever". It's not that react, redux and cie are something new or disruptive. It just enforces a set of design rules that turns a lot of concurrent inputs and outputs into a linear workflow.