4 ms·

So what is your preferred solution for programs that intrinsically need to deal with a lot of state ? Global variables ? I used to see that type of approach i

So what is your preferred solution for programs that intrinsically need to deal with a lot of state ? Global variables ? I used to see that type of approach in some complex Fortran simulation programs where every function referenced a huge common block. I really don't think that leads to easier to manage solutions.

lgessler 6y ago
Essentially, yes, but in a way that sounds more careful than what you saw in that Fortran program. If it absolutely has to be stateful, like with a db connection for example, then there's no getting around that, but what you can do is (1) not bury it in a bunch of stateful objects that didn't have to be stateful, which can cause cascading errors when either one of them misbehaves and also makes it difficult to isolate any one stateful component for testing or re-use, and (2) use special wrappers around it that will make the statefulness easier to deal with. (For examples of (2), see Clojure atoms and the Clojure library mount: https://github.com/tolitius/mount https://github.com/tolitius/mount )
- nostrademons 6y ago
  More concretely, for every bit of state in the program I'm interested in two big questions, each with two subquestions: 1.) Who can access this state, and which of those accesses allow writes vs. which are read-only? 2.) What is the lifetime of that state, and what portion of that lifetime is mutable vs. read-only? Java-style OOP can control #1, but makes no distinction between reads & writes. C++ const correctness and the val/var distinction in Kotlin/Ocaml/ES6/etc. can make that distinction, but only C++ forces you to think in terms of lifetimes. Rust borrowing probably comes closest to answering all 4. I'm still waiting for a language feature that lets you say "This field of this struct is only mutable while this module is running, and once it completes it will never be changed", though. A lot of data structures are initialized in passes and then passed along as constant data: you construct the basic object structure with nothing but a few IDs, then you compute extra fields as necessary, then you're done and the object is never written to again. If you could keep the fields mutable during initialization (which may be a longer process than just constructor calls) and then seal them off once that's complete, you eliminate a whole class of bugs where a read-only client decides to mutate a field that should only be mutated from designated initializer.
  - bpolverini 6y ago
    That would be really cool, but I think enforcing that is undecidable. My gut tells me that having that language feature at compile time is the same as the Halting Problem.
    - tgflynn 6y ago
      I suspect that the ideal language (at least for human use) is one that is just shy of Turing completeness. Once a program compiles and an initial configuration phase has passed all loops would be bounded and all recursions would be guaranteed to terminate. Except for the case of programs that aren't supposed to terminate for which there would be a single construct allowing wrapping everything inside one and only one infinite loop.
    - nostrademons 6y ago
      In its full generality it's undecidable, but there are probably restrictions you can build into the type system to make it decidable. I was thinking something along these lines: class Symbol { @init(constructor) originalToken: Token @init(constructor) position: SourceLocation @init(Parser.parse) scope: SymbolTable @init(Parser.parse) declaredType: Type @init(TypeChecker.typecheck) inferredType: Type @init(DataFlowAnalyzer.computeLiveness) usages: List<Expr> } And then the type system carries an extra bit of information around for whether a reference is fully-constructed or not, much like const-correctness. Fields marked with @init can only be written on a reference that's not fully-constructed. There's no compile-time enforcement for initialization order, though it'd be pretty easy to do this at runtime (convert them to null-checks or special not-initialized sentinel values). Newly-created references start out with the "initializing" bit set, but once they're returned from a function or passed into a function not in the list of legal accessors, they lose this bit unless explicitly declared. It's basically the same way "mutable" works (in languages that support it), but with a separate state bit in the type system and extra access checks within the mutating functions to make sure they only touch the fields they're declared to touch. You can fake this now by passing mutable references into initialization functions and then only using const, but it's a bit less specific because many classes are designed to be long-term mutable through 1-2 specific public APIs but also need to be mutated internally for deferred initialization, and lumping these use-cases together means that deferred fields can be touched by the public API.
- tgflynn 6y ago
  (2) sounds very similar to the way I have typically handled db access in C++. However the kind of state I was referring to is more about the internal computational data used by the program than about external resources.
waffletower 6y ago
You might want to consider examining another programming language for clues here. Perhaps look to a language that has a more modern approach to state such as Clojure.
- tgflynn 6y ago
  My understanding is that Clojure is basically lisp implemented inside the Java ecosystem. I'm not sure how that makes it "modern" given that the only higher level language older than lisp is the original version of Fortran.
  - waffletower 6y ago
    Once you have some understanding of the state management facilities native to Clojure, you might be willing to concede the "modern" adjective.
  - tsimionescu 6y ago
    Clojure has quite a few specific libraries. As far as I know, one of these implements a lot of persistent data structures - data structures that make it efficient to expose an API that returns a copy of the whole structure for any mutation (by sharing all of the unmodified substructure behind the scenes).
  - dkersten 6y ago
    That like saying Go/Rust/Kotlin are just Algol, what’s so modern about them? Lisp isn’t a language anymore, it’s a category of languages and it’s members are rather different from each other besides some superficial syntax style. Common Lisp, Scheme and Clojure are three very different languages, certainly more different than Ruby is from Python IMHO. Clojure, in 2020, is very very different from the Lisp of 50 years ago and is modern in many respects, even compared to other popular languages.
pjmlp 6y ago
Modules with responsibility to handle the specific set of states, which are basically final classes.
tabs_masterrace 6y ago
Ideally you want to have all your state in one place, the "single point of truth", in tightly modeled data structures, keeping it as lean and immutable as possible. The program is then just a function of the state, consisting mainly of self-contained functions, that take arguments and produce a result, without causing side effects. Realistically, that doesn't mean there are no classes or objects with internal vars, but these are more of ephemeral nature, and can be created predicatively and deterministically from your centralized global state. In the best case you can have complex applications with tens of thousands LoCs that feed from just a few dozen state variables.
- mhoad 6y ago
  I must admit that does sound like a really nice code base to work inside of.
fish45 6y ago
ECS architecture is IMO very frictionless no matter how much state you have

lgessler 6y ago

Essentially, yes, but in a way that sounds more careful than what you saw in that Fortran program. If it absolutely has to be stateful, like with a db connection for example, then there's no getting around that, but what you can do is (1) not bury it in a bunch of stateful objects that didn't have to be stateful, which can cause cascading errors when either one of them misbehaves and also makes it difficult to isolate any one stateful component for testing or re-use, and (2) use special wrappers around it that will make the statefulness easier to deal with. (For examples of (2), see Clojure atoms and the Clojure library mount: https://github.com/tolitius/mount https://github.com/tolitius/mount )

nostrademons 6y ago
More concretely, for every bit of state in the program I'm interested in two big questions, each with two subquestions: 1.) Who can access this state, and which of those accesses allow writes vs. which are read-only? 2.) What is the lifetime of that state, and what portion of that lifetime is mutable vs. read-only? Java-style OOP can control #1, but makes no distinction between reads & writes. C++ const correctness and the val/var distinction in Kotlin/Ocaml/ES6/etc. can make that distinction, but only C++ forces you to think in terms of lifetimes. Rust borrowing probably comes closest to answering all 4. I'm still waiting for a language feature that lets you say "This field of this struct is only mutable while this module is running, and once it completes it will never be changed", though. A lot of data structures are initialized in passes and then passed along as constant data: you construct the basic object structure with nothing but a few IDs, then you compute extra fields as necessary, then you're done and the object is never written to again. If you could keep the fields mutable during initialization (which may be a longer process than just constructor calls) and then seal them off once that's complete, you eliminate a whole class of bugs where a read-only client decides to mutate a field that should only be mutated from designated initializer.
- bpolverini 6y ago
  That would be really cool, but I think enforcing that is undecidable. My gut tells me that having that language feature at compile time is the same as the Halting Problem.
  - tgflynn 6y ago
    I suspect that the ideal language (at least for human use) is one that is just shy of Turing completeness. Once a program compiles and an initial configuration phase has passed all loops would be bounded and all recursions would be guaranteed to terminate. Except for the case of programs that aren't supposed to terminate for which there would be a single construct allowing wrapping everything inside one and only one infinite loop.
  - nostrademons 6y ago
    In its full generality it's undecidable, but there are probably restrictions you can build into the type system to make it decidable. I was thinking something along these lines: class Symbol { @init(constructor) originalToken: Token @init(constructor) position: SourceLocation @init(Parser.parse) scope: SymbolTable @init(Parser.parse) declaredType: Type @init(TypeChecker.typecheck) inferredType: Type @init(DataFlowAnalyzer.computeLiveness) usages: List<Expr> } And then the type system carries an extra bit of information around for whether a reference is fully-constructed or not, much like const-correctness. Fields marked with @init can only be written on a reference that's not fully-constructed. There's no compile-time enforcement for initialization order, though it'd be pretty easy to do this at runtime (convert them to null-checks or special not-initialized sentinel values). Newly-created references start out with the "initializing" bit set, but once they're returned from a function or passed into a function not in the list of legal accessors, they lose this bit unless explicitly declared. It's basically the same way "mutable" works (in languages that support it), but with a separate state bit in the type system and extra access checks within the mutating functions to make sure they only touch the fields they're declared to touch. You can fake this now by passing mutable references into initialization functions and then only using const, but it's a bit less specific because many classes are designed to be long-term mutable through 1-2 specific public APIs but also need to be mutated internally for deferred initialization, and lumping these use-cases together means that deferred fields can be touched by the public API.
tgflynn 6y ago
(2) sounds very similar to the way I have typically handled db access in C++. However the kind of state I was referring to is more about the internal computational data used by the program than about external resources.

waffletower 6y ago

You might want to consider examining another programming language for clues here. Perhaps look to a language that has a more modern approach to state such as Clojure.

tgflynn 6y ago
My understanding is that Clojure is basically lisp implemented inside the Java ecosystem. I'm not sure how that makes it "modern" given that the only higher level language older than lisp is the original version of Fortran.
- waffletower 6y ago
  Once you have some understanding of the state management facilities native to Clojure, you might be willing to concede the "modern" adjective.
- tsimionescu 6y ago
  Clojure has quite a few specific libraries. As far as I know, one of these implements a lot of persistent data structures - data structures that make it efficient to expose an API that returns a copy of the whole structure for any mutation (by sharing all of the unmodified substructure behind the scenes).
- dkersten 6y ago
  That like saying Go/Rust/Kotlin are just Algol, what’s so modern about them? Lisp isn’t a language anymore, it’s a category of languages and it’s members are rather different from each other besides some superficial syntax style. Common Lisp, Scheme and Clojure are three very different languages, certainly more different than Ruby is from Python IMHO. Clojure, in 2020, is very very different from the Lisp of 50 years ago and is modern in many respects, even compared to other popular languages.

pjmlp 6y ago

Modules with responsibility to handle the specific set of states, which are basically final classes.

tabs_masterrace 6y ago

Ideally you want to have all your state in one place, the "single point of truth", in tightly modeled data structures, keeping it as lean and immutable as possible. The program is then just a function of the state, consisting mainly of self-contained functions, that take arguments and produce a result, without causing side effects. Realistically, that doesn't mean there are no classes or objects with internal vars, but these are more of ephemeral nature, and can be created predicatively and deterministically from your centralized global state. In the best case you can have complex applications with tens of thousands LoCs that feed from just a few dozen state variables.

mhoad 6y ago
I must admit that does sound like a really nice code base to work inside of.

fish45 6y ago

ECS architecture is IMO very frictionless no matter how much state you have