7 ms·

Dynamic Scoping in C++

33 points by stettberger 6y ago

saurik 6y ago
This isn't dynamic scoping; this is just a global variable with a stack of values. I appreciate the syntax is sort of the same (though the * makes it different in a very important way) but the meaning isn't. You should at least implement this with thread local storage, though, if you are going to do this.
- jolmg 6y ago
  I think best practice among languages that support dynamic scoping is to only make use of it for global variables. As I understand it, one should only read or shadow, not modify, these variables. Since that's the case, besides the thread issue you mentioned, I'm not sure this solution is lacking. I don't know much C++, though, so I might be missing something.
  - catern 6y ago
    If you have multiple threads which try to bind the dynamically scoped variable to a new value, that should work fine and result in different values for the variable in each thread. In this implementation, the threads will corrupt the data structure and result in undefined behavior.
    - jolmg 6y ago
      Yes, that's why: > besides the thread issue you mentioned
    - masklinn 6y ago
      > If you have multiple threads which try to bind the dynamically scoped variable to a new value, that should work fine and result in different values for the variable in each thread. That would work with proper threadlocals. However with dynamic variables you'd also get this for different sub-thread stacks. Not all languages can expose that but languages with generators do e.g. in Python if you `yield` then the stack is suspended, this means a threadlocal you've changed would not be rolled back, but a dynamic variable (which Python does not have) would.
  - AnimalMuppet 6y ago
    OK, help me out here: If I'm only using it for global variables, what does the dynamic scoping do for me? Why not just use a normal global variable?
    - jolmg 6y ago
      The problem with local variables is that you can then not call a function that uses them unless you define such variables. One shouldn't be forced to know of such variables before calling such a function, since then they'd hardly be different from regular arguments. The difference with statically-scoped global variables is that you can't set them to different things in different threads, and you have to be careful to set it back to it's original value after you called the code you wanted. In other words, it's the same difference as using environment variables vs configuration files for executables.
      - AnimalMuppet 6y ago
        If I ever saw anyone I worked with trying to do something like "setting a global variable to different things in different threads", I would have very strong words for them. If they didn't listen, I'd have even stronger words for their manager: "Fire this bozo immediately". In what sane world is that considered a good idea? Is there any way in which doing so via "dynamic scoping" makes it any less of a horrible idea? I'm having trouble making sense of your first paragraph. Part of the point of local variables is that they're local. It doesn't matter whether the function you're calling uses them or not. If it uses something with the same name, it's their own copy, and you don't care whether they do or don't. If they want yours, you pass it in. What am I missing?
        jolmg 6y ago
        > If I ever saw anyone I worked with trying to do something like "setting a global variable to different things in different threads", I would have very strong words for them. Because global variables aren't used the same between the two types of languages. In lexically-scoped languages, it's bad practice overall to use global variables for anything except constants. In dynamically-scoped languages, they're used as an environment just like how environment variables are used among processes. You asked why not use "normal" (statically-scoped) global variables, and I replied on the difference. That doesn't mean that I support using them like that in such languages. Imagine environment variables didn't exist. A `su` command that modified the home directory in /etc/passwd for the duration of its subprocess to change it back when it dies would also seem pretty ugly for me. Indeed, if a language lacks dynamic scoping or environment variables didn't exist, the proper practice would be to pass the whole environment explicitly as arguments. That's what's typically done in statically-scoped languages, but it has the caveats I mentioned in this other comment: https://news.ycombinator.com/item?id=24545180 https://news.ycombinator.com/item?id=24545180 > I'm having trouble making sense of your first paragraph. Here's an example using Elisp: ; Turning off static-scoping (setq lexical-binding nil) ;; Bad practice (defun foo () bar) (foo) ;=> Debugger entered--Lisp error: (void-variable bar) (let ((bar 3)) (foo)) ;=> 3 ;; Good practice (defvar bar 2) (foo) ;=> 2 (let ((bar 3)) (foo)) ;=> 3
- kazinator 6y ago
  Thread awareness is not a required part of the description of dynamic scoping. Thread local storage does not make it absolutely re-entrant. We could move the True Scotsman goalposts even farther out and say that we appreciate that the syntax being fine, but your approach doesn't work with interrupt handlers.
  - saurik 6y ago
    The syntax sort of looks the same but it isn't the same or "fine" if what you want is "dynamic scoping" (as dynamic scoping clearly is a way to scope things, not a way to set things). Is "thread local storage" the same as "state threading"? No. Does it look vaguely similar? Sure. Is "function argument binding" the same as "currying"? No. Could you imagine the former providing many of the benefits of the latter, or being how you might implement it? I guess? I mentioned thread-local storage just because if you are going to develop this you should take that into consideration, as that's a common thing that will burn a lot of people; it was an unrelated code quality point for something you should do if you are going to do this kind of global variable stack thing. You could though use it to build something that was actually scoped by having a generic global dictionary and then keeping the names inside of the functions; at least then you are providing the core base noun of "dynamic scoping". (And of course, as someone who has spent all of their time programming in C++ coroutines for over a year now, I am well aware that the thread local storage isn't sufficient to make this trick work correctly in every case.)
- munificent 6y ago
  > This isn't dynamic scoping; this is just a global variable with a stack of values. This isn't a latte; this is just an espresso with steamed milk added to it. :) What you describe basically is how dynamic scoping is mechanically implemented under the hood.
  - masklinn 6y ago
    > What you describe basically is how dynamic scoping is mechanically implemented under the hood. It's certainly how that's commonly emulated but that can leak out e.g. CPython uses threadlocals for decimal contexts, but if you set a localcontext in a coroutine / generator and suspend that, the information leaks out. I assume the same happens with gevent unless you `patch_thread()`, and even then that assumes `decimal` always deref's threadlocals from the python-level module rather than statically resolve them.
  - dapids 6y ago
    Description !== Implementation
- brandmeyer 6y ago
  IMO, the most comprehensive solution to this mechanism is provided in Racket scheme's parameterize system. Racket's parameters are about as safe as global variables can get. https://docs.racket-lang.org/guide/parameterize.html https://docs.racket-lang.org/guide/parameterize.html What value does a forked thread get? The value at the dynamic scope of the parent at the point of thread creation. What happens if a delimited continuation is invoked by a different thread compared to the one that created the continuation? If a parameterize call was made within the continuation's delimited extent, then it moves with the continuation. If not, it'll be in the executing thread. In either case the answer is consistent: The value within the dynamic extent of the continuation is used. What happens to other threads if one overrides a parameter within its own dynamic scope? Nothing, threads don't have a dynamic scope relationship between them after thread creation.
  - soegaard 6y ago
    I agree that Racket parameters work very well. FWIW the Racket parameters are inspired by: "Processes vs. User-Level Threads in SCSH" by Martin Gasbichler and Michael Sperber https://www.researchgate.net/publication/2546137_Processes_vs_User-Level_Threads_in_SCSH https://www.researchgate.net/publication/2546137_Processes_v...
- chrisseaton 6y ago
  > This isn't dynamic scoping; this is just a global variable with a stack of values. I don't know what you think dynamic scoping is? Because 'global variable with a stack of values' is what it is.
  - tlb 6y ago
    Some implementations search up the stack for a binding. Which is slower, but works correctly with multithreading.
    - chrisseaton 6y ago
      I'm not sure what you think is the important difference between these implementations? 'Dynamic binding' is an abstract concept. The abstract concept is a global variable with a stack of values. How you implement it and whether you share the call-stack instead of a separate stack is is up to you. It's still the same dynamic scoping.
      - tlb 6y ago
        Only one of them works with multiple threads. Dynamic scoping based on save-and-restore of symbol value slots is completely incompatible with multithreading. That's why Emacs still doesn't have threads.
    - kazinator 6y ago
      The deep binding approach will not work correctly with multithreading any more correctly than shallow binding, if there is only one such stack, or any shared state whatsoever that is not atomically manipulated. It will work if there is a spaghetti stack. So that is to say, each thread extends the dynamic environment with a newly allocated frame that points upward to the parent environment. Each thread needs a thread-specific pointer to the top of its own dynamic environment chain.
      - tlb 6y ago
        What do you mean by a multithreaded runtime with only one stack?
jupp0r 6y ago
Dynamic scoping moves a bunch of correctness checks from compile time to runtime. It basically introduces all the problems that come with shared mutable state across different functions/methods. It becomes hard to reason about who mutates state where/when.
- wffurr 6y ago
  Exactly. I read this: Ergo, dynamically-scoped variables are shadowable side-channels that can influence the behavior of a function And thought that is exactly why not to use dynamic scoping. It makes every function impure by default.
DecoPerson 6y ago
I don’t see how this is functionally different to passing an object that contains references to the relate to variables; which I’ll call a context object. Practically, dynamic scoping is more confusing than context objects. void main() { int x = 2; fn(); } Does fn access or change x? You need to inspect the body of fn to know. I would call dynamic scoping a poor form of coupling. Instead of bundling your coupling wires in a neat little set of in/out arguments and a return value (the format of which only needs the function’s declaration, not its definition), you are instead reaching out of and into the function’s body, like sprawling tendrils, as your function has free pickings of your variables. It also strangely couples the names together. The outer function and the inner function may see the variable in completely different lights, yet dynamic scoping requires the outer use the name prescribed by the inner. Optimization would be hard without WPO. You’d essentially need to keep a run-time “scope” object for every function. Though, the author’s proposed design for dynamic scoping in C++ means you don’t need it for every function; however that design has its own issues: how would you optimize such a design? It would a puzzling challenge.
- hibbelig 6y ago
  Context objects need to be passed around, with dynamic scoping you do not need to. You would normally use dynamic scoping for certain global parameters that apply to a lot of operations. Unix shell scripts provide something similar to dynamic scoping with environment variables: If you write a shell script that sets LD_LIBRARY_PATH or TMPDIR, then all programs invoked from that shell script will inherit the values. And if your shell script calls another shell script, then that shell script can again set environment variables, and those are visible until it returns. I would say that environment variables have been a great success story, and folks aren't too confused.
- jolmg 6y ago
  > I don’t see how this is functionally different to passing an object that contains references to the relate to variables; which I’ll call a context object. It's the same difference as using environment variables vs command line arguments. Imagine all programs having to pass TERM, DISPLAY, HOME, etc. as arguments in case some descendant process wants to use it and the user override. Like passing TERM, DISPLAY to git in case you want to override them for the configured pager or editor. In other words, the issue is that when you have project A using project B using project C, project A has to manually carry around the context of B, and C in case the user wants to override them.
catern 6y ago
This implementation will not work with C++20 coroutines. With coroutines, implementing dynamic scope becomes a lot more interesting, because switching to different coroutines requires switching which dynamic bindings are active. The correct implementation is somewhat subtle and not immediately obvious if you haven't thought about it a lot. http://okmij.org/ftp/papers/DDBinding.pdf http://okmij.org/ftp/papers/DDBinding.pdf lays it out formally, but in the end the correct implementation is for each coroutine to have its own stack of dynamic bindings, and when you resume a coroutine in some context, you extend the bindings in that context with the coroutine's set of bindings while the coroutine is running, and remove those bindings again when the coroutine is done running. This preserves the intuitive behavior that one expects from dynamic scope - see the paper for more justification. Others have got this wrong too, so you're in good company. Python, for example, added contextvars with https://www.python.org/dev/peps/pep-0567/ https://www.python.org/dev/peps/pep-0567/, which have semantics which are usually identical to dynamic scope. But they chose an excessively-simple implementation, so the behavior diverges from proper dynamic scope when using coroutines in unusual ways, or using generators at all: https://www.python.org/dev/peps/pep-0568/ https://www.python.org/dev/peps/pep-0568/
zwieback 6y ago
I never understood the advantages of dynamic scoping. It always seems to just boil down to a worse global or thread-local variable. Is there a simple real-world example that would explain when dynamic scoping would be better than some kind of access protocol to a shared value?
- munificent 6y ago
  One of the key tenets of software engineering is encapsulation: minimizing the number of parts of the program that need to care about X for any given X. Languages have lots of different ways to encapsulate different kinds of stuff from different kinds of code. Local variables encapsulate variables from other functions. Interfaces encapsulate method bodies from invocations. OOP encapsulates state from operations that modify it. You get the idea. One pattern that most languages don't support encapsulating is this: Say you have a() which calls b() which calls c() which calls d(). d() needs to get some data from a(). The typical way to handle that is by passing that data as parameters through b() and c(), but that couples those middle-level functions to a() and d(). Any time you change the data a() needs to get to d(), you have to touch b() and c() too. You could wrap the data in some opaque "context" parameter and pass that through b() and c(). That's an OK solution and is pretty common. But a() still has to opt in to that pattern, which means b() and c() are still coupled to the choice to use any encapsulation at all. Dynamic scoping is a solution to this. a() can bind a value to a dynamic variable and d() can access it without it having to pass through b() and c() at all. It essentially gives you a side channel for parameters. A more concrete example is trees of UI components. Pretty often you have some big top level UI component that has a lot of application-level business state. Down in the leaves, you have UI components specific to that application that need that state and render it. But in between those you have a bunch of generic UI components like list views, frames, tabs views, radio button groups, that have nothing to do with your app and just visually arrange the UI. You really don't want to make a new frame widget class every time you need to pass a bit of business data through it into the thing inside the frame. So instead, what a lot of UI frameworks do is support dependency injection. A widget at the root of the tree can provide an object of some type, which makes it implicitly available to all child widgets (transivitely) of that widget. Children far down in the tree can request the object without widgets along that having to pass it along explicitly. Dependency injection is essentially a re-invention of dynamic scoping.
foota 6y ago
I understand that it could be built with libraries in some languages, but I think it would be neat for a low-level language with ecosystem wide support for call-stack context objects.
foota 6y ago
I've written enough GCL to know this can be god awful.
pierrebai 6y ago
So, dynamic scoping are global variables... except with even way way way worse unpredictable behavior. Any function from three-level remote libraries can invisibly modify the meaning of code. Sure, it allows for neat tricks, I suppose. It mostly allows impossible to diagnose error conditions since what happened actually depends on anything that may have happened before, invisibly. I find it particularly amusing since fighting off global states has been a worthy goals of languages, libraries and framework. Without it, you can say goodbye to reproducible behavior and multi-threading. (If dynamic scoping is thread-local, you still have the issue that anything can affect anything else, so nothing can be assumed to be reentrant anymore.)
- deleted 6y ago
  [deleted]
kazinator 6y ago
I implemented exactly the same thing 20 years ago. It looked something like: Dynamic<int> foo; // define at global scope { DynamicBind<int> foo; // re-bind dynamically } It used thread-local storage and all. The global constructor for the Dynamic<> template class would allocate the thread specific key. The DynamicBind<> template class did the saving, location altering, and restoring.
- stettberger 6y ago
  That is very cool! Do you have a link to that implementation? I would be very interested in the problems that arise when you want to provide a rock solid implementation of this.
stettberger 6y ago
Hi! Author of DynamicScope<T> here. Regarding threads: It is correct that the current version of the template has a problem with multi-threaded programs. However, as adding 'thread_local' to the global variable is sufficient to solve the problem, I did not mention this in the original post. However, I updated the blog post in this direction. Furthermore, I added a (run-time) check that ensures that you use DynamicScope<T> only with thread_local. Regarding Lambdas: I don't think there is a problem here. Dynamically scoped variables promise to return that value that is the most currenly bound in the current execution context. As the resolution is done on dereferencing, this is the exact behavior that DynamicScope<T> provides. This means that a lambda does not (lexically) catch the value of the dynamically-scope variable at definition time, but at the execution time of the lambda.

33 points by stettberger 6y ago

This isn't dynamic scoping; this is just a global variable with a stack of values. I appreciate the syntax is sort of the same (though the * makes it different in a very important way) but the meaning isn't. You should at least implement this with thread local storage, though, if you are going to do this.

jolmg 6y ago
I think best practice among languages that support dynamic scoping is to only make use of it for global variables. As I understand it, one should only read or shadow, not modify, these variables. Since that's the case, besides the thread issue you mentioned, I'm not sure this solution is lacking. I don't know much C++, though, so I might be missing something.
- catern 6y ago
  If you have multiple threads which try to bind the dynamically scoped variable to a new value, that should work fine and result in different values for the variable in each thread. In this implementation, the threads will corrupt the data structure and result in undefined behavior.
  - jolmg 6y ago
    Yes, that's why: > besides the thread issue you mentioned
  - masklinn 6y ago
    > If you have multiple threads which try to bind the dynamically scoped variable to a new value, that should work fine and result in different values for the variable in each thread. That would work with proper threadlocals. However with dynamic variables you'd also get this for different sub-thread stacks. Not all languages can expose that but languages with generators do e.g. in Python if you `yield` then the stack is suspended, this means a threadlocal you've changed would not be rolled back, but a dynamic variable (which Python does not have) would.
- AnimalMuppet 6y ago
  OK, help me out here: If I'm only using it for global variables, what does the dynamic scoping do for me? Why not just use a normal global variable?
  - jolmg 6y ago
    The problem with local variables is that you can then not call a function that uses them unless you define such variables. One shouldn't be forced to know of such variables before calling such a function, since then they'd hardly be different from regular arguments. The difference with statically-scoped global variables is that you can't set them to different things in different threads, and you have to be careful to set it back to it's original value after you called the code you wanted. In other words, it's the same difference as using environment variables vs configuration files for executables.
    - AnimalMuppet 6y ago
      If I ever saw anyone I worked with trying to do something like "setting a global variable to different things in different threads", I would have very strong words for them. If they didn't listen, I'd have even stronger words for their manager: "Fire this bozo immediately". In what sane world is that considered a good idea? Is there any way in which doing so via "dynamic scoping" makes it any less of a horrible idea? I'm having trouble making sense of your first paragraph. Part of the point of local variables is that they're local. It doesn't matter whether the function you're calling uses them or not. If it uses something with the same name, it's their own copy, and you don't care whether they do or don't. If they want yours, you pass it in. What am I missing?
      - jolmg 6y ago
        > If I ever saw anyone I worked with trying to do something like "setting a global variable to different things in different threads", I would have very strong words for them. Because global variables aren't used the same between the two types of languages. In lexically-scoped languages, it's bad practice overall to use global variables for anything except constants. In dynamically-scoped languages, they're used as an environment just like how environment variables are used among processes. You asked why not use "normal" (statically-scoped) global variables, and I replied on the difference. That doesn't mean that I support using them like that in such languages. Imagine environment variables didn't exist. A `su` command that modified the home directory in /etc/passwd for the duration of its subprocess to change it back when it dies would also seem pretty ugly for me. Indeed, if a language lacks dynamic scoping or environment variables didn't exist, the proper practice would be to pass the whole environment explicitly as arguments. That's what's typically done in statically-scoped languages, but it has the caveats I mentioned in this other comment: https://news.ycombinator.com/item?id=24545180 https://news.ycombinator.com/item?id=24545180 > I'm having trouble making sense of your first paragraph. Here's an example using Elisp: ; Turning off static-scoping (setq lexical-binding nil) ;; Bad practice (defun foo () bar) (foo) ;=> Debugger entered--Lisp error: (void-variable bar) (let ((bar 3)) (foo)) ;=> 3 ;; Good practice (defvar bar 2) (foo) ;=> 2 (let ((bar 3)) (foo)) ;=> 3
kazinator 6y ago
Thread awareness is not a required part of the description of dynamic scoping. Thread local storage does not make it absolutely re-entrant. We could move the True Scotsman goalposts even farther out and say that we appreciate that the syntax being fine, but your approach doesn't work with interrupt handlers.
- saurik 6y ago
  The syntax sort of looks the same but it isn't the same or "fine" if what you want is "dynamic scoping" (as dynamic scoping clearly is a way to scope things, not a way to set things). Is "thread local storage" the same as "state threading"? No. Does it look vaguely similar? Sure. Is "function argument binding" the same as "currying"? No. Could you imagine the former providing many of the benefits of the latter, or being how you might implement it? I guess? I mentioned thread-local storage just because if you are going to develop this you should take that into consideration, as that's a common thing that will burn a lot of people; it was an unrelated code quality point for something you should do if you are going to do this kind of global variable stack thing. You could though use it to build something that was actually scoped by having a generic global dictionary and then keeping the names inside of the functions; at least then you are providing the core base noun of "dynamic scoping". (And of course, as someone who has spent all of their time programming in C++ coroutines for over a year now, I am well aware that the thread local storage isn't sufficient to make this trick work correctly in every case.)
munificent 6y ago
> This isn't dynamic scoping; this is just a global variable with a stack of values. This isn't a latte; this is just an espresso with steamed milk added to it. :) What you describe basically is how dynamic scoping is mechanically implemented under the hood.
- masklinn 6y ago
  > What you describe basically is how dynamic scoping is mechanically implemented under the hood. It's certainly how that's commonly emulated but that can leak out e.g. CPython uses threadlocals for decimal contexts, but if you set a localcontext in a coroutine / generator and suspend that, the information leaks out. I assume the same happens with gevent unless you `patch_thread()`, and even then that assumes `decimal` always deref's threadlocals from the python-level module rather than statically resolve them.
- dapids 6y ago
  Description !== Implementation
brandmeyer 6y ago
IMO, the most comprehensive solution to this mechanism is provided in Racket scheme's parameterize system. Racket's parameters are about as safe as global variables can get. https://docs.racket-lang.org/guide/parameterize.html https://docs.racket-lang.org/guide/parameterize.html What value does a forked thread get? The value at the dynamic scope of the parent at the point of thread creation. What happens if a delimited continuation is invoked by a different thread compared to the one that created the continuation? If a parameterize call was made within the continuation's delimited extent, then it moves with the continuation. If not, it'll be in the executing thread. In either case the answer is consistent: The value within the dynamic extent of the continuation is used. What happens to other threads if one overrides a parameter within its own dynamic scope? Nothing, threads don't have a dynamic scope relationship between them after thread creation.
- soegaard 6y ago
  I agree that Racket parameters work very well. FWIW the Racket parameters are inspired by: "Processes vs. User-Level Threads in SCSH" by Martin Gasbichler and Michael Sperber https://www.researchgate.net/publication/2546137_Processes_vs_User-Level_Threads_in_SCSH https://www.researchgate.net/publication/2546137_Processes_v...
chrisseaton 6y ago
> This isn't dynamic scoping; this is just a global variable with a stack of values. I don't know what you think dynamic scoping is? Because 'global variable with a stack of values' is what it is.
- tlb 6y ago
  Some implementations search up the stack for a binding. Which is slower, but works correctly with multithreading.
  - chrisseaton 6y ago
    I'm not sure what you think is the important difference between these implementations? 'Dynamic binding' is an abstract concept. The abstract concept is a global variable with a stack of values. How you implement it and whether you share the call-stack instead of a separate stack is is up to you. It's still the same dynamic scoping.
    - tlb 6y ago
      Only one of them works with multiple threads. Dynamic scoping based on save-and-restore of symbol value slots is completely incompatible with multithreading. That's why Emacs still doesn't have threads.
  - kazinator 6y ago
    The deep binding approach will not work correctly with multithreading any more correctly than shallow binding, if there is only one such stack, or any shared state whatsoever that is not atomically manipulated. It will work if there is a spaghetti stack. So that is to say, each thread extends the dynamic environment with a newly allocated frame that points upward to the parent environment. Each thread needs a thread-specific pointer to the top of its own dynamic environment chain.
    - tlb 6y ago
      What do you mean by a multithreaded runtime with only one stack?

jupp0r 6y ago

Dynamic scoping moves a bunch of correctness checks from compile time to runtime. It basically introduces all the problems that come with shared mutable state across different functions/methods. It becomes hard to reason about who mutates state where/when.

wffurr 6y ago
Exactly. I read this: Ergo, dynamically-scoped variables are shadowable side-channels that can influence the behavior of a function And thought that is exactly why not to use dynamic scoping. It makes every function impure by default.

DecoPerson 6y ago

I don’t see how this is functionally different to passing an object that contains references to the relate to variables; which I’ll call a context object. Practically, dynamic scoping is more confusing than context objects. void main() { int x = 2; fn(); } Does fn access or change x? You need to inspect the body of fn to know. I would call dynamic scoping a poor form of coupling. Instead of bundling your coupling wires in a neat little set of in/out arguments and a return value (the format of which only needs the function’s declaration, not its definition), you are instead reaching out of and into the function’s body, like sprawling tendrils, as your function has free pickings of your variables. It also strangely couples the names together. The outer function and the inner function may see the variable in completely different lights, yet dynamic scoping requires the outer use the name prescribed by the inner. Optimization would be hard without WPO. You’d essentially need to keep a run-time “scope” object for every function. Though, the author’s proposed design for dynamic scoping in C++ means you don’t need it for every function; however that design has its own issues: how would you optimize such a design? It would a puzzling challenge.

hibbelig 6y ago
Context objects need to be passed around, with dynamic scoping you do not need to. You would normally use dynamic scoping for certain global parameters that apply to a lot of operations. Unix shell scripts provide something similar to dynamic scoping with environment variables: If you write a shell script that sets LD_LIBRARY_PATH or TMPDIR, then all programs invoked from that shell script will inherit the values. And if your shell script calls another shell script, then that shell script can again set environment variables, and those are visible until it returns. I would say that environment variables have been a great success story, and folks aren't too confused.
jolmg 6y ago
> I don’t see how this is functionally different to passing an object that contains references to the relate to variables; which I’ll call a context object. It's the same difference as using environment variables vs command line arguments. Imagine all programs having to pass TERM, DISPLAY, HOME, etc. as arguments in case some descendant process wants to use it and the user override. Like passing TERM, DISPLAY to git in case you want to override them for the configured pager or editor. In other words, the issue is that when you have project A using project B using project C, project A has to manually carry around the context of B, and C in case the user wants to override them.

catern 6y ago

This implementation will not work with C++20 coroutines. With coroutines, implementing dynamic scope becomes a lot more interesting, because switching to different coroutines requires switching which dynamic bindings are active. The correct implementation is somewhat subtle and not immediately obvious if you haven't thought about it a lot. http://okmij.org/ftp/papers/DDBinding.pdf http://okmij.org/ftp/papers/DDBinding.pdf lays it out formally, but in the end the correct implementation is for each coroutine to have its own stack of dynamic bindings, and when you resume a coroutine in some context, you extend the bindings in that context with the coroutine's set of bindings while the coroutine is running, and remove those bindings again when the coroutine is done running. This preserves the intuitive behavior that one expects from dynamic scope - see the paper for more justification. Others have got this wrong too, so you're in good company. Python, for example, added contextvars with https://www.python.org/dev/peps/pep-0567/ https://www.python.org/dev/peps/pep-0567/, which have semantics which are usually identical to dynamic scope. But they chose an excessively-simple implementation, so the behavior diverges from proper dynamic scope when using coroutines in unusual ways, or using generators at all: https://www.python.org/dev/peps/pep-0568/ https://www.python.org/dev/peps/pep-0568/

zwieback 6y ago

I never understood the advantages of dynamic scoping. It always seems to just boil down to a worse global or thread-local variable. Is there a simple real-world example that would explain when dynamic scoping would be better than some kind of access protocol to a shared value?

munificent 6y ago
One of the key tenets of software engineering is encapsulation: minimizing the number of parts of the program that need to care about X for any given X. Languages have lots of different ways to encapsulate different kinds of stuff from different kinds of code. Local variables encapsulate variables from other functions. Interfaces encapsulate method bodies from invocations. OOP encapsulates state from operations that modify it. You get the idea. One pattern that most languages don't support encapsulating is this: Say you have a() which calls b() which calls c() which calls d(). d() needs to get some data from a(). The typical way to handle that is by passing that data as parameters through b() and c(), but that couples those middle-level functions to a() and d(). Any time you change the data a() needs to get to d(), you have to touch b() and c() too. You could wrap the data in some opaque "context" parameter and pass that through b() and c(). That's an OK solution and is pretty common. But a() still has to opt in to that pattern, which means b() and c() are still coupled to the choice to use any encapsulation at all. Dynamic scoping is a solution to this. a() can bind a value to a dynamic variable and d() can access it without it having to pass through b() and c() at all. It essentially gives you a side channel for parameters. A more concrete example is trees of UI components. Pretty often you have some big top level UI component that has a lot of application-level business state. Down in the leaves, you have UI components specific to that application that need that state and render it. But in between those you have a bunch of generic UI components like list views, frames, tabs views, radio button groups, that have nothing to do with your app and just visually arrange the UI. You really don't want to make a new frame widget class every time you need to pass a bit of business data through it into the thing inside the frame. So instead, what a lot of UI frameworks do is support dependency injection. A widget at the root of the tree can provide an object of some type, which makes it implicitly available to all child widgets (transivitely) of that widget. Children far down in the tree can request the object without widgets along that having to pass it along explicitly. Dependency injection is essentially a re-invention of dynamic scoping.

foota 6y ago

I understand that it could be built with libraries in some languages, but I think it would be neat for a low-level language with ecosystem wide support for call-stack context objects.

foota 6y ago

I've written enough GCL to know this can be god awful.

pierrebai 6y ago

So, dynamic scoping are global variables... except with even way way way worse unpredictable behavior. Any function from three-level remote libraries can invisibly modify the meaning of code. Sure, it allows for neat tricks, I suppose. It mostly allows impossible to diagnose error conditions since what happened actually depends on anything that may have happened before, invisibly. I find it particularly amusing since fighting off global states has been a worthy goals of languages, libraries and framework. Without it, you can say goodbye to reproducible behavior and multi-threading. (If dynamic scoping is thread-local, you still have the issue that anything can affect anything else, so nothing can be assumed to be reentrant anymore.)

deleted 6y ago
[deleted]

kazinator 6y ago

I implemented exactly the same thing 20 years ago. It looked something like: Dynamic<int> foo; // define at global scope { DynamicBind<int> foo; // re-bind dynamically } It used thread-local storage and all. The global constructor for the Dynamic<> template class would allocate the thread specific key. The DynamicBind<> template class did the saving, location altering, and restoring.

stettberger 6y ago
That is very cool! Do you have a link to that implementation? I would be very interested in the problems that arise when you want to provide a rock solid implementation of this.

stettberger 6y ago

Hi! Author of DynamicScope<T> here. Regarding threads: It is correct that the current version of the template has a problem with multi-threaded programs. However, as adding 'thread_local' to the global variable is sufficient to solve the problem, I did not mention this in the original post. However, I updated the blog post in this direction. Furthermore, I added a (run-time) check that ensures that you use DynamicScope<T> only with thread_local. Regarding Lambdas: I don't think there is a problem here. Dynamically scoped variables promise to return that value that is the most currenly bound in the current execution context. As the resolution is done on dereferencing, this is the exact behavior that DynamicScope<T> provides. This means that a lambda does not (lexically) catch the value of the dynamically-scope variable at definition time, but at the execution time of the lambda.