18 ms·

Rust – How do I create a global, mutable singleton? (2015)

53 points by gitgud 6y ago

bluejekyll 6y ago
Don’t, is always the simplest answer. Global state that is non-constant, causes all sorts of testing issues down the line. It should be avoided. Edit: since a lot of folks are reading this as saying “never” rather than “avoided”. The point here is to keep access to data and state restricted. The entire application does not need to know that some piece of data is global. If you design a central container in a generic manner, it can restrict access to globals in a way that allows for easier refactoring in the future. It also allows one to design users of those globals in a way that doesn’t leak the fact that these things are global. By doing this, you don’t leak globals through the application, and you keep things isolated in a way that allows for easier testing.
- aphextron 6y ago
  It's not a simple yes/no question. For instance, is a database not ultimately a single mutable global state?
  - baq 6y ago
    A database connection perhaps, but different connections to the same database will see different states of the database depending on the isolation level of the current transaction.
  - chousuke 6y ago
    Databases are also designed for safe concurrent access with enforceable constraints on data. Would you like your application behaviour to depend on a transient "database" someone made up on the fly? :) I think the key is that your database stores data that lives outside your application, making it observable outside any particular application instance and allowing you to perform ad-hoc queries to validate consistency and fix problems. If you have such mutable state inside a process, it can be very difficult to debug problems when a series of interactions sends your process into an inconsistent state that is only detected some hours later when (if) it finally crashes.
- _28jh 6y ago
  I hate when I ask something about globals on SO and I get the same blanket answer. Why does every programmer feel the need to tell me that "globals make it hard to reason about the state of your program?" Unfortunately most systems are global based. OpenGL has one global state. A database connection is a global. Either you can use globals to model these things properly or you can complicate it by not using globals. The library will be using globals under the hood anyways.
  - swsieber 6y ago
    The OpenGL is a much better example than the database connection, FWIW.
  - logicslave 6y ago
    Its dogma. Like a religion. A whole generation of people believing in things with no root evidence except that the knowledge is ubiquitous
    - bluejekyll 6y ago
      I have plenty of history with global state, various contexts in large applications, that make testing code unnecessarily complex. So, yes, global state is sometimes necessary, but it absolutely should be avoided, and access to it should be restricted.
    - deleted 6y ago
      [deleted]
  - AnIdiotOnTheNet 6y ago
    Because that's the academic answer that doesn't have to deal with real world constraints and pressures. I too have a deep hatred of the "you don't want to do that" answer for this reason: I do deal with real world constraints and pressures and I'm asking my question because "the right way" can't be done in my circumstances!
  - cesarb 6y ago
    > OpenGL has one global state. As far as I know, it doesn't, it's actually thread-local state; you can have multiple states, and you use glXMakeCurrent/eglMakeCurrent to choose which one is active for the current thread. And the protocol which can be considered the successor of OpenGL (Vulkan) replaced the thread-local state with an explicit reference to the state. > A database connection is a global. The product I develop at work uses multiple database connections, to several different databases, which can be added and removed at runtime. They most definitely aren't global.
- dleslie 6y ago
  The hardware itself is implicit global state, and often with devices that behave like singletons.
- kjeetgill 6y ago
  This is very nearly the exact advice I would give a person working on my team. But I also think when the question is asked broadly it can be ... Presumptuous. Even though I've come to the same conclusions from my experience just discarding the premise of the question (without answering it especially) won't sit will.
- Const-me 6y ago
  The OP’s question was about OpenGL. GPUs have gigabytes of external mutable state. Specifically, in my PC the figure is “11GB of that state”. You can’t avoid that. If a language doesn’t offer convenient ways to deal with that state, it simply means that language ain’t a good choice for anything GPU-related, be it graphics or GPGPU.
  - bluejekyll 6y ago
    What do you by convenient? There is an extremely good response to how to design this in Rust on the SO post. This doesn't change the way that you should design the rest of the code. The convenience that Rust gives you in this context is strong ownership (i.e. can limit access to the Global except in very desirable patterns) and data-race free with compile time checks around mutability. The standard library has all of these options available, additionally there are a bunch of libraries from the ecosystem that make it even easier.
    - Const-me 6y ago
      > can limit access to the Global except in very desirable patterns It’s desirable when you don’t do much I/O, only implementing some algorithms that compute something. It’s not desirable at all when the majority of your complexity is I/O. An extreme example is a videogame, these can update global mutable state (the VRAM, abstracted behind GL or D3D) with the bandwidth measured in gigabytes/second, often concurrently so from multiple threads. Update: and even for computations, when you want performance, rust’s ownership shenanigans often get in the way. See this C++ code for example: https://github.com/Const-me/SimdPrecisionTest/blob/master/rcppsErrorSummary.cpp https://github.com/Const-me/SimdPrecisionTest/blob/master/rc... You gonna have hard time porting that to Rust, because this vector https://github.com/Const-me/SimdPrecisionTest/blob/master/rcppsErrorSummary.cpp#L212 https://github.com/Const-me/SimdPrecisionTest/blob/master/rc... is updated concurrently by multiple threads of the pool, without any locks.
      - bluejekyll 6y ago
        You're comparing an implicitly unsafe language implementation, C++, with a language that guarantees safety by default, Rust. There are many people working on non-blocking lock-free datastructures in Rust, but this often requires unsafe implementations. Essentially removing the safety restrictions to do things like what you're showing in the C++. Looking at the code you're showing, I might model this with deque from the Crossbeam project: https://docs.rs/crossbeam/0.7.3/crossbeam/deque/index.html https://docs.rs/crossbeam/0.7.3/crossbeam/deque/index.html I believe that's all lock-free, and offers a safe interface for working with it.
eden_h 6y ago
Ran into this issue with a crate that uses CUDA in Rust - how do you hang onto the pointer to CUBLAS/CUDNN so that all CUDA functions use the same pointers and also it goes out of scope at the end of execution. The big problem is the second part - lazy static doesn't allow you to run destructors, and this is one of the few times you really need a custom destructor to tell Rust to kill the CUBLAS/CUDNN pointers at appropriate times. Ended up connecting it to a struct so that the destructor could be tied to the object rather than anything else, but lazystatic's (lack of) story around destructors definitely created a bunch of problems that weren't apparent until we started finding memory leaks.
echelon 6y ago
Stop caring about OO patterns. Inheritance and the stuff that came with it during that era was a boondoggle. Arc<RwLock<T>> fits the bill. Just pass it around. Additional plumbing isn't the end of the world.
- Fiahil 6y ago
  In most cases, additional plumbing is not even necessary, as it's often tied to a single object instantiated in the main function (database client, connection pool, config map, ...). The Arc<RwLock<T>> would go in that object.
- eximius 6y ago
  Why is Arc needed? It's a global singleton so we don't need to worry about dropping it. Just a RwLock<T> should be fine or maybe even Cell if you don't need RwLock guarantees.
  - Fiahil 6y ago
    Arc is there if you need to send that singleton between threads. If your program is a single-thread non-async one, you can drop both Arc and Mutex, leaving you with just a regular data field :)
    - steveklabnik 6y ago
      If you really need something that's only for a single thread, then using TLS is gonna be better than a static. (I actually want to write a blog post about this... haven't done so yet though. People reach for static too often and thread_local! too little, IMHO. Of course, not needing either is best.)
      - bluejekyll 6y ago
        TLS has many of the same downsides of most global state. It does remove the concurrent access issues (as long as it is only used inside a single thread), but it still creates a testing problem if access to it is not restricted. That is, don’t make globals publicly accessible, just like you wouldn’t make all data in an object publicly accessible.
pornel 6y ago
Rust doesn't accept "but my program is single-threaded!" as an answer. The language doesn't believe you. Every global has to be thread-safe, or marked as `unsafe` to use.
- Rusky 6y ago
  The "my program is single-threaded" button is thread-local storage. If your program truly is single-threaded, then TLS is the same thing as a global!
deleted 6y ago
[deleted]
deleted 6y ago
[deleted]
gutino 6y ago
A legit singletons/global states use case are for WebAssembly bindings. So stop trolling you purists.
- wnoise 6y ago
  Can you explain how and why it's necessary?
richard_todd 6y ago
I would love to play with a programming language that has a notion of an initialization phase, where constraints are relaxed. So often, especially in cli programs, you just want to set up some ambient state based on arguments or whatever that will be constant after you set it up. After setup, then it might be read by multiple threads, etc, and it's totally safe, except you could never convince a language like Rust that it is. Even a series of one-time copies into global "constants" would be sufficient to do the kinds of things I'm thinking about... require it to happen from `main` and (harder to do...) maybe don't let the user create threads until the switch is flipped. But don't put run-time checks on access or whatever... make it truly global and constant once initialization is over.
- heinrich5991 6y ago
  You can do lazy_static! initialisation in Rust. > And it's totally safe, except you could never convince a language like Rust that it is. I think you should be able to create a safe abstraction around that in Rust.
  - richard_todd 6y ago
    Yeah, I've seen lazy_static. From the documenation [1]: > The Deref implementation uses a hidden static variable that is guarded by an atomic check on each access. That's totally unnecessary in the scenarios I'm talking about, but it's needed to make Rust happy. Also, it's not totally clear to me that I can feed a lazy_static information derived from main() like command-line arguments, though I did not try too hard to figure that part out. I just wound up plumbing arguments through all my functions. [1]: https://docs.rs/lazy_static/1.4.0/lazy_static/ https://docs.rs/lazy_static/1.4.0/lazy_static/
    - ChrisSD 6y ago
      You want OnceCell's unsync type for that: https://docs.rs/once_cell/1.4.0/once_cell/unsync/struct.OnceCell.html https://docs.rs/once_cell/1.4.0/once_cell/unsync/struct.Once... Assuming you're dead set against thread local storage.
jupp0r 6y ago
One much better solution than using any globals is passing an opaque context object through the call chain, much in the way many Go libraries do it. This allows you to scope what APIs have access to what parts of the context and prevents its implementation details to be splattered all over the place. It also allows for mocking parts of the runtime in unit tests and makes synchronization across threads an implementation detail of the APIs that downcast the context (as opposed to Arc<RwLock<T>>, which is not exactly a pleasure to use). Once you do pass context everywhere, it's easy to add new things to it without incurring the cost of adding a new method parameter. Of course this can be (and has been) abused and you need to use it carefully and as little as possible. I still think it's orders of magnitude better than mutable global variables.
luizfelberti 6y ago
Kinda off-topic, but can anyone explain to me why a good answer with 100+ upvotes is burried behind a "me too" answer with a score of -2, and why does StackOverflow consistently do this?
- mekoka 6y ago
  There's a setting to order answers based on votes in the page.

53 points by gitgud 6y ago

Don’t, is always the simplest answer. Global state that is non-constant, causes all sorts of testing issues down the line. It should be avoided. Edit: since a lot of folks are reading this as saying “never” rather than “avoided”. The point here is to keep access to data and state restricted. The entire application does not need to know that some piece of data is global. If you design a central container in a generic manner, it can restrict access to globals in a way that allows for easier refactoring in the future. It also allows one to design users of those globals in a way that doesn’t leak the fact that these things are global. By doing this, you don’t leak globals through the application, and you keep things isolated in a way that allows for easier testing.

aphextron 6y ago
It's not a simple yes/no question. For instance, is a database not ultimately a single mutable global state?
- baq 6y ago
  A database connection perhaps, but different connections to the same database will see different states of the database depending on the isolation level of the current transaction.
- chousuke 6y ago
  Databases are also designed for safe concurrent access with enforceable constraints on data. Would you like your application behaviour to depend on a transient "database" someone made up on the fly? :) I think the key is that your database stores data that lives outside your application, making it observable outside any particular application instance and allowing you to perform ad-hoc queries to validate consistency and fix problems. If you have such mutable state inside a process, it can be very difficult to debug problems when a series of interactions sends your process into an inconsistent state that is only detected some hours later when (if) it finally crashes.
_28jh 6y ago
I hate when I ask something about globals on SO and I get the same blanket answer. Why does every programmer feel the need to tell me that "globals make it hard to reason about the state of your program?" Unfortunately most systems are global based. OpenGL has one global state. A database connection is a global. Either you can use globals to model these things properly or you can complicate it by not using globals. The library will be using globals under the hood anyways.
- swsieber 6y ago
  The OpenGL is a much better example than the database connection, FWIW.
- logicslave 6y ago
  Its dogma. Like a religion. A whole generation of people believing in things with no root evidence except that the knowledge is ubiquitous
  - bluejekyll 6y ago
    I have plenty of history with global state, various contexts in large applications, that make testing code unnecessarily complex. So, yes, global state is sometimes necessary, but it absolutely should be avoided, and access to it should be restricted.
  - deleted 6y ago
    [deleted]
- AnIdiotOnTheNet 6y ago
  Because that's the academic answer that doesn't have to deal with real world constraints and pressures. I too have a deep hatred of the "you don't want to do that" answer for this reason: I do deal with real world constraints and pressures and I'm asking my question because "the right way" can't be done in my circumstances!
- cesarb 6y ago
  > OpenGL has one global state. As far as I know, it doesn't, it's actually thread-local state; you can have multiple states, and you use glXMakeCurrent/eglMakeCurrent to choose which one is active for the current thread. And the protocol which can be considered the successor of OpenGL (Vulkan) replaced the thread-local state with an explicit reference to the state. > A database connection is a global. The product I develop at work uses multiple database connections, to several different databases, which can be added and removed at runtime. They most definitely aren't global.
dleslie 6y ago
The hardware itself is implicit global state, and often with devices that behave like singletons.
kjeetgill 6y ago
This is very nearly the exact advice I would give a person working on my team. But I also think when the question is asked broadly it can be ... Presumptuous. Even though I've come to the same conclusions from my experience just discarding the premise of the question (without answering it especially) won't sit will.
Const-me 6y ago
The OP’s question was about OpenGL. GPUs have gigabytes of external mutable state. Specifically, in my PC the figure is “11GB of that state”. You can’t avoid that. If a language doesn’t offer convenient ways to deal with that state, it simply means that language ain’t a good choice for anything GPU-related, be it graphics or GPGPU.
- bluejekyll 6y ago
  What do you by convenient? There is an extremely good response to how to design this in Rust on the SO post. This doesn't change the way that you should design the rest of the code. The convenience that Rust gives you in this context is strong ownership (i.e. can limit access to the Global except in very desirable patterns) and data-race free with compile time checks around mutability. The standard library has all of these options available, additionally there are a bunch of libraries from the ecosystem that make it even easier.
  - Const-me 6y ago
    > can limit access to the Global except in very desirable patterns It’s desirable when you don’t do much I/O, only implementing some algorithms that compute something. It’s not desirable at all when the majority of your complexity is I/O. An extreme example is a videogame, these can update global mutable state (the VRAM, abstracted behind GL or D3D) with the bandwidth measured in gigabytes/second, often concurrently so from multiple threads. Update: and even for computations, when you want performance, rust’s ownership shenanigans often get in the way. See this C++ code for example: https://github.com/Const-me/SimdPrecisionTest/blob/master/rcppsErrorSummary.cpp https://github.com/Const-me/SimdPrecisionTest/blob/master/rc... You gonna have hard time porting that to Rust, because this vector https://github.com/Const-me/SimdPrecisionTest/blob/master/rcppsErrorSummary.cpp#L212 https://github.com/Const-me/SimdPrecisionTest/blob/master/rc... is updated concurrently by multiple threads of the pool, without any locks.
    - bluejekyll 6y ago
      You're comparing an implicitly unsafe language implementation, C++, with a language that guarantees safety by default, Rust. There are many people working on non-blocking lock-free datastructures in Rust, but this often requires unsafe implementations. Essentially removing the safety restrictions to do things like what you're showing in the C++. Looking at the code you're showing, I might model this with deque from the Crossbeam project: https://docs.rs/crossbeam/0.7.3/crossbeam/deque/index.html https://docs.rs/crossbeam/0.7.3/crossbeam/deque/index.html I believe that's all lock-free, and offers a safe interface for working with it.

eden_h 6y ago

Ran into this issue with a crate that uses CUDA in Rust - how do you hang onto the pointer to CUBLAS/CUDNN so that all CUDA functions use the same pointers and also it goes out of scope at the end of execution. The big problem is the second part - lazy static doesn't allow you to run destructors, and this is one of the few times you really need a custom destructor to tell Rust to kill the CUBLAS/CUDNN pointers at appropriate times. Ended up connecting it to a struct so that the destructor could be tied to the object rather than anything else, but lazystatic's (lack of) story around destructors definitely created a bunch of problems that weren't apparent until we started finding memory leaks.

echelon 6y ago

Stop caring about OO patterns. Inheritance and the stuff that came with it during that era was a boondoggle. Arc<RwLock<T>> fits the bill. Just pass it around. Additional plumbing isn't the end of the world.

Fiahil 6y ago
In most cases, additional plumbing is not even necessary, as it's often tied to a single object instantiated in the main function (database client, connection pool, config map, ...). The Arc<RwLock<T>> would go in that object.
eximius 6y ago
Why is Arc needed? It's a global singleton so we don't need to worry about dropping it. Just a RwLock<T> should be fine or maybe even Cell if you don't need RwLock guarantees.
- Fiahil 6y ago
  Arc is there if you need to send that singleton between threads. If your program is a single-thread non-async one, you can drop both Arc and Mutex, leaving you with just a regular data field :)
  - steveklabnik 6y ago
    If you really need something that's only for a single thread, then using TLS is gonna be better than a static. (I actually want to write a blog post about this... haven't done so yet though. People reach for static too often and thread_local! too little, IMHO. Of course, not needing either is best.)
    - bluejekyll 6y ago
      TLS has many of the same downsides of most global state. It does remove the concurrent access issues (as long as it is only used inside a single thread), but it still creates a testing problem if access to it is not restricted. That is, don’t make globals publicly accessible, just like you wouldn’t make all data in an object publicly accessible.

pornel 6y ago

Rust doesn't accept "but my program is single-threaded!" as an answer. The language doesn't believe you. Every global has to be thread-safe, or marked as `unsafe` to use.

Rusky 6y ago
The "my program is single-threaded" button is thread-local storage. If your program truly is single-threaded, then TLS is the same thing as a global!

deleted 6y ago

[deleted]

deleted 6y ago

[deleted]

gutino 6y ago

A legit singletons/global states use case are for WebAssembly bindings. So stop trolling you purists.

wnoise 6y ago
Can you explain how and why it's necessary?

richard_todd 6y ago

I would love to play with a programming language that has a notion of an initialization phase, where constraints are relaxed. So often, especially in cli programs, you just want to set up some ambient state based on arguments or whatever that will be constant after you set it up. After setup, then it might be read by multiple threads, etc, and it's totally safe, except you could never convince a language like Rust that it is. Even a series of one-time copies into global "constants" would be sufficient to do the kinds of things I'm thinking about... require it to happen from `main` and (harder to do...) maybe don't let the user create threads until the switch is flipped. But don't put run-time checks on access or whatever... make it truly global and constant once initialization is over.

heinrich5991 6y ago
You can do lazy_static! initialisation in Rust. > And it's totally safe, except you could never convince a language like Rust that it is. I think you should be able to create a safe abstraction around that in Rust.
- richard_todd 6y ago
  Yeah, I've seen lazy_static. From the documenation [1]: > The Deref implementation uses a hidden static variable that is guarded by an atomic check on each access. That's totally unnecessary in the scenarios I'm talking about, but it's needed to make Rust happy. Also, it's not totally clear to me that I can feed a lazy_static information derived from main() like command-line arguments, though I did not try too hard to figure that part out. I just wound up plumbing arguments through all my functions. [1]: https://docs.rs/lazy_static/1.4.0/lazy_static/ https://docs.rs/lazy_static/1.4.0/lazy_static/
  - ChrisSD 6y ago
    You want OnceCell's unsync type for that: https://docs.rs/once_cell/1.4.0/once_cell/unsync/struct.OnceCell.html https://docs.rs/once_cell/1.4.0/once_cell/unsync/struct.Once... Assuming you're dead set against thread local storage.

jupp0r 6y ago

One much better solution than using any globals is passing an opaque context object through the call chain, much in the way many Go libraries do it. This allows you to scope what APIs have access to what parts of the context and prevents its implementation details to be splattered all over the place. It also allows for mocking parts of the runtime in unit tests and makes synchronization across threads an implementation detail of the APIs that downcast the context (as opposed to Arc<RwLock<T>>, which is not exactly a pleasure to use). Once you do pass context everywhere, it's easy to add new things to it without incurring the cost of adding a new method parameter. Of course this can be (and has been) abused and you need to use it carefully and as little as possible. I still think it's orders of magnitude better than mutable global variables.

luizfelberti 6y ago

Kinda off-topic, but can anyone explain to me why a good answer with 100+ upvotes is burried behind a "me too" answer with a score of -2, and why does StackOverflow consistently do this?

mekoka 6y ago
There's a setting to order answers based on votes in the page.