5 ms·

hmm my first thought its, having to do that is a lot like c and cleaning up my own allocations. This feels like something rust should automatically do for me?

ReaLNero 6y ago
In C, if you forget to clean up, you have a memory leak which is hard to track down. In Rust, if you don't do this, you're not sacrificing memory leaks, only performance. A profiler can tell you when you should drop asynchronously.
- madmax96 6y ago
  >A profiler can tell you when you should drop asynchronously Is there any profiler that does this today? What are the drawbacks with asynchronous drops?
  - ehsanu1 6y ago
    See some discussion here: https://www.reddit.com/r/rust/comments/gntv7l/dropping_heavy_objects_in_another_thread_can_make/ https://www.reddit.com/r/rust/comments/gntv7l/dropping_heavy...
    - deleted 6y ago
      [deleted]
  - MaulingMonkey 6y ago
    > Is there any profiler that does this today? It requires interpretation, but yes. > What are the drawbacks with asynchronous drops? You pay some overhead for enqueuing work for later. Taken too far, lock contention / false sharing can make performance way worse. The allocators and destructors must be thread safe if offloading to a worker thread. The core running your thread is likely to have some of this data in L1/L2/L3 cache, which might not be true for whatever core would deque the work for asyncronously dropping. It can be harder to attribute the costs of dropping to the right code if it all gets mixed up into a single work queue cleaned up by opaque worker threads when profiling. If you don't use some kind of backpressure mechanism, allocations can potentially outpace deallocations and run you out of memory. ---- So, concrete example: Using Telemetry - a flamegraph style realtime profiler requiring invasive annotations - I was able to track down the cause of a framerate hitch in a game I was working on, to the sudden release of several graphics resources in a game. During events which would significantly restyle the look of some of the terrain, we'd eat several 10s/100s of milliseconds of overhead freeing things - more than enough to cause us to miss vsync. Would've stuck out like a sore thumb in any profiler capable of giving you a rough idea of the stack(s) involved in a 100ms timeframe that you can correlate to a vsync miss / missed frames. D3D9 isn't thread safe (although freeing resources might've been?), but I didn't need to offload the work onto another thread just to amortize the cost over a few frames. Instead, a simple work queue did the trick. Problem solved! New problem: level transitions took significantly longer when doing mass frees of the same resources - more than doubling the cost of deallocation IIRC, for reasons I never did fully understand. Cache thrashing of some sort? We were still maxing out the core running the main thread with mostly cleanup logic... Final code we shipped with used a hybrid solution that would choose between syncronous (high-throughput) and asyncronous (non-stalling) cleanup logic depending on what was happening in-game. Worked like a charm. Of course, this logic was hideously project specific and unable to be automatically chosen correctly for you by the programming language...
- deleted 6y ago
  [deleted]
ashtonkem 6y ago
Rust will automatically clean up data that’s left scope, but you can also manually accomplish this by the “drop” function, which is only necessary if you want to cleanup explicitly, such as in a different thread. Interestingly, the drop function is actually user-creatable. It’s actually an empty function with a very permissive Non-reference argument. The semantics of ownership in Rust makes that sufficient to trigger memory cleanup.
klyrs 6y ago
As I understand it, rust is automatically cleaning up, and that can cause glitchy timing. The clever hack is that rust lets you shunt that cleanup process off to another thread when you're the sole owner of that object. You can do the same thing in C, but unlike rust, the cost of cleanup is not hidden by the syntax.
- klyrs 6y ago
  On further reflection... I'm curious about how allocators would handle this -- if you return from this context only to make another heavyweight object, it seems like you'd be trading glitchy timing with allocator contention.
devit 6y ago
Because it's impossible to do this automatically in the general case. In particular, types may not be sendable to other threads, or may have side effects on dropping, and in those cases you would need to rearchitect the code before you can apply this technique. Also this technique adds overhead, so it should never be used (including not doing it conditionally) if you don't care about latency or if the objects are always small, and the compiler cannot know whether that is the case.

ReaLNero 6y ago

In C, if you forget to clean up, you have a memory leak which is hard to track down. In Rust, if you don't do this, you're not sacrificing memory leaks, only performance. A profiler can tell you when you should drop asynchronously.

madmax96 6y ago
>A profiler can tell you when you should drop asynchronously Is there any profiler that does this today? What are the drawbacks with asynchronous drops?
- ehsanu1 6y ago
  See some discussion here: https://www.reddit.com/r/rust/comments/gntv7l/dropping_heavy_objects_in_another_thread_can_make/ https://www.reddit.com/r/rust/comments/gntv7l/dropping_heavy...
  - deleted 6y ago
    [deleted]
- MaulingMonkey 6y ago
  > Is there any profiler that does this today? It requires interpretation, but yes. > What are the drawbacks with asynchronous drops? You pay some overhead for enqueuing work for later. Taken too far, lock contention / false sharing can make performance way worse. The allocators and destructors must be thread safe if offloading to a worker thread. The core running your thread is likely to have some of this data in L1/L2/L3 cache, which might not be true for whatever core would deque the work for asyncronously dropping. It can be harder to attribute the costs of dropping to the right code if it all gets mixed up into a single work queue cleaned up by opaque worker threads when profiling. If you don't use some kind of backpressure mechanism, allocations can potentially outpace deallocations and run you out of memory. ---- So, concrete example: Using Telemetry - a flamegraph style realtime profiler requiring invasive annotations - I was able to track down the cause of a framerate hitch in a game I was working on, to the sudden release of several graphics resources in a game. During events which would significantly restyle the look of some of the terrain, we'd eat several 10s/100s of milliseconds of overhead freeing things - more than enough to cause us to miss vsync. Would've stuck out like a sore thumb in any profiler capable of giving you a rough idea of the stack(s) involved in a 100ms timeframe that you can correlate to a vsync miss / missed frames. D3D9 isn't thread safe (although freeing resources might've been?), but I didn't need to offload the work onto another thread just to amortize the cost over a few frames. Instead, a simple work queue did the trick. Problem solved! New problem: level transitions took significantly longer when doing mass frees of the same resources - more than doubling the cost of deallocation IIRC, for reasons I never did fully understand. Cache thrashing of some sort? We were still maxing out the core running the main thread with mostly cleanup logic... Final code we shipped with used a hybrid solution that would choose between syncronous (high-throughput) and asyncronous (non-stalling) cleanup logic depending on what was happening in-game. Worked like a charm. Of course, this logic was hideously project specific and unable to be automatically chosen correctly for you by the programming language...
deleted 6y ago
[deleted]

ashtonkem 6y ago

Rust will automatically clean up data that’s left scope, but you can also manually accomplish this by the “drop” function, which is only necessary if you want to cleanup explicitly, such as in a different thread. Interestingly, the drop function is actually user-creatable. It’s actually an empty function with a very permissive Non-reference argument. The semantics of ownership in Rust makes that sufficient to trigger memory cleanup.

klyrs 6y ago

As I understand it, rust is automatically cleaning up, and that can cause glitchy timing. The clever hack is that rust lets you shunt that cleanup process off to another thread when you're the sole owner of that object. You can do the same thing in C, but unlike rust, the cost of cleanup is not hidden by the syntax.

klyrs 6y ago
On further reflection... I'm curious about how allocators would handle this -- if you return from this context only to make another heavyweight object, it seems like you'd be trading glitchy timing with allocator contention.

devit 6y ago

Because it's impossible to do this automatically in the general case. In particular, types may not be sendable to other threads, or may have side effects on dropping, and in those cases you would need to rearchitect the code before you can apply this technique. Also this technique adds overhead, so it should never be used (including not doing it conditionally) if you don't care about latency or if the objects are always small, and the compiler cannot know whether that is the case.