We've actually implemented something like this ourselves with our linear algebra library. We wrote our own gc for cuda: https://github.com/deeplearning4j/nd4j/

We've actually implemented something like this ourselves with our linear algebra library. We wrote our own gc for cuda: https://github.com/deeplearning4j/nd4j/tree/master/nd4j-backends/nd4j-backend-impls/nd4j-cuda/src/main/java/org/nd4j/jita https://github.com/deeplearning4j/nd4j/tree/master/nd4j-back... as well as: https://deeplearning4j.org/workspaces https://deeplearning4j.org/workspaces It also integrates with the JVM via weak references as well. I can attest to: https://news.ycombinator.com/item?id=15269537 https://news.ycombinator.com/item?id=15269537 as well, you really need this 3rd party due to the frequency of how often cuda changes. We usually end up having to support the 2 most recent cuda versions.