7 ms·
About 8 years ago an NVIDIA developer released a tool called Copperhead that let you write CUDA kernels in straight Python that were then compiled to C, no "C-i
by michelpp 5y ago
About 8 years ago an NVIDIA developer released a tool called Copperhead that let you write CUDA kernels in straight Python that were then compiled to C, no "C-in-a-string" like is shown here. I always thought it was so elegant and had great potential, and I introduced a lot of people in my circle to it, but then it seems NVIDIA buried it.
This blog post is great, and we need these kind of tools for sure, but we also need high level expressibility that doesn't require writing kernels in C. I know there are other projects that have taken up that cause, but it would be great to see NVIDIA double down on something like Copperhead.
- albertzeyer 5y agoOh that sounds interesting. Do you know what happened to it? I think I found it here: https://github.com/bryancatanzaro/copperhead https://github.com/bryancatanzaro/copperhead But I'm not sure what the state is. Looks dead (last commit 8 years ago). Probably just a proof of concept. But why hasn't this been continued? Blog post and example: https://developer.nvidia.com/blog/copperhead-data-parallel-python/ https://developer.nvidia.com/blog/copperhead-data-parallel-p... https://github.com/bryancatanzaro/copperhead/blob/master/samples/axpy.py https://github.com/bryancatanzaro/copperhead/blob/master/sam... Btw, for compiling on-the-fly from a string, I made something similar for our RETURNN project. Example for LSTM: https://github.com/rwth-i6/returnn/blob/a5eaa4ab1bfd5f157628fe375bb9b2995bf6b2bc/returnn/native_op.py#L1044 https://github.com/rwth-i6/returnn/blob/a5eaa4ab1bfd5f157628... This is made in a way that it compiles automatically into an op for Theano or TensorFlow (PyTorch could easily be added as well) and for both CPU and CUDA/GPU.
- dwrodri 5y agoI don't know specifics about Copperhead in particular, but Bryan Catanzaro (creator of Copperhead) is now the VP of Applied Deep Learning Research at Nvidia. He gave a talk at GTC this year, which is how I heard about all of this in the first place. Source: https://www.linkedin.com/in/bryancatanzaro/ https://www.linkedin.com/in/bryancatanzaro/
- anon_tor_12345 5y agothat project might be abandoned but this strategy is used in nvidia and nvidia adjacent projects (through llvm): https://github.com/rapidsai/cudf/blob/branch-0.20/python/cudf/cudf/utils/cudautils.py#L215 https://github.com/rapidsai/cudf/blob/branch-0.20/python/cud... https://github.com/gmarkall/numba/blob/master/numba/cuda/compiler.py https://github.com/gmarkall/numba/blob/master/numba/cuda/com... >but we also need high level expressibility that doesn't require writing kernels in C the above are possible because C is actually just a frontend to PTX https://docs.nvidia.com/cuda/parallel-thread-execution/index.html https://docs.nvidia.com/cuda/parallel-thread-execution/index... fundamentally you are not going to ever be able to have a way to write cuda kernels without thinking about cuda architecture anymore so than you'll ever be able to write async code without thinking about concurrency.
- ZeroCool2u 5y agoTotally agree, Copperhead looks much easier to use. Perhaps one of the reasons they went and rebuilt from scratch is because Copperhead relies on Thrust and a couple other dependencies?
- eslaught 5y agoAs it turns out, NVIDIA just open sourced a product called Legate which does not just GPUs but distributed as well. Right now it supports NumPy and Pandas but perhaps they'll add others in the future. Just thought this might be up your alley since it works at a higher level than the glorified CUDA in the article. https://github.com/nv-legate/legate.numpy https://github.com/nv-legate/legate.numpy Disclaimer: I work on the project they used to do the distributed execution, but otherwise have no connection with Legate. Edit: And this library was developed by a team managed by one of the original Copperhead developers, in case you're wondering.
- BiteCode_dev 5y agoIn the IP world, there are some-hidden gems that disappear with no trace one day. I worked for a client that had this wonderful Python dsl that compiled to Verilog and VHDL. It was much easier to use than writing the stuff the old way. Much more composable too, not to mention tooling. They created that by forking an open source project dating back to Python 2.5 that I could never find again. Imagine if that stuff would still be alive today. You could have a market for paid pypi.org instances providing you with pip installable IP components you can compose and customize easily. But in this market, sharing is not really a virtue.
- ekiwi 5y agoSounds like nmigen might be a good open source successor to the project that you describe: https://github.com/nmigen/nmigen https://github.com/nmigen/nmigen There are lots of open-source (n)migen components available through the litex framework: https://github.com/enjoy-digital/litex https://github.com/enjoy-digital/litex
- BiteCode_dev 5y agoLooks awesome.
- antpls 5y agoClosest thing to mind is Numba's cuda JIT compilation : https://numba.pydata.org/numba-doc/latest/cuda/index.html https://numba.pydata.org/numba-doc/latest/cuda/index.html Then you have Cudapy : https://github.com/oulgen/CudaPy https://github.com/oulgen/CudaPy But in my opinion, the most future proof solutions are higher level frameworks like Numpy, Jax and Tensorflow. TensorFlow and Jax can JIT compile Python functions to GPU (tf.function).
- bcatanzaro 5y agoI’m so glad you liked it. It was a labor of love and I was super proud of it. Means a lot to me that you would still remember it. I flatter myself to think that some ideas from that project have lived on in Tensorflow and PyTorch, etc. But the project itself wrapped up when I decided to focus on DL, and it would take a lot of work to bring it back to life.
- cycomanic 5y agoThere is arrayfire which does something like that and has python bindings