12 ms·

Nvidia Warp: A Python framework for high performance GPU simulation and graphics

490 points by jarmitage 2y ago

eigenvalue 2y ago
I really like how nvidia started doing more normal open source and not locking stuff behind a login to their website. It makes it so much easier now that you can just pip install all the cuda stuff for torch and other libraries without authenticating and downloading from websites and other nonsense. I guess they realized that it was dramatically reducing the engagement with their work. If it’s open source anyway then you should make it as accessible as possible.
- jjmarr 2y ago
  It being on GitHub doesn't mean it's open-source. https://github.com/NVIDIA/warp?tab=License-1-ov-file#readme https://github.com/NVIDIA/warp?tab=License-1-ov-file#readme Looks more "source available" to me.
  - nitinreddy88 2y ago
    That's what open-source means. Source code is open for reading. It has nothing to do with Licensing. You can have any type of license on top of that based on your business needs
    - dagenix 2y ago
      That may be your definition, but that's not everyone's definition. Wikipedia, for example, says: > Open-source software (OSS) is computer software that is released under a license in which the copyright holder grants users the rights to use, study, change, and distribute the software and its source code to anyone and for any purpose. https://en.m.wikipedia.org/wiki/Open-source_software https://en.m.wikipedia.org/wiki/Open-source_software
      - deleted 2y ago
        [deleted]
    - j-r-d 2y ago
      No. That's not how it works. It's great that they're making source available but if I can't modify and distribute it, it's not open.
    - TimeBearingDown 2y ago
      No. The Open Source Initiative maintains the definition, which is accepted internationally by multiple government agencies. https://opensource.org/osd https://opensource.org/osd https://opensource.org/authority https://opensource.org/authority
- foresterre 2y ago
  I would argue that this isn't "normal open source", though it is indeed not locked behind a login on their website. The license (1) is feels very much proprietary, even if the source code is available. (1) https://github.com/NVIDIA/warp/blob/main/LICENSE.md https://github.com/NVIDIA/warp/blob/main/LICENSE.md
  - bionhoward 2y ago
    Agreed, especially given this "2.7 You may not use the Software for the purpose of developing competing products or technologies or assist a third party in such activities." vs "California’s public policy provides that every contract that restrains anyone from engaging in a lawful profession, trade, or business of any kind is, to that extent, void, except under limited statutory exceptions." https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml?bill_id=202320240SB699 https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml... (Owner/Partner who sold business, may voluntarily agree to a noncompete, (which is now federally https://www.ftc.gov/legal-library/browse/rules/noncompete-rule https://www.ftc.gov/legal-library/browse/rules/noncompete-ru... banned) is the only exception I found). I'm not a lawyer. Any lawyers around? Could the 2nd provision invalidate the 1st, or not?
    - philipov 2y ago
      You're free to engage in a lawful profession, just not using that Software for it. "to that extent" is not there merely for show.
      - bionhoward 2y ago
        Hey, that's a real argument, and it makes sense. Thank you for helping to clarify this topic. Question: why would NVIDIA, makers of general intelligence, which seems to compete with everyone, publish code for software nobody can use without breaking NVIDIA rules? Wouldn't it be better for everyone if they just kept that code private?
        bionhoward 2y ago
- dagmx 2y ago
  This isn’t open source. It s the equivalent of headers being available to a dylib, just that they happen to be a python API. Most of the magic is behind closed source components, and it’s posted with a fairly restrictive license.
  - fragmede 2y ago
    And people say nvida doesn't have a moat.
    - markhahn 2y ago
      donno. it's clear they have no effective moat. they still try hard, making themselves quite customer-hostile.
      - yosefk 2y ago
        CUDA is a moat. What's ineffective about it?
  - boywitharupee 2y ago
    In a similar fashion, you'll see that JAX has frontend code being open-sourced, while device-related code is distributed as binaries. For example, if you're on Google's TPU, you'll see libtpu.so, and on macOS, you'll see pjrt_plugin_metal_1.x.dylib. The main optimizations (scheduler, vectorizer, etc.) are hidden behind these shared libraries. If open-sourced, they might reveal hints about proprietary algorithms and provide clues to various hardware components, which could potentially be exploited.
- water-your-self 2y ago
  Accessible, as long as you purchase their very contested hardware.
- deleted 2y ago
  [deleted]
- markhahn 2y ago
  it's not open source and can only be used with nvidia gpus (by license).
- rldjbpin 2y ago
  there has been a rise of "open"-access and freeware software/services in this space. see hugging face and certain models tied to accounts that accept some eula before downloading model weights, or weird wrapper code by ai library creators which makes it harder to run offline (ultralytics library comes to mind for instance). i like the value they bring, but the trend is against the existing paradigm of how python ecosystem used to be.
jorlow 2y ago
Does this compete at all with openAI's triton (which is sort of a higher level cuda without the vendor lock in)?
arvinsim 2y ago
As someone who is not in the simulation and graphic space, what does this library bring that current libraries do not?
- ok123456 2y ago
  It overlaps a lot with the library Taichi, which Disney supports. It's noteworthy that Taichi also supports AMD, MPI, and Kokkos.
paulluuk 2y ago
While this is really cool, I have to say.. > import warp as wp Can we please not copy this convention over from numpy? In the example script, you use 17 characters to write this just to save 18 characters later on in the script. Just import the warp commands you use, or if you really want "import warp", but don't rename imported libraries, please.
- dahfizz 2y ago
  Strongly agreed! This convention has even infected internal tooling at my company. Scripts end up with tons of cryptic three letter names. It saves a couple keystrokes but wastes engineering time to maintain
  - physicsguy 2y ago
    The convention is a convention because the libraries are used so commonly. If you give anyone in scientific computing Python world something with “np” or “pd” then they know what that is. Doing something other than what is convention for those libraries wastes more time when people jump into a file because people have to work out now whether “array” is some bespoke type or the NumPy one they’re used to.
    - paulluuk 2y ago
      There is no way that "warp" is already such a household name that it's common enough to shorten it to "wp". Likewise, the libraries at OP's company are for sure not going to be common to anyone starting out at the company, and might still be confusing to anyone who has worked there for years but just hasn't had to use that specific library. Pandas and Numpy are popular, sure. As is Tensorflow (often shortened to tf). But where do you draw the line, then? should the openai library be imported as oa? should flask be imported as fk? should requests be imported as rq? It seems to happen mostly to libraries that are commonly used by one specific audience: scientists who are forced to use a programming language, and who think that 1-letter variables are good variable names, and who prefer using notebooks over scripts with functions. Don't get me wrong, I'm glad that Python gets so much attention from the scientific community, but I feel that small little annoyances like this creep in because of it, too.
w-m 2y ago
I was playing around with taichi a little bit for a project. Taichi lives in a similar space, but has more than an NVIDIA backend. But its development has stalled, so I’m considering switching to warp now. It’s quite frustrating that there’s seemingly no long-lived framework that allows me to write simple numba-like kernels and try them out in NVIDIA GPUs and Apple GPUs. Even with taichi, the Metal backend was definitely B-tier or lower: Not offering 64 bit ints, and randomly crashing/not compiling stuff. Here’s hoping that we’ll solve the GPU programming space in the next couple years, but after ~15 years or so of waiting, I’m no longer holding my breath. https://github.com/taichi-dev/taichi https://github.com/taichi-dev/taichi
- panagathon 2y ago
  This is the library I've always wanted. Look at that Julia set. Gorgeous. Thanks for this. I'm sorry to hear about the dev issues. I wish I could help.
- paulmd 2y ago
  the problem with the GPGPU space is that everything except CUDA is so fractally broken that everything eventually converges to the NVIDIA stuff that actually works. yes, the heterogeneous compute frameworks are largely broken, except for OneAPI, which does work, but only on CUDA. SPIR-V, works best on CUDA. OpenCL: works best on CUDA. Even once you get past the topline "does it even attempt to support that", you'll find that AMD's runtimes are broken too. Their OpenCL runtime is buggy and has a bunch of paper features which don't work, and a bunch of AMD-specific behavior and bugs that aren't spec-compliant. So basically you have to have an AMD-specific codepath anyway to handle the bugs. Same for SPIR-V: the biggest thing they have working against them is that AMD's Vulkan Compute support is incomplete and buggy too. https://devtalk.blender.org/t/was-gpu-support-just-outright-dropped-for-gpus-that-are-not-very-very-new/23724/3 https://devtalk.blender.org/t/was-gpu-support-just-outright-... https://render.otoy.com/forum/viewtopic.php?f=7&t=75411 https://render.otoy.com/forum/viewtopic.php?f=7&t=75411 ("As of right now, the Vulkan drivers on AMD and Intel are not mature enough to compile (much less ship) Octane for Vulkan") If you are going to all that effort anyway, why are you (a) targeting AMD at all, and (b) why don't you just use CUDA in the first place? So everyone writes more CUDA and nothing gets done. Cue some new whippersnapper who thinks they're gonna cure all AMD's software problems in a month, they bash into the brick wall, write blog post, becomes angry forums commenter, rinse and repeat. And now you have another abandoned cross-platform project that basically only ever supported NVIDIA anyway. Intel, bless their heart, is actually trying and their stuff largely does just work, supposedly, although I'm trying to get their linux runtime up and running on a Serpent Canyon NUC with A770m and am having a hell of a time. But supposedly it does work especially on windows (and I may just have to knuckle under and use windows, or put a pcie card in a server pc). But they just don't have the marketshare to make it stick. AMD is stuck in this perpetual cycle of expecting anyone else but themselves to write the software, and then not even providing enough infrastructure to get people to the starting line, and then surprised-pikachu nothing works, and surprise-pikachu they never get any adoption. Why has nvidia done this!?!? /s The other big exception is Metal, which both works and has an actual userbase. The reason they have Metal support for cycles and octane is because they contribute the code, that's really what needs to happen (and I think what Intel is doing - there's just a lot of work to come from zero). But of course Metal is apple-only, so really ideally you would have a layer that goes over the top...
dudus 2y ago
Gotta keep digging that CUDA moat as hard and as fast as possible.
- astromaniak 2y ago
  Exactly. and that's why it's valued at $3T++, about 10x of AMD and Intel put together.
  - markhahn 2y ago
    you mean because that's how you get to be a meme stock? yep. stock markets are casinos filled with know-nothing high-rollers and pension sheep.
    - talldayo 2y ago
      How many meme stocks are TSMC customers?
    - astromaniak 2y ago
      doing business is hard this days. you can't be just a rich a*hole while working with people. have to care about your image. and this is one of the ways of doing it. hanging out free stuff. sort of selfless donations. but in fact this rises the bar and makes competitors' life much harder. of course you can be rich and narrow minded, like intel. but then it's hard to attract external developers and make them believe in you future. nvidia's stock rise is based on the vision, investors believe in it. while other giants are being dominated by carrier managers. who know the procedures, but absolutely blind when it comes to technology evaluation. if someone comes to them with a great idea they first evaluate how it fits in their plans. sometimes they their own primitive vision, like in facebook. which proved to be a... not that good. so, all this sort of managers can do is look at what is _alrady_ successful and try to replicate it throwing a lot of money. it may be not enough. like intel still lags behind in GPUs.
- tomjen3 2y ago
  Thats the part I don't get. When you are developing AI, how much code are you really running on GPUs? How bad would it be to write it for something else if you could get 10% more compute per dollar?
VyseofArcadia 2y ago
Aren't warps already architectural elements of nvidia graphics cards? This name collision is going to muddy search results.
- logicchains 2y ago
  >Aren't warps already architectural elements of nvidia graphics cards? Architectural elements of _all_ graphics cards.
  - VyseofArcadia 2y ago
    Unsure of how authoritative this is, but this article[0] seems to imply it's a matter of branding. > The efficiency of executing threads in groups, which is known as warps in NVIDIA and wavefronts in AMD, is crucial for maximizing core utilization. [0] https://www.xda-developers.com/how-does-a-graphics-card-actually-work/ https://www.xda-developers.com/how-does-a-graphics-card-actu...
    - logicchains 2y ago
      ROCm also refers to them as warps https://rocm.docs.amd.com/projects/HIP/en/latest/understand/hardware_implementation.html https://rocm.docs.amd.com/projects/HIP/en/latest/understand/... : >The threads are executed in groupings called warps. The amount of threads making up a warp is architecture dependent. On AMD GPUs the warp size is commonly 64 threads, except in RDNA architectures which can utilize a warp size of 32 or 64 respectively. The warp size of supported AMD GPUs is listed in the Accelerator and GPU hardware specifications. NVIDIA GPUs have a warp size of 32.
      - int_19h 2y ago
        It actually kinda makes some sense when you realize that "warp" is a reference to warp threads in actual weaving: https://en.wikipedia.org/wiki/Warp_and_weft https://en.wikipedia.org/wiki/Warp_and_weft.
- ahfeah7373 2y ago
nurettin 2y ago
How is this different than taichi? Even the decorators look similar.
raytopia 2y ago
I love how many python to native/gpu code projects there are now. It's nice to see a lot of competition in the space. An alternative to this one could be Taichi Lang [0] it can use your gpu through Vulkan so you don't have to own Nvidia hardware. Numba [1] is another alternative that's very popular. I'm still waiting on a Python project that compiles to pure C (unlike Cython [2] which is hard to port) so you can write homebrew games or other embedded applications. [0] https://www.taichi-lang.org/ https://www.taichi-lang.org/ [1] http://numba.pydata.org/ http://numba.pydata.org/ [2] https://cython.readthedocs.io/en/stable/ https://cython.readthedocs.io/en/stable/
- setopt 2y ago
  CuPy is also great – makes it trivial to port existing numerical code from NumPy/SciPy to CUDA, or to write code than can run either on CPU or on GPU. I recently saw a 2-3 orders of magnitude speed-up of some physics code when I got a mid-range nVidia card and replaced a few NumPy and SciPy calls with CuPy.
  - 6gvONxR4sf7o 2y ago
    Don’t forget JAX! It’s my preferred library for “i want to write numpy but want it to run on gpu/tpu with auto diff etc”
    - westurner 2y ago
      From https://news.ycombinator.com/item?id=37686351 https://news.ycombinator.com/item?id=37686351 : >> sympy.utilities.lambdify.lambdify() https://github.com/sympy/sympy/blob/a76b02fcd3a8b7f79b3a88df50c19eb7aee33c17/sympy/utilities/lambdify.py#L182 https://github.com/sympy/sympy/blob/a76b02fcd3a8b7f79b3a88df... : >> """Convert a SymPy expression into a function that allows for fast numeric evaluation""" [e.g. the CPython math module, mpmath, NumPy, SciPy, CuPy, JAX, TensorFlow, SymPy, numexpr,] sympy#20516: "re-implementation of torch-lambdify" https://github.com/sympy/sympy/pull/20516 https://github.com/sympy/sympy/pull/20516
- skrhee 2y ago
owenpalmer 2y ago
> Warp is designed for spatial computing What does this mean? I've mainly heard the term "spatial computing" in the context of the Vision Pro release. It doesn't seem like this was intended for AR/VR
- educasean 2y ago
  As someone not in this space, I was immediately tripped up by this as well. Does spatial computing mean something else in this context?
  - basiccalendar74 2y ago
    main use case seems to be simulations in 2D, 3D or nD spaces. spaces -> spatial.
water-your-self 2y ago
>GPU support requires a CUDA-capable NVIDIA GPU and driver (minimum GeForce GTX 9xx). Very tactful from nvidia. I have a lovely AMD gpu and this library is worthless for it.
- coldtea 2y ago
  Err, it is nvidia. Why would they support AMD?
jarmitage 2y ago
> What's Taichi's take on NVIDIA's Warp? > Overall the biggest distinction as of now is that Taichi operates at a slightly higher level. E.g. implict loop parallelization, high level spatial data structures, direct interops with torch, etc. > We are trying to implement support for lower level programming styles to accommodate such things as native intrinsics, but we do think of those as more advanced optimization techniques, and at the same time we strive for easier entry and usage for beginners or people not so used to CUDA's programming model – https://github.com/taichi-dev/taichi/discussions/8184 https://github.com/taichi-dev/taichi/discussions/8184
BenoitP 2y ago
This should be seen in light of the Great Differentiable Convergence™: NERFs backpropagating pixels colors into the volume, but also semantic information from the image label, embedded from an LLM reading a multimedia document. Or something like this. Anyway, wanna buy an NVIDIA GPU ;)?
deleted 2y ago
[deleted]
wallscratch 2y ago
Can anyone comment on how efficient the Warp code is compared to manually written / fine-tuned CUDA?
jokoon 2y ago
funny that now some softwares are hardware dependent OpenCL seems like it's just obsolete
- pjmlp 2y ago
  OpenCL has been obsolete for years, as Intel, AMD and Google never provided a proper development experience with good drivers. The fact that OpenCL 3.0 is basically OpenCL 1.0 rebranded, as acknwoledgement of OpenCL 2.0 adoption failure, doesn't help either.
TNWin 2y ago
Slightly related What's this community's take on Triton? https://openai.com/index/triton/ https://openai.com/index/triton/ Are there better alternatives?
beebmam 2y ago
Why Python? I really don't understand this choice of language other than accessibility.
- danielmarkbruce 2y ago
  Because accessibility.
- mkl 2y ago
  I think you answered your own question there. Python is very accessible, very popular, and already widely used for GPU-based things like machine learning.
- pzo 2y ago
  Huge ecosystem starting with numpy, pandas, mathplot et al for data science, pytorch, tensorflow, jax for ML, gradio, rerun for visualization, opencv, open3d for image/pointcloud processing, pyside for gui and others.
- int_19h 2y ago
  Because that's where the vast majority of DS/ML is already, and they are too busy to learn something else.
- rldjbpin 2y ago
  to me it goes beyond that. many leetcode grinders swear by specific data structures such as hashmaps, which python makes available as dictionaries. behind the sytax, there is plenty of heavy lifting for writing sophisticated code, when need be. that surely helps with the network effect.
bytesandbits 2y ago
How is this different than Triton?
jkbbwr 2y ago
I really wish python would stop being the go-to language for GPU orchestration or machine learning, having worked with it again recently for some proof of concepts its been a massive pain in the ass.
- seydor 2y ago
  We should have by now a new language for AI systems, not just frameworks
- FrozenSynapse 2y ago
  seeing as every big corp chooses it for their libraries, I'd say it's a skill issue
marmaduke 2y ago
Ive dredged though Julia, Numba, Jax, Futhark, looking a way to have good CPU performance in absence of GPU, and I'm not really happy with any of them. Especially given how many want you to lug LLVM along with. A recent simulation code when pushed with gcc openmp-simd matched performance on a 13900K vs jax.jit on a rtx 4090. This case worked because the overall computation can be structured into pieces that fit in L1/L2 cache, but I had to spend a ton of time writing the C code, whereas jax.jit was too easy. So I'd still like to see something like this but which really works for CPU as well.
- mccoyb 2y ago
  Agreed, JAX is specialized for GPU computation -- I'd really like similar capabilities with more permissive constructs, maybe even co-effect tagging of pieces of code (which part goes on GPU, which part goes on CPU), etc. I've thought about extending JAX with custom primitives and a custom lowering process to support constructs which work on CPU (but don't work on GPU) -- but if I did that, and wanted a nice programmable substrate -- I'd need to define my own version of abstract tracing (because necessarily, permissive CPU constructs might imply array type permissiveness like dynamic shapes, etc). You start heading towards something that looks like Julia -- the problem (for my work) with Julia is that it doesn't support composable transformations like JAX does. Julia + JAX might be offered as a solution -- but it's quite unsatisfying to me.

490 points by jarmitage 2y ago

I really like how nvidia started doing more normal open source and not locking stuff behind a login to their website. It makes it so much easier now that you can just pip install all the cuda stuff for torch and other libraries without authenticating and downloading from websites and other nonsense. I guess they realized that it was dramatically reducing the engagement with their work. If it’s open source anyway then you should make it as accessible as possible.

jjmarr 2y ago
It being on GitHub doesn't mean it's open-source. https://github.com/NVIDIA/warp?tab=License-1-ov-file#readme https://github.com/NVIDIA/warp?tab=License-1-ov-file#readme Looks more "source available" to me.
- nitinreddy88 2y ago
  That's what open-source means. Source code is open for reading. It has nothing to do with Licensing. You can have any type of license on top of that based on your business needs
  - dagenix 2y ago
    That may be your definition, but that's not everyone's definition. Wikipedia, for example, says: > Open-source software (OSS) is computer software that is released under a license in which the copyright holder grants users the rights to use, study, change, and distribute the software and its source code to anyone and for any purpose. https://en.m.wikipedia.org/wiki/Open-source_software https://en.m.wikipedia.org/wiki/Open-source_software
    - deleted 2y ago
      [deleted]
  - j-r-d 2y ago
    No. That's not how it works. It's great that they're making source available but if I can't modify and distribute it, it's not open.
  - TimeBearingDown 2y ago
    No. The Open Source Initiative maintains the definition, which is accepted internationally by multiple government agencies. https://opensource.org/osd https://opensource.org/osd https://opensource.org/authority https://opensource.org/authority
foresterre 2y ago
I would argue that this isn't "normal open source", though it is indeed not locked behind a login on their website. The license (1) is feels very much proprietary, even if the source code is available. (1) https://github.com/NVIDIA/warp/blob/main/LICENSE.md https://github.com/NVIDIA/warp/blob/main/LICENSE.md
- bionhoward 2y ago
  Agreed, especially given this "2.7 You may not use the Software for the purpose of developing competing products or technologies or assist a third party in such activities." vs "California’s public policy provides that every contract that restrains anyone from engaging in a lawful profession, trade, or business of any kind is, to that extent, void, except under limited statutory exceptions." https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml?bill_id=202320240SB699 https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml... (Owner/Partner who sold business, may voluntarily agree to a noncompete, (which is now federally https://www.ftc.gov/legal-library/browse/rules/noncompete-rule https://www.ftc.gov/legal-library/browse/rules/noncompete-ru... banned) is the only exception I found). I'm not a lawyer. Any lawyers around? Could the 2nd provision invalidate the 1st, or not?
  - philipov 2y ago
    You're free to engage in a lawful profession, just not using that Software for it. "to that extent" is not there merely for show.
    - bionhoward 2y ago
      Hey, that's a real argument, and it makes sense. Thank you for helping to clarify this topic. Question: why would NVIDIA, makers of general intelligence, which seems to compete with everyone, publish code for software nobody can use without breaking NVIDIA rules? Wouldn't it be better for everyone if they just kept that code private?
      - bionhoward 2y ago
dagmx 2y ago
This isn’t open source. It s the equivalent of headers being available to a dylib, just that they happen to be a python API. Most of the magic is behind closed source components, and it’s posted with a fairly restrictive license.
- fragmede 2y ago
  And people say nvida doesn't have a moat.
  - markhahn 2y ago
    donno. it's clear they have no effective moat. they still try hard, making themselves quite customer-hostile.
    - yosefk 2y ago
      CUDA is a moat. What's ineffective about it?
- boywitharupee 2y ago
  In a similar fashion, you'll see that JAX has frontend code being open-sourced, while device-related code is distributed as binaries. For example, if you're on Google's TPU, you'll see libtpu.so, and on macOS, you'll see pjrt_plugin_metal_1.x.dylib. The main optimizations (scheduler, vectorizer, etc.) are hidden behind these shared libraries. If open-sourced, they might reveal hints about proprietary algorithms and provide clues to various hardware components, which could potentially be exploited.
water-your-self 2y ago
Accessible, as long as you purchase their very contested hardware.
deleted 2y ago
[deleted]
markhahn 2y ago
it's not open source and can only be used with nvidia gpus (by license).
rldjbpin 2y ago
there has been a rise of "open"-access and freeware software/services in this space. see hugging face and certain models tied to accounts that accept some eula before downloading model weights, or weird wrapper code by ai library creators which makes it harder to run offline (ultralytics library comes to mind for instance). i like the value they bring, but the trend is against the existing paradigm of how python ecosystem used to be.

jorlow 2y ago

Does this compete at all with openAI's triton (which is sort of a higher level cuda without the vendor lock in)?

arvinsim 2y ago

As someone who is not in the simulation and graphic space, what does this library bring that current libraries do not?

ok123456 2y ago
It overlaps a lot with the library Taichi, which Disney supports. It's noteworthy that Taichi also supports AMD, MPI, and Kokkos.

paulluuk 2y ago

While this is really cool, I have to say.. > import warp as wp Can we please not copy this convention over from numpy? In the example script, you use 17 characters to write this just to save 18 characters later on in the script. Just import the warp commands you use, or if you really want "import warp", but don't rename imported libraries, please.

dahfizz 2y ago
Strongly agreed! This convention has even infected internal tooling at my company. Scripts end up with tons of cryptic three letter names. It saves a couple keystrokes but wastes engineering time to maintain
- physicsguy 2y ago
  The convention is a convention because the libraries are used so commonly. If you give anyone in scientific computing Python world something with “np” or “pd” then they know what that is. Doing something other than what is convention for those libraries wastes more time when people jump into a file because people have to work out now whether “array” is some bespoke type or the NumPy one they’re used to.
  - paulluuk 2y ago
    There is no way that "warp" is already such a household name that it's common enough to shorten it to "wp". Likewise, the libraries at OP's company are for sure not going to be common to anyone starting out at the company, and might still be confusing to anyone who has worked there for years but just hasn't had to use that specific library. Pandas and Numpy are popular, sure. As is Tensorflow (often shortened to tf). But where do you draw the line, then? should the openai library be imported as oa? should flask be imported as fk? should requests be imported as rq? It seems to happen mostly to libraries that are commonly used by one specific audience: scientists who are forced to use a programming language, and who think that 1-letter variables are good variable names, and who prefer using notebooks over scripts with functions. Don't get me wrong, I'm glad that Python gets so much attention from the scientific community, but I feel that small little annoyances like this creep in because of it, too.

w-m 2y ago

I was playing around with taichi a little bit for a project. Taichi lives in a similar space, but has more than an NVIDIA backend. But its development has stalled, so I’m considering switching to warp now. It’s quite frustrating that there’s seemingly no long-lived framework that allows me to write simple numba-like kernels and try them out in NVIDIA GPUs and Apple GPUs. Even with taichi, the Metal backend was definitely B-tier or lower: Not offering 64 bit ints, and randomly crashing/not compiling stuff. Here’s hoping that we’ll solve the GPU programming space in the next couple years, but after ~15 years or so of waiting, I’m no longer holding my breath. https://github.com/taichi-dev/taichi https://github.com/taichi-dev/taichi

panagathon 2y ago
This is the library I've always wanted. Look at that Julia set. Gorgeous. Thanks for this. I'm sorry to hear about the dev issues. I wish I could help.
paulmd 2y ago
the problem with the GPGPU space is that everything except CUDA is so fractally broken that everything eventually converges to the NVIDIA stuff that actually works. yes, the heterogeneous compute frameworks are largely broken, except for OneAPI, which does work, but only on CUDA. SPIR-V, works best on CUDA. OpenCL: works best on CUDA. Even once you get past the topline "does it even attempt to support that", you'll find that AMD's runtimes are broken too. Their OpenCL runtime is buggy and has a bunch of paper features which don't work, and a bunch of AMD-specific behavior and bugs that aren't spec-compliant. So basically you have to have an AMD-specific codepath anyway to handle the bugs. Same for SPIR-V: the biggest thing they have working against them is that AMD's Vulkan Compute support is incomplete and buggy too. https://devtalk.blender.org/t/was-gpu-support-just-outright-dropped-for-gpus-that-are-not-very-very-new/23724/3 https://devtalk.blender.org/t/was-gpu-support-just-outright-... https://render.otoy.com/forum/viewtopic.php?f=7&t=75411 https://render.otoy.com/forum/viewtopic.php?f=7&t=75411 ("As of right now, the Vulkan drivers on AMD and Intel are not mature enough to compile (much less ship) Octane for Vulkan") If you are going to all that effort anyway, why are you (a) targeting AMD at all, and (b) why don't you just use CUDA in the first place? So everyone writes more CUDA and nothing gets done. Cue some new whippersnapper who thinks they're gonna cure all AMD's software problems in a month, they bash into the brick wall, write blog post, becomes angry forums commenter, rinse and repeat. And now you have another abandoned cross-platform project that basically only ever supported NVIDIA anyway. Intel, bless their heart, is actually trying and their stuff largely does just work, supposedly, although I'm trying to get their linux runtime up and running on a Serpent Canyon NUC with A770m and am having a hell of a time. But supposedly it does work especially on windows (and I may just have to knuckle under and use windows, or put a pcie card in a server pc). But they just don't have the marketshare to make it stick. AMD is stuck in this perpetual cycle of expecting anyone else but themselves to write the software, and then not even providing enough infrastructure to get people to the starting line, and then surprised-pikachu nothing works, and surprise-pikachu they never get any adoption. Why has nvidia done this!?!? /s The other big exception is Metal, which both works and has an actual userbase. The reason they have Metal support for cycles and octane is because they contribute the code, that's really what needs to happen (and I think what Intel is doing - there's just a lot of work to come from zero). But of course Metal is apple-only, so really ideally you would have a layer that goes over the top...

dudus 2y ago

Gotta keep digging that CUDA moat as hard and as fast as possible.

astromaniak 2y ago
Exactly. and that's why it's valued at $3T++, about 10x of AMD and Intel put together.
- markhahn 2y ago
  you mean because that's how you get to be a meme stock? yep. stock markets are casinos filled with know-nothing high-rollers and pension sheep.
  - talldayo 2y ago
    How many meme stocks are TSMC customers?
  - astromaniak 2y ago
    doing business is hard this days. you can't be just a rich a*hole while working with people. have to care about your image. and this is one of the ways of doing it. hanging out free stuff. sort of selfless donations. but in fact this rises the bar and makes competitors' life much harder. of course you can be rich and narrow minded, like intel. but then it's hard to attract external developers and make them believe in you future. nvidia's stock rise is based on the vision, investors believe in it. while other giants are being dominated by carrier managers. who know the procedures, but absolutely blind when it comes to technology evaluation. if someone comes to them with a great idea they first evaluate how it fits in their plans. sometimes they their own primitive vision, like in facebook. which proved to be a... not that good. so, all this sort of managers can do is look at what is _alrady_ successful and try to replicate it throwing a lot of money. it may be not enough. like intel still lags behind in GPUs.
tomjen3 2y ago
Thats the part I don't get. When you are developing AI, how much code are you really running on GPUs? How bad would it be to write it for something else if you could get 10% more compute per dollar?

VyseofArcadia 2y ago

Aren't warps already architectural elements of nvidia graphics cards? This name collision is going to muddy search results.

logicchains 2y ago
>Aren't warps already architectural elements of nvidia graphics cards? Architectural elements of _all_ graphics cards.
- VyseofArcadia 2y ago
  Unsure of how authoritative this is, but this article[0] seems to imply it's a matter of branding. > The efficiency of executing threads in groups, which is known as warps in NVIDIA and wavefronts in AMD, is crucial for maximizing core utilization. [0] https://www.xda-developers.com/how-does-a-graphics-card-actually-work/ https://www.xda-developers.com/how-does-a-graphics-card-actu...
  - logicchains 2y ago
    ROCm also refers to them as warps https://rocm.docs.amd.com/projects/HIP/en/latest/understand/hardware_implementation.html https://rocm.docs.amd.com/projects/HIP/en/latest/understand/... : >The threads are executed in groupings called warps. The amount of threads making up a warp is architecture dependent. On AMD GPUs the warp size is commonly 64 threads, except in RDNA architectures which can utilize a warp size of 32 or 64 respectively. The warp size of supported AMD GPUs is listed in the Accelerator and GPU hardware specifications. NVIDIA GPUs have a warp size of 32.
    - int_19h 2y ago
      It actually kinda makes some sense when you realize that "warp" is a reference to warp threads in actual weaving: https://en.wikipedia.org/wiki/Warp_and_weft https://en.wikipedia.org/wiki/Warp_and_weft.
ahfeah7373 2y ago

nurettin 2y ago

How is this different than taichi? Even the decorators look similar.

raytopia 2y ago

I love how many python to native/gpu code projects there are now. It's nice to see a lot of competition in the space. An alternative to this one could be Taichi Lang [0] it can use your gpu through Vulkan so you don't have to own Nvidia hardware. Numba [1] is another alternative that's very popular. I'm still waiting on a Python project that compiles to pure C (unlike Cython [2] which is hard to port) so you can write homebrew games or other embedded applications. [0] https://www.taichi-lang.org/ https://www.taichi-lang.org/ [1] http://numba.pydata.org/ http://numba.pydata.org/ [2] https://cython.readthedocs.io/en/stable/ https://cython.readthedocs.io/en/stable/

setopt 2y ago
CuPy is also great – makes it trivial to port existing numerical code from NumPy/SciPy to CUDA, or to write code than can run either on CPU or on GPU. I recently saw a 2-3 orders of magnitude speed-up of some physics code when I got a mid-range nVidia card and replaced a few NumPy and SciPy calls with CuPy.
- 6gvONxR4sf7o 2y ago
  Don’t forget JAX! It’s my preferred library for “i want to write numpy but want it to run on gpu/tpu with auto diff etc”
  - westurner 2y ago
    From https://news.ycombinator.com/item?id=37686351 https://news.ycombinator.com/item?id=37686351 : >> sympy.utilities.lambdify.lambdify() https://github.com/sympy/sympy/blob/a76b02fcd3a8b7f79b3a88df50c19eb7aee33c17/sympy/utilities/lambdify.py#L182 https://github.com/sympy/sympy/blob/a76b02fcd3a8b7f79b3a88df... : >> """Convert a SymPy expression into a function that allows for fast numeric evaluation""" [e.g. the CPython math module, mpmath, NumPy, SciPy, CuPy, JAX, TensorFlow, SymPy, numexpr,] sympy#20516: "re-implementation of torch-lambdify" https://github.com/sympy/sympy/pull/20516 https://github.com/sympy/sympy/pull/20516
skrhee 2y ago

owenpalmer 2y ago

> Warp is designed for spatial computing What does this mean? I've mainly heard the term "spatial computing" in the context of the Vision Pro release. It doesn't seem like this was intended for AR/VR

educasean 2y ago
As someone not in this space, I was immediately tripped up by this as well. Does spatial computing mean something else in this context?
- basiccalendar74 2y ago
  main use case seems to be simulations in 2D, 3D or nD spaces. spaces -> spatial.

water-your-self 2y ago

>GPU support requires a CUDA-capable NVIDIA GPU and driver (minimum GeForce GTX 9xx). Very tactful from nvidia. I have a lovely AMD gpu and this library is worthless for it.

coldtea 2y ago
Err, it is nvidia. Why would they support AMD?

jarmitage 2y ago

> What's Taichi's take on NVIDIA's Warp? > Overall the biggest distinction as of now is that Taichi operates at a slightly higher level. E.g. implict loop parallelization, high level spatial data structures, direct interops with torch, etc. > We are trying to implement support for lower level programming styles to accommodate such things as native intrinsics, but we do think of those as more advanced optimization techniques, and at the same time we strive for easier entry and usage for beginners or people not so used to CUDA's programming model – https://github.com/taichi-dev/taichi/discussions/8184 https://github.com/taichi-dev/taichi/discussions/8184

BenoitP 2y ago

This should be seen in light of the Great Differentiable Convergence™: NERFs backpropagating pixels colors into the volume, but also semantic information from the image label, embedded from an LLM reading a multimedia document. Or something like this. Anyway, wanna buy an NVIDIA GPU ;)?

deleted 2y ago

[deleted]

wallscratch 2y ago

Can anyone comment on how efficient the Warp code is compared to manually written / fine-tuned CUDA?

jokoon 2y ago

funny that now some softwares are hardware dependent OpenCL seems like it's just obsolete

pjmlp 2y ago
OpenCL has been obsolete for years, as Intel, AMD and Google never provided a proper development experience with good drivers. The fact that OpenCL 3.0 is basically OpenCL 1.0 rebranded, as acknwoledgement of OpenCL 2.0 adoption failure, doesn't help either.

TNWin 2y ago

Slightly related What's this community's take on Triton? https://openai.com/index/triton/ https://openai.com/index/triton/ Are there better alternatives?

beebmam 2y ago

Why Python? I really don't understand this choice of language other than accessibility.

danielmarkbruce 2y ago
Because accessibility.
mkl 2y ago
I think you answered your own question there. Python is very accessible, very popular, and already widely used for GPU-based things like machine learning.
pzo 2y ago
Huge ecosystem starting with numpy, pandas, mathplot et al for data science, pytorch, tensorflow, jax for ML, gradio, rerun for visualization, opencv, open3d for image/pointcloud processing, pyside for gui and others.
int_19h 2y ago
Because that's where the vast majority of DS/ML is already, and they are too busy to learn something else.
rldjbpin 2y ago
to me it goes beyond that. many leetcode grinders swear by specific data structures such as hashmaps, which python makes available as dictionaries. behind the sytax, there is plenty of heavy lifting for writing sophisticated code, when need be. that surely helps with the network effect.

bytesandbits 2y ago

How is this different than Triton?

jkbbwr 2y ago

I really wish python would stop being the go-to language for GPU orchestration or machine learning, having worked with it again recently for some proof of concepts its been a massive pain in the ass.

seydor 2y ago
We should have by now a new language for AI systems, not just frameworks
FrozenSynapse 2y ago
seeing as every big corp chooses it for their libraries, I'd say it's a skill issue

marmaduke 2y ago

Ive dredged though Julia, Numba, Jax, Futhark, looking a way to have good CPU performance in absence of GPU, and I'm not really happy with any of them. Especially given how many want you to lug LLVM along with. A recent simulation code when pushed with gcc openmp-simd matched performance on a 13900K vs jax.jit on a rtx 4090. This case worked because the overall computation can be structured into pieces that fit in L1/L2 cache, but I had to spend a ton of time writing the C code, whereas jax.jit was too easy. So I'd still like to see something like this but which really works for CPU as well.

mccoyb 2y ago
Agreed, JAX is specialized for GPU computation -- I'd really like similar capabilities with more permissive constructs, maybe even co-effect tagging of pieces of code (which part goes on GPU, which part goes on CPU), etc. I've thought about extending JAX with custom primitives and a custom lowering process to support constructs which work on CPU (but don't work on GPU) -- but if I did that, and wanted a nice programmable substrate -- I'd need to define my own version of abstract tracing (because necessarily, permissive CPU constructs might imply array type permissiveness like dynamic shapes, etc). You start heading towards something that looks like Julia -- the problem (for my work) with Julia is that it doesn't support composable transformations like JAX does. Julia + JAX might be offered as a solution -- but it's quite unsatisfying to me.