5 ms·

Nvidia announces CUDA 4.0

41 points by m0nastic 16y ago

m0nastic 16y ago
The listed link is PR, but here is also some more details about what 4.0 contains: http://developer.nvidia.com/object/cuda_4_0_RC_downloads.html http://developer.nvidia.com/object/cuda_4_0_RC_downloads.htm...
bryanlarsen 16y ago
anandtech has some good info, too: http://www.anandtech.com/show/4198/nvidia-announces-cuda-40 http://www.anandtech.com/show/4198/nvidia-announces-cuda-40
tmurray 16y ago
Been working on the API improvements for this for quite a while now. If there are any questions I can answer, I'll tell you whatever I can. (edit: in case it's not obvious, I work for NVIDIA on CUDA)
- deleted 16y ago
  [deleted]
- lrm242 16y ago
  I haven't done any CUDA programming, however I have wanted to dive in. I've been to some conferences where the breakpoint between whether CUDA gives enough benefit to the computation is largely driven by time to load a dataset into memory on the GPU. From what I've been told this has limited the use of CUDA to batch processing on large scale datasets. Is this still a limitation?
  - tmurray 16y ago
    PCIe can be a limitation, but there are a lot of ways to amortize the latency of copying data to/from the GPU. Overlapping transfers with kernels, direct load/store from system memory to a kernel, multiple kernels running on the chip at the same time--there are a lot of things you can do. But in general, you're looking at a runtime of at least a few hundred microseconds before you're going to be able to get a benefit from the GPU.
- marshray 16y ago
  Congrats on your upcoming major release! I'm sure it was a lot of work. I'm a detail freak so I worry about this kind of thing more than most, but wouldn't a unified address space tend to hide large bus transfers from the programmer? Will the APIs still let you anticipate where the costs are, or are developers dependent on Windows-only profiling tools?
  - tmurray 16y ago
    It's not a true unified address space in the way I think you're getting at. We carve out the address space, but you can't just directly dereference a GPU pointer from CPU code or vice-versa. What we guarantee is that you can determine the location of a pointer based only on the value of the pointer so you can make the appropriate API calls to do a copy if necessary.
    - iskander 16y ago
      If the programmer still has to perform manual copies, what's the advantage?
      - tmurray 16y ago
        You can take an arbitrary pointer value in a function, look up its location, and do the copy if necessary, instead of requiring multiple entry points for CPU and GPU pointers. This also simplifies multi-GPU programming a lot--copy from one arbitrary pointer to another and it just works, instead of having to specify which device the copies are going to or anything like that.
- iskander 16y ago
  One thing that completely puzzles me: How does dynamic memory allocation/deallocation in GPU kernels work? Also, will the programmer be able to use virtual functions in CUDA kernels? If so, does the usual thinking about warp divergence apply? That is--- should all threads in a warp ideally be calling the same function address?
  - tmurray 16y ago
    I think there's some preallocated memory for dynamic allocations from the device. As far as I know virtual functions work as well, and the usual thinking about warp divergence does apply.
    - iskander 16y ago
      Do you know if the dynamic allocation pool is per-SM or in global memory?
      - tmurray 16y ago
        Global memory.
- maximilian 16y ago
  Have you done any benchmarks regarding inter-GPU memory bandwidth (i.e. GPU1->GPU2) vs. GPU1->host->GPU2? I am working on multi-gpu problems, which are often throttled by bandwidth limitations. Does the new SDK have examples of this and the new unified memory architecture (which looks handy)?
matclayton 16y ago
If you are interested in GPU coding, checkout www.tidepowerd.com for a .net cross vendor system, also falls back to CPU and they are a startup :)
ericxtang 16y ago
CUDA has come a long way since it's birth as a student project.
- iskander 16y ago
  Who wrote the first version? I always assumed it was nvidia's version of Brook.
  - tmurray 16y ago
    CUDA and BrookGPU share the same creator, but there's not really anything else in common. The programming models are fundamentally different (multi-level memory hierarchy with sharing of local data with fast synchronization versus pure streaming).
babs474 16y ago
Is anybody who is interested in CUDA also interested in the idea of GPGPU computation in the browser using webGL? Here is a link to a proof of concept I did a while ago http://learningwebgl.com/blog/?p=1828 http://learningwebgl.com/blog/?p=1828 I think the concept is pretty compelling but I don't get that much traction when I bring it up. Any thoughts?
ciupicri 16y ago
Seeing all the buzz about CUDA I can not stop wondering if OpenCL has any future. Does it?
- vilya 16y ago
  It does, I think. OpenCL's trump card is that it works on non-Nvidia devices as well. I believe that'll ultimately be enough for it to win out, despite the technical merit being in favour of CUDA right now.
phlux 16y ago
So forgive my ignorance, but would the following be of value in a system design/architecture. (I dont know what type of cards you have which you would use in this, such as a multi-GPU cored PCI-e card, so in my ignorance, I will assume these exist and are separate from a traditional "video card") I would be interested to see a system, as a brick, which has a mobo with multiple PCI-e slots, as many as one can get then fill it with both GPU based cards, as well as Fusion IO cards that provide extremely fast IOPs on the storage side. The main CPU in the box, is effectively used to boot the OS - all the other processing and IOPs occur on the GPUs and the Fusion IO cards as data storage and caching... Is this a realistic idea?
- lwat 16y ago
  GPUs can't run arbitrary code. You have to code specifically for them. They don't run operating systems or web servers or whatever. These are for massively parallel mathematical computations, not for system calls and threads and stuff like that.
  - phlux 16y ago
    Right, I do know that -- What Iam saying is that a system that uses both GPUs and the fusion IO cards would seem to provide a kickass platform for mass computation on lots of data in a small form factor.
    - lwat 16y ago
      Cuda rack servers are available for multiple vendors. http://www.geeks3d.com/20091009/cxt8000-nvidia-tesla-based-server-with-1920-cuda-cores-for-gpu-computing/ http://www.geeks3d.com/20091009/cxt8000-nvidia-tesla-based-s...
      - phlux 16y ago
        haha, When I was new to IT in Silicon Valley in 1997, we bought a bunch of clone servers from Colfax International - dual pentium II something-or-others, based on Super Micro mobos. We were running Windows NT with SQL Server as well as Linux (the founders of Linux Care, Dave Sifry and Art Tide set them up - we were doing EDI with sun, and were one of the first companies to use XML for the docs, which were manufacturing orders for copies of Sun OS -- we physically manufactured all their software (cd, manuals, boxes etc, and dropped shipped them to the customer) Anyway - after a few months we were having HORRID problems with the colfax boxes always locking up. They were about 12 or 14,000 bucks each at the time - which was insanely expensive for what we were getting. I think they had like a gig or ram or something small. We also had AS/400s which were many more times that, but that was for the JDE ERP system... We tried to work through the problems with Colfax, and finally called a meeting with them. They came in and my manager asks, "Colfax.... Colfax International. Where are your offices?" CI: "Sunnyvale." US: "So, any other offices" CI: "Nope - we are all based in the US" US: "I see, any international customers?" CI: "Nope, all in the bay area" US: "So why do you call yourselves "international"" CI: "...." --- Anyway, we ended up scrapping the boxes as high end IT workstations and bought name brand systems. Last time I bought clone HW. We even had some of the first VA Linux and Penguin systems -- when they were barely seen as anything but clones...

41 points by m0nastic 16y ago

The listed link is PR, but here is also some more details about what 4.0 contains: http://developer.nvidia.com/object/cuda_4_0_RC_downloads.html http://developer.nvidia.com/object/cuda_4_0_RC_downloads.htm...

bryanlarsen 16y ago

anandtech has some good info, too: http://www.anandtech.com/show/4198/nvidia-announces-cuda-40 http://www.anandtech.com/show/4198/nvidia-announces-cuda-40

tmurray 16y ago

Been working on the API improvements for this for quite a while now. If there are any questions I can answer, I'll tell you whatever I can. (edit: in case it's not obvious, I work for NVIDIA on CUDA)

deleted 16y ago
[deleted]
lrm242 16y ago
I haven't done any CUDA programming, however I have wanted to dive in. I've been to some conferences where the breakpoint between whether CUDA gives enough benefit to the computation is largely driven by time to load a dataset into memory on the GPU. From what I've been told this has limited the use of CUDA to batch processing on large scale datasets. Is this still a limitation?
- tmurray 16y ago
  PCIe can be a limitation, but there are a lot of ways to amortize the latency of copying data to/from the GPU. Overlapping transfers with kernels, direct load/store from system memory to a kernel, multiple kernels running on the chip at the same time--there are a lot of things you can do. But in general, you're looking at a runtime of at least a few hundred microseconds before you're going to be able to get a benefit from the GPU.
marshray 16y ago
Congrats on your upcoming major release! I'm sure it was a lot of work. I'm a detail freak so I worry about this kind of thing more than most, but wouldn't a unified address space tend to hide large bus transfers from the programmer? Will the APIs still let you anticipate where the costs are, or are developers dependent on Windows-only profiling tools?
- tmurray 16y ago
  It's not a true unified address space in the way I think you're getting at. We carve out the address space, but you can't just directly dereference a GPU pointer from CPU code or vice-versa. What we guarantee is that you can determine the location of a pointer based only on the value of the pointer so you can make the appropriate API calls to do a copy if necessary.
  - iskander 16y ago
    If the programmer still has to perform manual copies, what's the advantage?
    - tmurray 16y ago
      You can take an arbitrary pointer value in a function, look up its location, and do the copy if necessary, instead of requiring multiple entry points for CPU and GPU pointers. This also simplifies multi-GPU programming a lot--copy from one arbitrary pointer to another and it just works, instead of having to specify which device the copies are going to or anything like that.
iskander 16y ago
One thing that completely puzzles me: How does dynamic memory allocation/deallocation in GPU kernels work? Also, will the programmer be able to use virtual functions in CUDA kernels? If so, does the usual thinking about warp divergence apply? That is--- should all threads in a warp ideally be calling the same function address?
- tmurray 16y ago
  I think there's some preallocated memory for dynamic allocations from the device. As far as I know virtual functions work as well, and the usual thinking about warp divergence does apply.
  - iskander 16y ago
    Do you know if the dynamic allocation pool is per-SM or in global memory?
    - tmurray 16y ago
      Global memory.
maximilian 16y ago
Have you done any benchmarks regarding inter-GPU memory bandwidth (i.e. GPU1->GPU2) vs. GPU1->host->GPU2? I am working on multi-gpu problems, which are often throttled by bandwidth limitations. Does the new SDK have examples of this and the new unified memory architecture (which looks handy)?

matclayton 16y ago

If you are interested in GPU coding, checkout www.tidepowerd.com for a .net cross vendor system, also falls back to CPU and they are a startup :)

ericxtang 16y ago

CUDA has come a long way since it's birth as a student project.

iskander 16y ago
Who wrote the first version? I always assumed it was nvidia's version of Brook.
- tmurray 16y ago
  CUDA and BrookGPU share the same creator, but there's not really anything else in common. The programming models are fundamentally different (multi-level memory hierarchy with sharing of local data with fast synchronization versus pure streaming).

babs474 16y ago

Is anybody who is interested in CUDA also interested in the idea of GPGPU computation in the browser using webGL? Here is a link to a proof of concept I did a while ago http://learningwebgl.com/blog/?p=1828 http://learningwebgl.com/blog/?p=1828 I think the concept is pretty compelling but I don't get that much traction when I bring it up. Any thoughts?

ciupicri 16y ago

Seeing all the buzz about CUDA I can not stop wondering if OpenCL has any future. Does it?

vilya 16y ago
It does, I think. OpenCL's trump card is that it works on non-Nvidia devices as well. I believe that'll ultimately be enough for it to win out, despite the technical merit being in favour of CUDA right now.

phlux 16y ago

So forgive my ignorance, but would the following be of value in a system design/architecture. (I dont know what type of cards you have which you would use in this, such as a multi-GPU cored PCI-e card, so in my ignorance, I will assume these exist and are separate from a traditional "video card") I would be interested to see a system, as a brick, which has a mobo with multiple PCI-e slots, as many as one can get then fill it with both GPU based cards, as well as Fusion IO cards that provide extremely fast IOPs on the storage side. The main CPU in the box, is effectively used to boot the OS - all the other processing and IOPs occur on the GPUs and the Fusion IO cards as data storage and caching... Is this a realistic idea?

lwat 16y ago
GPUs can't run arbitrary code. You have to code specifically for them. They don't run operating systems or web servers or whatever. These are for massively parallel mathematical computations, not for system calls and threads and stuff like that.
- phlux 16y ago
  Right, I do know that -- What Iam saying is that a system that uses both GPUs and the fusion IO cards would seem to provide a kickass platform for mass computation on lots of data in a small form factor.
  - lwat 16y ago
    Cuda rack servers are available for multiple vendors. http://www.geeks3d.com/20091009/cxt8000-nvidia-tesla-based-server-with-1920-cuda-cores-for-gpu-computing/ http://www.geeks3d.com/20091009/cxt8000-nvidia-tesla-based-s...
    - phlux 16y ago
      haha, When I was new to IT in Silicon Valley in 1997, we bought a bunch of clone servers from Colfax International - dual pentium II something-or-others, based on Super Micro mobos. We were running Windows NT with SQL Server as well as Linux (the founders of Linux Care, Dave Sifry and Art Tide set them up - we were doing EDI with sun, and were one of the first companies to use XML for the docs, which were manufacturing orders for copies of Sun OS -- we physically manufactured all their software (cd, manuals, boxes etc, and dropped shipped them to the customer) Anyway - after a few months we were having HORRID problems with the colfax boxes always locking up. They were about 12 or 14,000 bucks each at the time - which was insanely expensive for what we were getting. I think they had like a gig or ram or something small. We also had AS/400s which were many more times that, but that was for the JDE ERP system... We tried to work through the problems with Colfax, and finally called a meeting with them. They came in and my manager asks, "Colfax.... Colfax International. Where are your offices?" CI: "Sunnyvale." US: "So, any other offices" CI: "Nope - we are all based in the US" US: "I see, any international customers?" CI: "Nope, all in the bay area" US: "So why do you call yourselves "international"" CI: "...." --- Anyway, we ended up scrapping the boxes as high end IT workstations and bought name brand systems. Last time I bought clone HW. We even had some of the first VA Linux and Penguin systems -- when they were barely seen as anything but clones...