4 ms·

My summary (from someone who is not in the field but likes backpropagation): The core idea behind this type of approach ("parametric encoding") is that you lea

My summary (from someone who is not in the field but likes backpropagation): The core idea behind this type of approach ("parametric encoding") is that you learn a scene as some spatial data + a (small) neural network. For example, a 128^3 grid of data values and a 10k parameter model. In the forward pass you feed whatever data is at the voxel(s) in question to the network, and the backward pass updates both the network and the same voxel(s). The innovation in this paper is in how the spatial data is represented. Prior work includes dense grids, multi-resolution grids and octrees to name some - but all of them are either GPU-unfriendly or waste parameters on empty space. They figured that they can just hash the coordinates and use them directly as an index into a data array (edit: A multi-resolution stack of data arrays - sorry for not getting this right initially), with hash collisions left to the network to figure out (it's gonna figure out whether there's a collision on fine layer through info from the coarser ones, I guess). (Relatively) few parameters + GPU-friendly data structure = fast training. Tempted to try and implement this myself...

amelius 5y ago
Isn't the effect of hashing the same as sampling more coarsely?
- MauranKilom 5y ago
  I think the key here is that e.g. surface information only grows at O(N²) rate whereas number of grid points scales as O(N³). The hash function approach means your arrays will be filled with detailed information densely, whereas sampling coarsely would still leave most of the array with "nothing here" information. Your comment made me realize that I forgot to mention the multi-resolution aspect of their hash encoding (there are several data arrays corresponding to different resolutions - coarse ones are 1:1 indexed but finer ones have hash collisions for the network to deal with). It's in the title, but I should still include it.
ReactiveJelly 5y ago
If it's so fast, I'd like to see it working on a smaller scale on a CPU. Every new deep learning paper that comes out, I'm disappointed that it needs... - A $500-$1,000 GPU - A huge proprietary NVidia driver - Some odd language or language extensions, usually CUDA - Python
- rdedev 5y ago
  Just curious. What is your objections wrt the last 3 points ?
  - ReactiveJelly 5y ago
    I like FOSS a lot. Normal programming languages have relatively small downloads and run on normal CPUs dating about 10 years back with almost no issue. GPU workloads always want some odd driver that has a gigantic download, and they're constantly coming up with new reasons to force you to the newest APIs, which means you have to buy new chips that have the right architecture or firmware for the new APIs. So I have to buy this co-processor, and then I can't even treat it like a black box that I send commands to, I need a gigabyte-scale SDK or something to issue the commands on my behalf. I can't stand it. It's as if there was a tiny window when programming was simple, after I learned about FOSS, and before GPGPU caught on. As if the personal computer really will turn out to have been a fad.
    - astrange 5y ago
      This research uses Nvidia products because it was released by Nvidia, who have hired approximately every graphics researcher out there.
    - olliej 5y ago
      Ok, GPGPU isn’t “general purpose” in the basic sense, it means “not just graphics”. No CPU is going to be able to get performance in NNs that matches that of a GPU. A CPU simply cannot do the work. The closest a “general” CPU gets to that kind of thing are the big vector machines like the old Crays or Itanium’s packet architecture. Programming for either of those architectures is non trivial, and for normal software those architectures are slower than normal CPUs. Despite the trade offs those systems made, consumer GPUs ended up with better performance because a lot of the things and general CPU has to do interfere with performance of pure numerical computation.
- jonas21 5y ago
  Why? The point of research is to push the limits of what's possible, not to build something that runs on every single platform. I find it remarkable that most recent deep learning papers release the source code needed to reproduce their result -- and even more remarkable that many papers, like this one, can be reproduced on hardware that a hobbyist can afford. And if you'd like this to run on a CPU, you're welcome to port it. The code is open source after all.

amelius 5y ago

Isn't the effect of hashing the same as sampling more coarsely?

MauranKilom 5y ago
I think the key here is that e.g. surface information only grows at O(N²) rate whereas number of grid points scales as O(N³). The hash function approach means your arrays will be filled with detailed information densely, whereas sampling coarsely would still leave most of the array with "nothing here" information. Your comment made me realize that I forgot to mention the multi-resolution aspect of their hash encoding (there are several data arrays corresponding to different resolutions - coarse ones are 1:1 indexed but finer ones have hash collisions for the network to deal with). It's in the title, but I should still include it.

ReactiveJelly 5y ago

If it's so fast, I'd like to see it working on a smaller scale on a CPU. Every new deep learning paper that comes out, I'm disappointed that it needs... - A $500-$1,000 GPU - A huge proprietary NVidia driver - Some odd language or language extensions, usually CUDA - Python

rdedev 5y ago
Just curious. What is your objections wrt the last 3 points ?
- ReactiveJelly 5y ago
  I like FOSS a lot. Normal programming languages have relatively small downloads and run on normal CPUs dating about 10 years back with almost no issue. GPU workloads always want some odd driver that has a gigantic download, and they're constantly coming up with new reasons to force you to the newest APIs, which means you have to buy new chips that have the right architecture or firmware for the new APIs. So I have to buy this co-processor, and then I can't even treat it like a black box that I send commands to, I need a gigabyte-scale SDK or something to issue the commands on my behalf. I can't stand it. It's as if there was a tiny window when programming was simple, after I learned about FOSS, and before GPGPU caught on. As if the personal computer really will turn out to have been a fad.
  - astrange 5y ago
    This research uses Nvidia products because it was released by Nvidia, who have hired approximately every graphics researcher out there.
  - olliej 5y ago
    Ok, GPGPU isn’t “general purpose” in the basic sense, it means “not just graphics”. No CPU is going to be able to get performance in NNs that matches that of a GPU. A CPU simply cannot do the work. The closest a “general” CPU gets to that kind of thing are the big vector machines like the old Crays or Itanium’s packet architecture. Programming for either of those architectures is non trivial, and for normal software those architectures are slower than normal CPUs. Despite the trade offs those systems made, consumer GPUs ended up with better performance because a lot of the things and general CPU has to do interfere with performance of pure numerical computation.
jonas21 5y ago
Why? The point of research is to push the limits of what's possible, not to build something that runs on every single platform. I find it remarkable that most recent deep learning papers release the source code needed to reproduce their result -- and even more remarkable that many papers, like this one, can be reproduced on hardware that a hobbyist can afford. And if you'd like this to run on a CPU, you're welcome to port it. The code is open source after all.