4 ms·

what does he mean with regards to GPUs and CPUs "emulating"? they are computation machines doing computations. an algorithm running on one or the other isn't em

what does he mean with regards to GPUs and CPUs "emulating"? they are computation machines doing computations. an algorithm running on one or the other isn't emulating anything in this context. running on "bare metal" is just an optimization along a certain axis. it isn't any more real of a computation. and talking about this now seems like another desperate attempt at generating hype.

rhema 8y ago
I think he is saying that CPUs and GPUs are generalized computational mechanisms. They have increasingly complex overhead that attempt to optimize general problems. By limiting the context of use, specialized hardware can perform better than CPU/GPU. I think his thinking relates strongly to ASIC crypto mining hardware https://en.bitcoin.it/wiki/ASIC https://en.bitcoin.it/wiki/ASIC . "bare metal" is just a shorthand for the level of access to hardware that does not require a kernel/OS.
Ultimatt 8y ago
Ultimately its down to dimensionality CPUs (at least originally) were designed for scalar calculation, you loaded something a piece at a time from RAM and did a single scalar calculation before putting it back into RAM, rinse repeat. A GPU works in a more vectorised/matrix level way with essentially a pipeline of linear transforms on a set of data values all at once. The issue with both of these two is they still have a linear memory model, because that works for more serial lower dimension operations, usually three dimensions for a GPU maybe a couple more. Networks or graphs as objects aren't neatly linear in memory or nicely shaped into regular dimensions for doing pipelined vectorised calculations. Though the GPU model is a little closer than the CPU one. Graph algorithms its not random access or serial access either its /structured/ access based on the graphs connections. So ideally you want memory shaped and optimised for this sort of access, as well as computation well linked to utilise this rearchitected RAM. Many companies have been exploring newer architectures for dealing with ANN calculations that inherently benefit from changing the model of memory access and computation. So you can "emulate" as in reformulate the mathematical representation of a graph into a list or a matrix, an adjacency list based algorithm would work well on a CPU and an adjacency matrix based one would work better on a GPU. But neither of those fit perfectly to either model they are just dimensionally the nearest with many tradeoffs you have to make for each.
tokipin 8y ago
As a naive example, if in a CPU you are doing some specific 3-step algorithm which would normally require 3 clock cycles, you can instead bake that algorithm into a circuit along with the memory it needs that will run in 1 cycle. In Elon's terminology, the CPU has to "emulate" what the baked-in circuit simply is. But from the sounds of it this chip goes beyond that. It sounds like some kind of RAM/GPU hybrid with matrix-specific instructions specifically targeting NN inference for Tesla's specific style of NNs, which as a first guess I take to mean "lots and lots of layers." In the call they mentioned increasing their investment in the chip team and technology, which is a sign that they're confident it's going to pay off.
- nikofeyn 8y ago
  what i was getting at that the term "emulation" is rarely used, at least from my experience, in this context except in cases where something is truly faking it like game emulation, OS emulation, hardware emulation, etc. i personally feel like musk is using this term as a marketing and hype technique, that all of the sudden CPUs and GPUs are "faking it" for these applications.

rhema 8y ago

I think he is saying that CPUs and GPUs are generalized computational mechanisms. They have increasingly complex overhead that attempt to optimize general problems. By limiting the context of use, specialized hardware can perform better than CPU/GPU. I think his thinking relates strongly to ASIC crypto mining hardware https://en.bitcoin.it/wiki/ASIC https://en.bitcoin.it/wiki/ASIC . "bare metal" is just a shorthand for the level of access to hardware that does not require a kernel/OS.

Ultimatt 8y ago

Ultimately its down to dimensionality CPUs (at least originally) were designed for scalar calculation, you loaded something a piece at a time from RAM and did a single scalar calculation before putting it back into RAM, rinse repeat. A GPU works in a more vectorised/matrix level way with essentially a pipeline of linear transforms on a set of data values all at once. The issue with both of these two is they still have a linear memory model, because that works for more serial lower dimension operations, usually three dimensions for a GPU maybe a couple more. Networks or graphs as objects aren't neatly linear in memory or nicely shaped into regular dimensions for doing pipelined vectorised calculations. Though the GPU model is a little closer than the CPU one. Graph algorithms its not random access or serial access either its /structured/ access based on the graphs connections. So ideally you want memory shaped and optimised for this sort of access, as well as computation well linked to utilise this rearchitected RAM. Many companies have been exploring newer architectures for dealing with ANN calculations that inherently benefit from changing the model of memory access and computation. So you can "emulate" as in reformulate the mathematical representation of a graph into a list or a matrix, an adjacency list based algorithm would work well on a CPU and an adjacency matrix based one would work better on a GPU. But neither of those fit perfectly to either model they are just dimensionally the nearest with many tradeoffs you have to make for each.

tokipin 8y ago

As a naive example, if in a CPU you are doing some specific 3-step algorithm which would normally require 3 clock cycles, you can instead bake that algorithm into a circuit along with the memory it needs that will run in 1 cycle. In Elon's terminology, the CPU has to "emulate" what the baked-in circuit simply is. But from the sounds of it this chip goes beyond that. It sounds like some kind of RAM/GPU hybrid with matrix-specific instructions specifically targeting NN inference for Tesla's specific style of NNs, which as a first guess I take to mean "lots and lots of layers." In the call they mentioned increasing their investment in the chip team and technology, which is a sign that they're confident it's going to pay off.

nikofeyn 8y ago
what i was getting at that the term "emulation" is rarely used, at least from my experience, in this context except in cases where something is truly faking it like game emulation, OS emulation, hardware emulation, etc. i personally feel like musk is using this term as a marketing and hype technique, that all of the sudden CPUs and GPUs are "faking it" for these applications.