4 ms·

> There is a lot of jargon and you do have to sit down and learn how things work, but once you do, deep learning is fairly simple. This is my realization too.

> There is a lot of jargon and you do have to sit down and learn how things work, but once you do, deep learning is fairly simple. This is my realization too. I think ML for SWE courses should focus on "translation" first. Like by "kernel" they mean this specific thing, not the normal meaning of kernel. This is similar to other fields like finance (which I'm working on). After you learn the language, it's actually not too terrible to understand.

anon291 3y ago
What's the normal meaning of 'kernel'? EDIT: Hacker news won't let me respond, but the answers below all seem to be because the original meaning has been lost on everyone. In English, the word 'kernel' means 'core'. An OS kernel is the core of an operating system. In linear algebra, the kernel of a matrix (or a linear transformation, same thing) is the set of vectors it maps to zero, which is also in a sense the 'core' of the mapping (in so far that zero can be seen to be at the 'core' of the vector space / number line). So actually, the definition is the same, it's just that the word kernel is rather rare, despite having a well understood meaning. Nevertheless, it is the kernel of many common English idioms such as a 'kernel of truth'. Going to ML kernels... the etymology is a bit convoluted. I believe they come from operator theory and support vector machines. But nevertheless, the 'normal' meaning of kernel works out because kernels are typically the core fundamental operations supported by a machine learning framework atop which the other operations are built. In that sense, regardless of the etymology, the name actually fits.
- deleted 3y ago
  [deleted]
- ozr 3y ago
  The bit of code at the center of an operating system that mediates access between userspace code and the hardware. Whereas in CUDA programming, a kernel is just the code running on the device. When I first heard kernel in relation to CUDA programming, I expected it to be a) the actual OS kernel, and then b) something akin to a GPU driver. I can see how NVIDIA got there, but it's not immediately obvious if you're coming from a SWE context.
  - tayo42 3y ago
    Kernel is also used in convolutions... Ugh
  - joecool1029 3y ago
    > Whereas in CUDA programming, a kernel is just the code running on the device. When I first heard kernel in relation to CUDA programming, I expected it to be a) the actual OS kernel, and then b) something akin to a GPU driver. I think this is still not the machine learning 'kernel' being talked about. While you'd inevitably need to know about this for tooling in the space, looks like kernel is also a mathematical term.
- noodlesUK 3y ago
  I think that to most software people kernel means something like the Linux kernel, but to an ML engineer it is a specific mathematical term-of-art.
- wholinator2 3y ago
  Well, I'm not actually sure which one they're referring to but there is two meanings that I know of. There's the Software Developer Kernel which is the core of an operating system, and there's the Linear Algebra Kernel which is the null space of a map. Most of us here are familiar with the operating system type but the linear algebra kernel describes the set of vectors which are sent to the null vector (0) by your transformation/map/operation. It's very useful in understanding what exactly your operator does and how it changes a space.
- blackbear_ 3y ago
  In a ML context kernel would probably be understood as a way to quantify similarity between different examples: https://en.m.wikipedia.org/wiki/Kernel_method https://en.m.wikipedia.org/wiki/Kernel_method But it was hilarious to see how many other "normal" meanings came up in the comments.
  - tnecniv 3y ago
    Normal is just the first one you learn. Hell, in math, normal even has multiple meanings. You have the normal distribution and surface normals for example
- dpig_ 3y ago
  Although, funnily enough, we don't use kernel to mean 'core' in the idiom 'kernel of truth.'
  - anon291 3y ago
    How so? Saying something has a 'kernel of truth' means that, while there is some fluff, the core idea holds water.
    - dpig_ 3y ago
      I guess I have always interpreted the use of kernel there as meaning something small. But you are probably right.
fgdelcueto 3y ago
Of course you mean the "normal" meaning of a kernel pertaining to Linear Algebra, right? https://en.wikipedia.org/wiki/Kernel_(linear_algebra) https://en.wikipedia.org/wiki/Kernel_(linear_algebra) Of course I'm kidding. It's one of those terms that many fields adopt and give it completely different meaning. There's no "normal" meaning of `kernel`.
tnecniv 3y ago
Kernel is an overloaded term outside of software engineering. In linear algebra it means the null space, but that has no connection I’ve ever found to kernel methods or kernels more generally in functional analysis.

anon291 3y ago

What's the normal meaning of 'kernel'? EDIT: Hacker news won't let me respond, but the answers below all seem to be because the original meaning has been lost on everyone. In English, the word 'kernel' means 'core'. An OS kernel is the core of an operating system. In linear algebra, the kernel of a matrix (or a linear transformation, same thing) is the set of vectors it maps to zero, which is also in a sense the 'core' of the mapping (in so far that zero can be seen to be at the 'core' of the vector space / number line). So actually, the definition is the same, it's just that the word kernel is rather rare, despite having a well understood meaning. Nevertheless, it is the kernel of many common English idioms such as a 'kernel of truth'. Going to ML kernels... the etymology is a bit convoluted. I believe they come from operator theory and support vector machines. But nevertheless, the 'normal' meaning of kernel works out because kernels are typically the core fundamental operations supported by a machine learning framework atop which the other operations are built. In that sense, regardless of the etymology, the name actually fits.

deleted 3y ago
[deleted]
ozr 3y ago
The bit of code at the center of an operating system that mediates access between userspace code and the hardware. Whereas in CUDA programming, a kernel is just the code running on the device. When I first heard kernel in relation to CUDA programming, I expected it to be a) the actual OS kernel, and then b) something akin to a GPU driver. I can see how NVIDIA got there, but it's not immediately obvious if you're coming from a SWE context.
- tayo42 3y ago
  Kernel is also used in convolutions... Ugh
- joecool1029 3y ago
  > Whereas in CUDA programming, a kernel is just the code running on the device. When I first heard kernel in relation to CUDA programming, I expected it to be a) the actual OS kernel, and then b) something akin to a GPU driver. I think this is still not the machine learning 'kernel' being talked about. While you'd inevitably need to know about this for tooling in the space, looks like kernel is also a mathematical term.
noodlesUK 3y ago
I think that to most software people kernel means something like the Linux kernel, but to an ML engineer it is a specific mathematical term-of-art.
wholinator2 3y ago
Well, I'm not actually sure which one they're referring to but there is two meanings that I know of. There's the Software Developer Kernel which is the core of an operating system, and there's the Linear Algebra Kernel which is the null space of a map. Most of us here are familiar with the operating system type but the linear algebra kernel describes the set of vectors which are sent to the null vector (0) by your transformation/map/operation. It's very useful in understanding what exactly your operator does and how it changes a space.
blackbear_ 3y ago
In a ML context kernel would probably be understood as a way to quantify similarity between different examples: https://en.m.wikipedia.org/wiki/Kernel_method https://en.m.wikipedia.org/wiki/Kernel_method But it was hilarious to see how many other "normal" meanings came up in the comments.
- tnecniv 3y ago
  Normal is just the first one you learn. Hell, in math, normal even has multiple meanings. You have the normal distribution and surface normals for example
dpig_ 3y ago
Although, funnily enough, we don't use kernel to mean 'core' in the idiom 'kernel of truth.'
- anon291 3y ago
  How so? Saying something has a 'kernel of truth' means that, while there is some fluff, the core idea holds water.
  - dpig_ 3y ago
    I guess I have always interpreted the use of kernel there as meaning something small. But you are probably right.

fgdelcueto 3y ago

Of course you mean the "normal" meaning of a kernel pertaining to Linear Algebra, right? https://en.wikipedia.org/wiki/Kernel_(linear_algebra) https://en.wikipedia.org/wiki/Kernel_(linear_algebra) Of course I'm kidding. It's one of those terms that many fields adopt and give it completely different meaning. There's no "normal" meaning of `kernel`.

tnecniv 3y ago

Kernel is an overloaded term outside of software engineering. In linear algebra it means the null space, but that has no connection I’ve ever found to kernel methods or kernels more generally in functional analysis.