4 ms·

Linux 6.3 Introducing Hardware Noise “Hwnoise” Tool

145 points by isaacfrond 4y ago

ilkkao 4y ago
Needed to find this document to understand what noise means in this context https://www.kernel.org/doc/html/next/trace/osnoise-tracer.html https://www.kernel.org/doc/html/next/trace/osnoise-tracer.ht...
- jmclnx 4y ago
  I read this but I do not get the point of OSNOISE. I was thinking it would be used for better random number generation but the doc does not seem to confirm that.
  - monocasa 4y ago
    It's about measuring jitter caused by the OS. When you have a large message passing system like you see in HPC environments, delays in processing exactly the right messages add up to huge failures of utilization because of the dependency graph of work to be done. In the past I had even seen national labs write their own Linux syscall compatible kernels in order to combat jitter. For example: https://github.com/HobbesOSR/kitten https://github.com/HobbesOSR/kitten
    - guenthert 4y ago
      Afaik (Linux compatible) Lightweight kernels (LWK) have been used in the past in lieu of Linux in (single application) HPC scenarios in order to reduce context switching and consequential cache misses and TLB thrashing. I wouldn't know why 'the right message' needs to be processed in a HPC application (as opposed to real time applications). Where can I learn more about such?
  - jabl 4y ago
    It's about jitter caused by the OS, or in this case HW (e.g. due to SMM or similar shenanigans), that can be problematic for real time or HPC applications.
- taneq 4y ago
  Thanks! Even this still defines noise only in terms of 'interference', and then takes a bit of digging to infer that it's something to do with timing and interrupts, and so it's probably jitter in execution time rather than referring to RF interference or electrical noise. I'm no kernel hacker but I've spent a lot of time close to the hardware and I've never heard jitter referred to as 'noise'.
  - javier_e06 4y ago
    I agree indeed. In statistical analysis or signal processing noise is described as an element presenting data/energy other than energy/data to be observed. I guess any data collecting system can call noise anything that is filtered out from that being measured.
- gorkish 4y ago
  Windows folks may be familiar with LatencyMon or DPC Checker which do similar things on windows to find misbehaving hardware or drivers which can cause issues for realtime/pseudo-realtime applications like AV capture/mixing.
rurban 4y ago
If they are talking about jitter they should call it jitter. Noise is something completely different. The RT folks will just shake their heads.
- 2OEH8eoCRo0 4y ago
  Jitter deals with signals. At this level we are not dealing with signals but operating system tasks.
  - rurban 4y ago
    The main cause of jitter are of course external interrupts, bad hardware or drivers. The main task of the OS is to provide drivers which do not block too long. Internal kernel book-keeping tasks of course also, but the main point of OS tasks are driver requests, interrupting system tasks. Calling it noise is just strange. Noise is usually the rand() source.
- mlyle 4y ago
  Jitter and phase noise are closely related. Here the units are a bit weird, but we're talking about accumulated noise. Systems seek to reach 0. This really means "no variability in performance", not just "tasks start at the right time". Performance noise is the correct term for what's measured. See e.g. https://www.researchgate.net/publication/353781546_Influence_of_Noisy_Environments_on_Behavior_of_HPC_Applications https://www.researchgate.net/publication/353781546_Influence... for a discussion.
_joel 4y ago
As did most others in this thread it seems, assumed this was for reducing RF noise, maybe reducing attack surface for some (tbf difficult) exploits
- sandworm101 4y ago
  I'm worse. I thought it might be for actual physical vibrations, such as can be measured by from HDD armatures or dedicated detectors. If for nothing else, such things might make for a good source of entropy.
  - _joel 4y ago
    Obligatory https://www.youtube.com/watch?v=tDacjrSCeq4 https://www.youtube.com/watch?v=tDacjrSCeq4
    - wafflemaker 4y ago
      as a norwegian factory worker i'm appaled by lack of ear protection.
  - heavyset_go 4y ago
    I thought it had to do with coil whine.
alangibson 4y ago
I'm definitely going to make use of this. I've currently got an Odroid N2+ running realtime Linux with ~20 use latency under 100% CPU load. I need something like this tool to get it even lower.
kobekobe 4y ago
jb\

145 points by isaacfrond 4y ago

Needed to find this document to understand what noise means in this context https://www.kernel.org/doc/html/next/trace/osnoise-tracer.html https://www.kernel.org/doc/html/next/trace/osnoise-tracer.ht...

jmclnx 4y ago
I read this but I do not get the point of OSNOISE. I was thinking it would be used for better random number generation but the doc does not seem to confirm that.
- monocasa 4y ago
  It's about measuring jitter caused by the OS. When you have a large message passing system like you see in HPC environments, delays in processing exactly the right messages add up to huge failures of utilization because of the dependency graph of work to be done. In the past I had even seen national labs write their own Linux syscall compatible kernels in order to combat jitter. For example: https://github.com/HobbesOSR/kitten https://github.com/HobbesOSR/kitten
  - guenthert 4y ago
    Afaik (Linux compatible) Lightweight kernels (LWK) have been used in the past in lieu of Linux in (single application) HPC scenarios in order to reduce context switching and consequential cache misses and TLB thrashing. I wouldn't know why 'the right message' needs to be processed in a HPC application (as opposed to real time applications). Where can I learn more about such?
- jabl 4y ago
  It's about jitter caused by the OS, or in this case HW (e.g. due to SMM or similar shenanigans), that can be problematic for real time or HPC applications.
taneq 4y ago
Thanks! Even this still defines noise only in terms of 'interference', and then takes a bit of digging to infer that it's something to do with timing and interrupts, and so it's probably jitter in execution time rather than referring to RF interference or electrical noise. I'm no kernel hacker but I've spent a lot of time close to the hardware and I've never heard jitter referred to as 'noise'.
- javier_e06 4y ago
  I agree indeed. In statistical analysis or signal processing noise is described as an element presenting data/energy other than energy/data to be observed. I guess any data collecting system can call noise anything that is filtered out from that being measured.
gorkish 4y ago
Windows folks may be familiar with LatencyMon or DPC Checker which do similar things on windows to find misbehaving hardware or drivers which can cause issues for realtime/pseudo-realtime applications like AV capture/mixing.

rurban 4y ago

If they are talking about jitter they should call it jitter. Noise is something completely different. The RT folks will just shake their heads.

2OEH8eoCRo0 4y ago
Jitter deals with signals. At this level we are not dealing with signals but operating system tasks.
- rurban 4y ago
  The main cause of jitter are of course external interrupts, bad hardware or drivers. The main task of the OS is to provide drivers which do not block too long. Internal kernel book-keeping tasks of course also, but the main point of OS tasks are driver requests, interrupting system tasks. Calling it noise is just strange. Noise is usually the rand() source.
mlyle 4y ago
Jitter and phase noise are closely related. Here the units are a bit weird, but we're talking about accumulated noise. Systems seek to reach 0. This really means "no variability in performance", not just "tasks start at the right time". Performance noise is the correct term for what's measured. See e.g. https://www.researchgate.net/publication/353781546_Influence_of_Noisy_Environments_on_Behavior_of_HPC_Applications https://www.researchgate.net/publication/353781546_Influence... for a discussion.

_joel 4y ago

As did most others in this thread it seems, assumed this was for reducing RF noise, maybe reducing attack surface for some (tbf difficult) exploits

sandworm101 4y ago
I'm worse. I thought it might be for actual physical vibrations, such as can be measured by from HDD armatures or dedicated detectors. If for nothing else, such things might make for a good source of entropy.
- _joel 4y ago
  Obligatory https://www.youtube.com/watch?v=tDacjrSCeq4 https://www.youtube.com/watch?v=tDacjrSCeq4
  - wafflemaker 4y ago
    as a norwegian factory worker i'm appaled by lack of ear protection.
- heavyset_go 4y ago
  I thought it had to do with coil whine.

alangibson 4y ago

I'm definitely going to make use of this. I've currently got an Odroid N2+ running realtime Linux with ~20 use latency under 100% CPU load. I need something like this tool to get it even lower.

kobekobe 4y ago

jb\