5 ms·

Userland rootkits are lame (2022)

45 points by qweqwe14 2y ago

from-nibly 2y ago
This sounds like a food youtuber trying to explain how easy cooking a Turkey is. Yeah sure in a vaccum maybe, but I don't spend all day cooking food. Ive got other stuff to do. Maybe its easy to find rootkits if you know what you are doing and use a statically compiled tcpdump. Do you know how many times I've done a TCP dump on my computer to check for root kits statically compiled or not? 0. I don't spend my time doing that. All it has to do is use less CPU and RAM than my corporate anti virus already does ( which is a lot ) and I'm never going to find it, cause im never gonna look.
- AdeptusAquinas 2y ago
  Yeah but this is more from the perspective of a blue teamer, someone actively looking for/protecting against malware or performing forensics. If you're not in that position, just have some good AV and don't do anything that might make you a target.
  - Joker_vD 2y ago
    > don't do anything that might make you a target. Like "don't get connected to the Internet"? That's a tall order.
    - denkmoon 2y ago
      You may not have noticed, everyone is on the internet. From the penthouses in new york to small villages in the savannah. Just being connected to the internet isn't noteworthy or targeted. Being targeted by an adversary is not the same as being hit by random drive-by attacks hitting every IP in the relatively limited IPv4 address space.
    - gizmo686 2y ago
      That is what the antivirus is for. If you are not a high value target, then your risk is virus that are being widely distributed; and those will quickly find their way into an AV database. If you are a high value target, AV is useless, since the attackers you care about will target you with bespoke mallare, that will not be in an AV database.
- deleted 2y ago
  [deleted]
- varenc 2y ago
  Agreed that it still seems decently hard to find if you aren't looking for it. Though the tone of the blog post made a lot more sense to me when I saw the over-top-top title of the article[0] it's responding to: 'Analysis of a New, Nearly-Impossible-to-Detect Linux Threat': [0] https://intezer.com/blog/research/new-linux-threat-symbiote/ https://intezer.com/blog/research/new-linux-threat-symbiote/
  - grugq 2y ago
    In retrospect I really should have included the title with the link.
- viraptor 2y ago
  The target here is not a random YouTube watcher though. If you're doing actual rootkit hunting, you're in a larger org with lots of other indicators in place. You'll have DNS and traffic logs (hopefully), some local agents, maybe managed devices. You're not going to find rootkits randomly by deciding to check with tcpdump. You're looking for them in the first place, because they tripped some other detector. That's a big prerequisite.
- riwsky 2y ago
  Cooking turkey in a vacuum… sous vide is your idea of easy? Props, dude!
  - abofh 2y ago
    Put meat in bag, put bag in water, receive protein? A turkey might be hard by size, but sous vide is basically the easy-bake of cooking, takes no effort or creativity during the cook and you can only really impact it before and after.
mike_d 2y ago
Adversaries live by the saying "if it works it isn't stupid."
- lotharcable 2y ago
  What they actually did was to implement kernel-level rootkits. This is why things 'secure boot' and requirement for signed drivers are a thing.
  - qweqwe14 2y ago
    The downside with kernel-level rootkits is you essentially have to compile it for many different kernel versions if you want it to work everywhere. I think I've read about some malware that literally contacted a server, sent the kernel version, and the server would compile the rootkit on demand.
    - abofh 2y ago
      And that's service right there - they tried to distribute the most compatible malicious kernel exploits for you, but sometimes you need a bespoke compilation for your system -- and these guys step up and make sure you get one! That's customer service, I think a lot of (all of) our trillion dollar overlords could do a thing or to and learn about providing reliable service.
  - worthless-trash 2y ago
    Kernel level code injection doesn't require code signing.
throwaway984393 2y ago
[dead]
rascul 2y ago
Is it really practical to just statically compile everything so you don't get a rootkit? Is that really the solution?
- wmf 2y ago
  I don't think that's what the article is suggesting. It's saying if you suspect a system to be infected you can install some statically-linked tools to diagnose it.
- worthless-trash 2y ago
  Some of the rootkits i've written attack statically compiled executables, so shrug.
  - nvy 2y ago
    Have you written about this? I'm fascinated by exploit dev.
    - qweqwe14 2y ago
      In case you're interested, it's usually done via ptrace by attaching to every process and modifiying the syscall arguments every time it makes one. There are performance issues associated with that (ptracing is rather expensive) and IMO it's more complex than LD_PRELOAD. Furthermore, ptrace may be disabled altogether on some installations, even for root. See yama_ptrace_scope I wouldn't say it's a practical approach. Works for a cool demo, sure, but as an adversary I would be hesitant to use this widely.
      - grugq 2y ago
        And you can detect when you are being ptrace()d because a process cannot be ptrace()d twice. Unless they changed Linux again. There are also timing issues that show up, and you can do any number of anti-debugging tricks which would reveal that the environment is being manipulated. Which is an instant red flag. In general if the attacker is running at the same privilege level, you can probably evade it or at least detect it. I’m somewhat surprised there isn’t a basic forensics tool that automates all of these tests already. “sus: [-h] [-v] [-V] [-o file] [-O format] Sus tests for common indicators of compromise using on generic tests for common methods of implementing userland rootkits. It will check for LD_PRELOAD, ptrace(), inotify() and verify the system binaries match the upstream distribution hashsums. It can be used to dump the file system directly (warning, slow!) for comparison against the output of `find`. See EXAMPLES for more.” Implementation is left as an exercise for the reader.
benob 2y ago
What are uses of LD_PRELOAD that make us keep it?
- worthless-trash 2y ago
  Fixing bugs in a function in a library. Overriding libraries.
  - freeone3000 2y ago
    Patching the binary seems like a better way to do this.
    - worthless-trash 2y ago
      Better long term,but also much harder. Some anti cheat software will be tripped if you do this as they checksum the binaries on disk, that of course can be worked around so its just a matter of understanding your adversary.
    - saagarjha 2y ago
      Patching libraries is error-prone and annoying.
- deleted 2y ago
  [deleted]
- viraptor 2y ago
  Replacing allocators https://github.com/jemalloc/jemalloc/wiki/Getting-Started https://github.com/jemalloc/jemalloc/wiki/Getting-Started Pretending you're running on a remote system https://github.com/viraptor/libremotec https://github.com/viraptor/libremotec Cheat in games https://github.com/ProfessionallyEvil/LD_PRELOAD-rand-Hijack-Example https://github.com/ProfessionallyEvil/LD_PRELOAD-rand-Hijack... Fixing bugs where the upstream can't do it.
- LukeShu 2y ago
  Some uses of LD_PRELOAD off the top of my head: - GNU stdbuf (BSDs use an env-var instead of hijacking libc) - fakeroot - gprofng's data collector - For a long time it was necessary to use LD_PRELOADto hack around a bug in flashplayer because flashplayer used memcpy when it should have used memmove.
alkonaut 2y ago
If it’s weak, I’d expect a standard distribution to detect it by itself and inform the user. Only revealing itself to a defender who looks for it sounds like it’ll be working as intended in almost all cases.
notepad0x90 2y ago
ld_preload is the lazy way of doing userland rootkits. there are other ways. For example, you can patch libc, and by patch I mean there are different ways of patching. a simply binary patch is unreliable because it'd be pinned to a specific version but you can parse the elf symbols dynamically and patch them, even when libc is updated (assuming certain critical symbols are unlikely to ever change) so it becomes reliable. a lot of "static" programs still dynamically load libc. There is also a more aggressive but easily detectable approach, just patch every elf file on the system so that that the _start/entry always does something to prevent discovery of the malware like messing with the plt so that all function calls are proxied through rootkit code, where the rootkit filters for parameters and return values it doesn't like. Kernel rookits are great but secure boot is also a thing, if implemented correctly it would be very difficult to beat. you don't even have to go full on rootkit, just a stealthy implant running as root via a hijacked dynamic library (doesn't have to be libc) will do. this isn't windows, it's not like there is a reliable/common way of making sure system libraries are legit (like authenticode/pe signing). The package managers make sure the correct binaries are being installed but they don't have something like sfc on windows (system file checker) that refuses to let the system boot if files are corrupt and/or restore tampered system executables.
- grugq 2y ago
  Bringing your own static linked busybox will still evade that rootkit. If the attacker has modified the environment to present a specific view of system state, bringing your own environment defeats it. There are tricks which are better than modifying things to hide. For example, there is a race condition between opendir() and readdir() which you can win by using inotify(). Then you can unlink() whatever, wait a while, then link() it back in. During that time it will be deleted and thus invisible to any detection. (I saw a demo of this 12 years ago, so I might be misremembering a bit. I know it used inotify() and unlink())
  - worthless-trash 2y ago
    Not if your running processes infect running userspace processes.
  - 2y ago
grugq 2y ago
Author here. The context of this post is somewhat important. It is a direct response to a post titled: Symbiote Deep-Dive: Analysis of a New, Nearly-Impossible-to-Detect Linux Threat Userland rootkits are not “nearly-impossible-to-detect.” They are not novel, they are not impossible to detect, and they are not the pinnacle of hacker techniques. I felt that it was worth pointing out that the history of userland rootkits goes back a ways and that they were very easy to detect because they rely on proxying all access to the system. If you bypass the hook they use to enter their proxy, they you evade them entirely. Forensic and incident response guides used to advise using static linked binaries for exactly this reason. There are guides from the 1990s telling people to do this because userland rootkits were an issue (before kernel rootkits everyone used userland rootkits.) Here is an example from 2013 which points out that you can’t trust any binaries/libraries on the potentially compromised machine and should use statically linked tools. [0] LD_PRELOAD rootkits are not new and they are not nearly-impossible-to-detect to detect. My post listed a number of ways to detect them, all of which have been known for decades. [0] https://www.forensicfocus.com/forums/general/trusted-static-linked-binaries-for-linux-live-forensic/ https://www.forensicfocus.com/forums/general/trusted-static-...
- fullspectrumdev 2y ago
  > they rely on proxying all access to the system It’s worth noting that this has a measurable and enormous impact on system performance, because they usually are adding a bunch of strcmp or similar to every incantation of a bunch of different libc calls.

45 points by qweqwe14 2y ago

This sounds like a food youtuber trying to explain how easy cooking a Turkey is. Yeah sure in a vaccum maybe, but I don't spend all day cooking food. Ive got other stuff to do. Maybe its easy to find rootkits if you know what you are doing and use a statically compiled tcpdump. Do you know how many times I've done a TCP dump on my computer to check for root kits statically compiled or not? 0. I don't spend my time doing that. All it has to do is use less CPU and RAM than my corporate anti virus already does ( which is a lot ) and I'm never going to find it, cause im never gonna look.

AdeptusAquinas 2y ago
Yeah but this is more from the perspective of a blue teamer, someone actively looking for/protecting against malware or performing forensics. If you're not in that position, just have some good AV and don't do anything that might make you a target.
- Joker_vD 2y ago
  > don't do anything that might make you a target. Like "don't get connected to the Internet"? That's a tall order.
  - denkmoon 2y ago
    You may not have noticed, everyone is on the internet. From the penthouses in new york to small villages in the savannah. Just being connected to the internet isn't noteworthy or targeted. Being targeted by an adversary is not the same as being hit by random drive-by attacks hitting every IP in the relatively limited IPv4 address space.
  - gizmo686 2y ago
    That is what the antivirus is for. If you are not a high value target, then your risk is virus that are being widely distributed; and those will quickly find their way into an AV database. If you are a high value target, AV is useless, since the attackers you care about will target you with bespoke mallare, that will not be in an AV database.
deleted 2y ago
[deleted]
varenc 2y ago
Agreed that it still seems decently hard to find if you aren't looking for it. Though the tone of the blog post made a lot more sense to me when I saw the over-top-top title of the article[0] it's responding to: 'Analysis of a New, Nearly-Impossible-to-Detect Linux Threat': [0] https://intezer.com/blog/research/new-linux-threat-symbiote/ https://intezer.com/blog/research/new-linux-threat-symbiote/
- grugq 2y ago
  In retrospect I really should have included the title with the link.
viraptor 2y ago
The target here is not a random YouTube watcher though. If you're doing actual rootkit hunting, you're in a larger org with lots of other indicators in place. You'll have DNS and traffic logs (hopefully), some local agents, maybe managed devices. You're not going to find rootkits randomly by deciding to check with tcpdump. You're looking for them in the first place, because they tripped some other detector. That's a big prerequisite.
riwsky 2y ago
Cooking turkey in a vacuum… sous vide is your idea of easy? Props, dude!
- abofh 2y ago
  Put meat in bag, put bag in water, receive protein? A turkey might be hard by size, but sous vide is basically the easy-bake of cooking, takes no effort or creativity during the cook and you can only really impact it before and after.

mike_d 2y ago

Adversaries live by the saying "if it works it isn't stupid."

lotharcable 2y ago
What they actually did was to implement kernel-level rootkits. This is why things 'secure boot' and requirement for signed drivers are a thing.
- qweqwe14 2y ago
  The downside with kernel-level rootkits is you essentially have to compile it for many different kernel versions if you want it to work everywhere. I think I've read about some malware that literally contacted a server, sent the kernel version, and the server would compile the rootkit on demand.
  - abofh 2y ago
    And that's service right there - they tried to distribute the most compatible malicious kernel exploits for you, but sometimes you need a bespoke compilation for your system -- and these guys step up and make sure you get one! That's customer service, I think a lot of (all of) our trillion dollar overlords could do a thing or to and learn about providing reliable service.
- worthless-trash 2y ago
  Kernel level code injection doesn't require code signing.

throwaway984393 2y ago

[dead]

rascul 2y ago

Is it really practical to just statically compile everything so you don't get a rootkit? Is that really the solution?

wmf 2y ago
I don't think that's what the article is suggesting. It's saying if you suspect a system to be infected you can install some statically-linked tools to diagnose it.
worthless-trash 2y ago
Some of the rootkits i've written attack statically compiled executables, so shrug.
- nvy 2y ago
  Have you written about this? I'm fascinated by exploit dev.
  - qweqwe14 2y ago
    In case you're interested, it's usually done via ptrace by attaching to every process and modifiying the syscall arguments every time it makes one. There are performance issues associated with that (ptracing is rather expensive) and IMO it's more complex than LD_PRELOAD. Furthermore, ptrace may be disabled altogether on some installations, even for root. See yama_ptrace_scope I wouldn't say it's a practical approach. Works for a cool demo, sure, but as an adversary I would be hesitant to use this widely.
    - grugq 2y ago
      And you can detect when you are being ptrace()d because a process cannot be ptrace()d twice. Unless they changed Linux again. There are also timing issues that show up, and you can do any number of anti-debugging tricks which would reveal that the environment is being manipulated. Which is an instant red flag. In general if the attacker is running at the same privilege level, you can probably evade it or at least detect it. I’m somewhat surprised there isn’t a basic forensics tool that automates all of these tests already. “sus: [-h] [-v] [-V] [-o file] [-O format] Sus tests for common indicators of compromise using on generic tests for common methods of implementing userland rootkits. It will check for LD_PRELOAD, ptrace(), inotify() and verify the system binaries match the upstream distribution hashsums. It can be used to dump the file system directly (warning, slow!) for comparison against the output of `find`. See EXAMPLES for more.” Implementation is left as an exercise for the reader.

benob 2y ago

What are uses of LD_PRELOAD that make us keep it?

worthless-trash 2y ago
Fixing bugs in a function in a library. Overriding libraries.
- freeone3000 2y ago
  Patching the binary seems like a better way to do this.
  - worthless-trash 2y ago
    Better long term,but also much harder. Some anti cheat software will be tripped if you do this as they checksum the binaries on disk, that of course can be worked around so its just a matter of understanding your adversary.
  - saagarjha 2y ago
    Patching libraries is error-prone and annoying.
deleted 2y ago
[deleted]
viraptor 2y ago
Replacing allocators https://github.com/jemalloc/jemalloc/wiki/Getting-Started https://github.com/jemalloc/jemalloc/wiki/Getting-Started Pretending you're running on a remote system https://github.com/viraptor/libremotec https://github.com/viraptor/libremotec Cheat in games https://github.com/ProfessionallyEvil/LD_PRELOAD-rand-Hijack-Example https://github.com/ProfessionallyEvil/LD_PRELOAD-rand-Hijack... Fixing bugs where the upstream can't do it.
LukeShu 2y ago
Some uses of LD_PRELOAD off the top of my head: - GNU stdbuf (BSDs use an env-var instead of hijacking libc) - fakeroot - gprofng's data collector - For a long time it was necessary to use LD_PRELOADto hack around a bug in flashplayer because flashplayer used memcpy when it should have used memmove.

alkonaut 2y ago

If it’s weak, I’d expect a standard distribution to detect it by itself and inform the user. Only revealing itself to a defender who looks for it sounds like it’ll be working as intended in almost all cases.

notepad0x90 2y ago

ld_preload is the lazy way of doing userland rootkits. there are other ways. For example, you can patch libc, and by patch I mean there are different ways of patching. a simply binary patch is unreliable because it'd be pinned to a specific version but you can parse the elf symbols dynamically and patch them, even when libc is updated (assuming certain critical symbols are unlikely to ever change) so it becomes reliable. a lot of "static" programs still dynamically load libc. There is also a more aggressive but easily detectable approach, just patch every elf file on the system so that that the _start/entry always does something to prevent discovery of the malware like messing with the plt so that all function calls are proxied through rootkit code, where the rootkit filters for parameters and return values it doesn't like. Kernel rookits are great but secure boot is also a thing, if implemented correctly it would be very difficult to beat. you don't even have to go full on rootkit, just a stealthy implant running as root via a hijacked dynamic library (doesn't have to be libc) will do. this isn't windows, it's not like there is a reliable/common way of making sure system libraries are legit (like authenticode/pe signing). The package managers make sure the correct binaries are being installed but they don't have something like sfc on windows (system file checker) that refuses to let the system boot if files are corrupt and/or restore tampered system executables.

grugq 2y ago
Bringing your own static linked busybox will still evade that rootkit. If the attacker has modified the environment to present a specific view of system state, bringing your own environment defeats it. There are tricks which are better than modifying things to hide. For example, there is a race condition between opendir() and readdir() which you can win by using inotify(). Then you can unlink() whatever, wait a while, then link() it back in. During that time it will be deleted and thus invisible to any detection. (I saw a demo of this 12 years ago, so I might be misremembering a bit. I know it used inotify() and unlink())
- worthless-trash 2y ago
  Not if your running processes infect running userspace processes.
- 2y ago

grugq 2y ago

Author here. The context of this post is somewhat important. It is a direct response to a post titled: Symbiote Deep-Dive: Analysis of a New, Nearly-Impossible-to-Detect Linux Threat Userland rootkits are not “nearly-impossible-to-detect.” They are not novel, they are not impossible to detect, and they are not the pinnacle of hacker techniques. I felt that it was worth pointing out that the history of userland rootkits goes back a ways and that they were very easy to detect because they rely on proxying all access to the system. If you bypass the hook they use to enter their proxy, they you evade them entirely. Forensic and incident response guides used to advise using static linked binaries for exactly this reason. There are guides from the 1990s telling people to do this because userland rootkits were an issue (before kernel rootkits everyone used userland rootkits.) Here is an example from 2013 which points out that you can’t trust any binaries/libraries on the potentially compromised machine and should use statically linked tools. [0] LD_PRELOAD rootkits are not new and they are not nearly-impossible-to-detect to detect. My post listed a number of ways to detect them, all of which have been known for decades. [0] https://www.forensicfocus.com/forums/general/trusted-static-linked-binaries-for-linux-live-forensic/ https://www.forensicfocus.com/forums/general/trusted-static-...

fullspectrumdev 2y ago
> they rely on proxying all access to the system It’s worth noting that this has a measurable and enormous impact on system performance, because they usually are adding a bunch of strcmp or similar to every incantation of a bunch of different libc calls.