4 ms·
When root meets immutable: OpenBSD chflags vs. log tampering
- johnisgood 1y agoIt is not the same, but I do use "chattr +i" on a file (which applies the immutable attribute) on Linux to a file that otherwise would have been overwritten by programs that do not give a damn whether I want it to or not, and in my case it was easier to just make that file immutable, mainly: /etc/resolv.conf.
- knorker 1y ago[flagged]
- johnisgood 1y agoThat actually does not sound too far-fetched. I believe there is a high chance that it might eventually happen, at least to files systemd """cares""" about.
- JdeBP 1y agoIn reality, this is something that people have been doing to systemd-resolved in desperation since the late 2010s (e.g. https://unix.stackexchange.com/q/421977/5132 https://unix.stackexchange.com/q/421977/5132 for one of many examples), and systemd has not been changed in all of that time to undo this. It is actually an unnecessary act of desperation, as systemd does have a way of setting things up so that /etc/resolv.conf is entirely manually configured. It's just (a) not completely obvious to deduce from the systemd doco, and (b) something that gets the blame for things that actually lie elsewhere in the system that also need disconnection, like /etc/nsswitch.conf giving nss-resolve priority over nss-dns. That that situation remains the case today (five years after I wrote https://unix.stackexchange.com/a/612434/5132 https://unix.stackexchange.com/a/612434/5132 for example, and almost a decade since the desperate making resolv.conf immutable idea took off) is a good indicator that the systemd developers aren't in reality in any hurry to force some hypothetical total ownership of /etc/resolv.conf by systemd.
- johnisgood 1y agoThe system on which I am using "chattr +i" is not even a systemd one. I forgot why I set the attribute in the first place. I might be able to figure it out, but it was the simplest and least time-consuming yet effective solution at the time. I think there were more than one reasons for it.
- JdeBP 1y agoMy educated guess would be either NetworkManager or resolveconf (or a DHCP client not even going through resolvconf), with fun with package upgrades resetting customizations as my second choice educated guess. (-:
- throw0101c 1y agoI find it baffling that distributions are making all of these system-level changes away from how a 'traditional' Linux/Unix does/did things, towards handling 'dynamic' situations, when a large portion of the installed of Linux is probably servers: i.e., why would you even consider the DHCP client situation for a server? As a sysadmin I want a static files that never change, and whose mtime is from when I installed the thing two years ago. I'll either launch vi to make changes or Ansible/Puppet/etc. Make the default the default as static-y as possible, and if someone is installing a personal system have a "laptop-config" or "desktop-config" package that adds all the dynamic bits separately.
- johnisgood 1y agoI think the reason was resolveconf and package updates, yes.
- deleted 1y ago[deleted]
- Levitating 1y agoI am curious, what files do you use it on specifically?
- mmsc 1y ago.bash_history
- claviola 1y agoAre you using systemd-resolved? If so, it's actually a symlink to /run/systemd/resolve/stub-resolv.conf, and all you need to do to manage it yourself is to turn it into a regular file of your own. This is explained in the admittedly very long commented out text above the actual directives.
- johnisgood 1y agoNo, it is a runit-based system. I think the issue was related to resolvconf + package updates overwriting the file. That said, I am sure your comment will be useful to some!
- comex 1y ago> Once the system reaches normal security level, even root cannot tamper with these logs without rebooting into single-user mode What stops the attacker from just editing /etc/rc.securelevel and then doing a normal reboot?
- TacticalCoder 1y ago> What stops the attacker from just editing /etc/rc.securelevel and then doing a normal reboot? Certainly a full reboot leaves more tracks than no full reboot? So it's harder to hide?
- kstrauser 1y agoMake that file immutable so that you can only edit it in single-user mode. This is definitely one of those “security vs convenience” situations where you can easily shoot yourself in the foot, but it’s great to have the option when you need it.
- dgl 1y agoExcept it is sourced from /etc/rc, and that’s a shell script which obviously depends on the shell and some other pieces. If you want an immutable base you kind of need to make the whole (base) system immutable (and that is possibly best designed as such to start with). I don’t think this is “security vs convenience”, I’d more argue it’s possible to think you’ve made this secure but you’ve missed something and haven’t configured it to be as secure as you think. An approach like others have suggested with remote logging is at least easier to reason about.
- h43z 1y agoDo I understand that correctly that in order for logs to rotate you have to reboot?
- deleted 1y ago[deleted]
- jelder 1y agoMy thoughts exactly. And couldn’t an attacker just fill the logging volume with uninteresting events to prevent certain other events from being recorded?
- jorvi 1y agoLog filtering via severity / keywords prevents this, assuming the logs are regularly and properly checked.
- gertrunde 1y agoThat would be where something like auditd would come in, configured so that if the audit logs location runs low on space (or out of space), it will halt the system. (Yes, quite harsh, but for some use cases it may be the right thing to do, i.e. to fail closed).
- louwrentius 1y agoIf you want immutable logs, you log to an external log server. Anything else seems security theater to me. That log server is properly firewalled/hardened so a hacked server can’t be used as a stepping stone to compromise the log server. Maybe you even have access restrictions in place for the log server so people can’t wipe their own misdeeds (4-eyes principle). This is how it’s been done for 35+ years, nothing special about this.
- holowoodman 1y agoYes, so much this. It used to be that important logs (filtered by severity and keywords) were even continuously live-printed by a line printer, so that there was always a current paper copy of the really important stuff for forensics. See e.g. https://www.youtube.com/watch?v=FiEGoVzmyvs https://www.youtube.com/watch?v=FiEGoVzmyvs but dot-matrix was also used and at least a little less noisy.
- accrual 1y agotsch! tsch! tsch! tsch! "Ah, someone is trying to login as root again"
- pjmlp 1y agoExactly the right approach.
- eternauta3k 1y agoIs root prevented from directly writing to the underlying block device?
- dspillett 1y agoI feel this is fixating on the wrong problem. Even with immutable flags there are various ways an attacker with root access could, after getting what they want from the system, cover their tracks by trashing the whole system⁰, and as usual if someone has physical access all bets are off. I see filesystem level flags like that to be more tools to stop you or a bug accidentally doing something stupid, than to get in the way of a malicious action by someone else. While the standard might effectively call for immutable logs¹, he needs to read between the lines one step further: those logs do not need to be on the same machine. You could stream logs to another system that stores them immutably from the PoV of anyone except those with root or physical access to it. You still have a problem if an attacker gets access to both the source system(s) and the log sinks², there might be a latency issue meaning you could easily lose the last few log entries in the case of a complete disaster, and you have an extra moving part in your infrastructure to monitor, but it satisfies the requirement where immutable filesystem flags can not. ---- [0] Yes, you'll know something happened, and you might guess it was malicious and not random corruption, but enough tracks might be covered to stop you working out the initial who & how. [1] and some standars explicitly call for them [2] Careful granular access management should largely mitigate that risk. That could be a problem if you are a small organisation trying to protect against internal disgruntled admins³, but you could use a a 3rd party log-sink service in that case. [3] This may seem overly paranoid, but if it is required for the standard your target audience wants you to have a certificate for…, and TBH it isn't that paranoid.
- Rygian 1y agoThat point should not require "reading between the lines" and that's why other standards (e.g. PCI) require explicitly that the logs are sent to a separate "central server" that provides guarantees of immutability.
- dspillett 1y agoStandards like ISO27001 and such are deliberately not prescriptive in that way. You might argue that this makes them less useful, and I would agree… The standard states that you should do something about X, and perhaps that your choice of how to do X should have property Y, but won't go into thither specifics. All the certificate you have, of you have one, really says is that you seem to have covered the relevant points in what you decided to do, and that you are actually implementing what you decided to do. This is one of the reasons why, despite companies having a pile of certificates like that, large prospective clients send a huge questionnaire to anyone wishing to tender for a job: the questionnaire on post fills in the gaps by requesting further detail on how you implemented the requirements of the standard (and in many cases makes it obvious that their are wrong answers that you could give).
- bananapub 1y agoimmutable is a handy advisory feature, but the actual answer for log tampering is "get them off the box in to a different security domain", e.g. a log server this machine can't access and is securely backed up so logs that make it there can be fairly well trusted.
- mrtesthah 1y agomacOS has always had these chflags attributes, by the way, long before System Integrity Protection existed. Changing or removing the system-immutable/append-only flags required booting into single user mode. Even the macOS installer application itself was unable to clear them otherwise.
- spauldo 1y agoThat's not surprising given MacOS' lineage.
- naltun 1y agoGreat article.