6 ms·

OS X Auditor

348 points by evandrix 12y ago

rrggrr 12y ago
Great idea. Love to run it, but... crash and burn: ~/Library/Safari/LastSession.plist Traceback (most recent call last): File "osxauditor.py", line 1702, in <module> Main() File "osxauditor.py", line 1663, in Main ParseBrowsers() File "osxauditor.py", line 808, in ParseBrowsers ParseSafari() File "osxauditor.py", line 745, in ParseSafari ParseSafariProfile(User, UserSafariProfilePath) File "osxauditor.py", line 717, in ParseSafariProfile LastSession = LastSessionPlist["SessionWindows"][0]["TabStates"][0] File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC/objc/_convenience.py", line 451, in __getitem__objectAtIndex_ return container_unwrap(self.objectAtIndex_(idx), RuntimeError) IndexError: NSRangeException - -[__NSCFArray objectAtIndex:]: index (0) beyond bounds (0)
- bsmartt 12y ago
  I can't even get pyobjc running on 10.9..
  - rmetzler 12y ago
    I also have this problem. I'm not sure, but it might be caused by using brew python instead of the systems original python
- yohanatan 12y ago
  Looks like you have an empty session list. Have you ever used Safari? Quick workaround might be to pop open Safari once (and browse a little) and then try again.
  - mwfunk 12y ago
    Why on earth would anyone downvote this comment? The lack of a Safari user profile is the most obvious potential issue here, based on the backtrace.
    - w0rd-driven 12y ago
      That's the precise problem I had. Opening Safari got me one step closer and then another failure. I wanted to run all options but got tired of all these errors so I commented out ParseSafari, ParseFirefox, and ParseChrome calls inside the ParseBrowsers function. I could've left Firefox or Chrome but not wanting to shut Chrome down or not using Firefox much, I didn't want to go through this again for each browser.
- mdaniel 12y ago
  And that, folks, is why we don't do chained expressions. Variables increase readability and also offer one the opportunity to check a condition that should be true. This, KeyError, NoneType exceptions and its Java friend NullPointerException drive me batty. They are so simple to defend against, if the author just took the time.
  - nsxwolf 12y ago
    I love Optionals for this.
  - chc 12y ago
    Chaining itself isn't really the problem here IMO. Creating a variable for each level of the nested data structure and manually checking for the validity of each subsequent one would really, really not produce more readable code. It would turn a "get this value out of a data structure" expression into about 15 lines of code that each need to be individually read and understood. The problem is that Cocoa doesn't provide a great way to say "Access this stuff in this data structure unless it isn't there." An option type is really the best way to kill this breed of problem, and thankfully Apple is including one in Swift, though it's still somewhat green.
    - photojosh 12y ago
      That's a Obj-C problem, because it's being fixed with what Swift does with it's '?' on variables. :)
      - chc 12y ago
        Not exactly. This program would have the same problem in Swift, because rather than returning optionals, Swift arrays have the same explode-on-invalid-accesses behavior as NSArray.
  - arianvanp 12y ago
    Maybe monads make this kind of chaining safe.
  - 12y ago
mcescalante 12y ago
Very interested to try this out on my own machines to see the results. I know somebody who does computer security at a University and the staff there has been frustrated with the lack of available forensics tools for OSX, so this may nicely fill an empty niche for some industry people as well.
c0wl 12y ago
OSXAuditor is pretty dope and we used it a bunch at Yelp. Over time, we created what we think is an inspired next version - https://github.com/Yelp/osxcollector https://github.com/Yelp/osxcollector
- deleted 12y ago
  [deleted]
- traveller_57681 12y ago
  The passive scan approach has the single weakness that it appears to be effective, but mostly is security theatre. Take for example, when we travelling into Australia, and they asked if we had any criminal convictions - I didn't know that a conviction was still an entry requirement for the island!
- crag 12y ago
  I run OSXCollector and I get missing modules error. I'm not a python guy, but looking at the code, it requires: import Foundation, import calendar, import os, import sys, import shutil I assuming os, sys, and shutil are part of the system (are they?). What about the other two? Also, which version of Python do I run this under?
  - ddp 12y ago
    ...not a python user either. The README mentions 'pip' but I don't seem to have one of those either.
    - texuf 12y ago
      Here's a "non-magical introduction to pip" http://www.dabapps.com/blog/introduction-to-pip-and-virtualenv-python/ http://www.dabapps.com/blog/introduction-to-pip-and-virtuale... Pip is an excellent package management system.
    - deleted 12y ago
      [deleted]
  - 12y ago
korzun 12y ago
Cool idea. But using raw md5 hashes to verify against a blacklist is kind of useless. Especially now. You should be using smarter file signatures: http://hooked-on-mnemonics.blogspot.com/2011/01/intro-to-creating-anti-virus-signatures.html http://hooked-on-mnemonics.blogspot.com/2011/01/intro-to-cre...
- jnazario 12y ago
  so, if i understand it correctly you're saying create AV signatures (the link you point to is how to create ClamAV signatures). to do so in a worthwhile, comprehensive way would required the maintainer to basically become an AV company. possible, but requires a lot of effort to scale. by calling out to various services, the creator of this tool is essentially outsourcing maintaining a file reputation blacklist. MD5s are, despite their limitations, the lingua franca of the security industry. nearly everyone who provides a file reputation query service supports them (as opposed to SHA1s or other hashes like ssdeep). so, i think i get what you're saying, but i don't think it's a relevant suggestion here.
  - korzun 12y ago
    I'm pointing out that this tool is essentially useless from security perspective. Most of the payloads will have different hash results due to extensive use of packers. If you are going to get hit with variant #11929 before the online databases obtains a hash of it, this tool is not going to pick it up but it will tell you that you are secure.
  - hueving 12y ago
    >so, i think i get what you're saying, but i don't think it's a relevant suggestion here. It's pretty relevant. Without sending the whole file to the third party, the file reputation service isn't 'outsourced' as you suggest. Sending the MD5 will not do any good if the program makes non-deterministic modifications to its binary.
SmileyKeith 12y ago
This looks nice but there are a ton of issues around pyobjc. I've never been able to successfully install it on 10.9.
- pudquick 12y ago
  This is because you don't need to. Especially for this particular script. If you're using the natively included python, pyobjc comes free. As soon as I saw that mentioned in the documentation, I knew this project was going to have issues.
  - johndoze 12y ago
    I'm familiar with python bridges at all - how would someone transition something with pyobjc as a dependency to the native alternative you mention?
    - pudquick 12y ago
      You misunderstand. OS X has shipped with python for quite some time. In addition, OS X python started including a build of pyobjc since OS X 10.5 You literally can fire up python from the terminal on OS X 10.5 and later and type: import Foundation and it will "just work". No additional installs required.
spiralganglion 12y ago
This looks like an interesting project. But, there's one bit of information, which is missing from the description and this discussion, that would help a lot with my understanding. Is this a tool for a user who wants to learn more about their own machine, or a non-user who wants to know how a given machine has been used? More information about the use cases of such a tool would be most helpful.
- thinkling 12y ago
  Ditto, in particular would be good to know whether this is only for post--intrusion forensics or also useful ahead of time.
entelechy0 12y ago
I'm getting an error after running it: [INFO] Users' LoginItems [INFO] 's LoginItems [INFO] /Users//Library/Preferences/com.apple.loginitems.plist [INFO] Cannot parse /Users//Library/Preferences/com.apple.loginitems.plist (Binary or JSON plist may FAIL) Traceback (most recent call last): File "osxauditor.py", line 1702, in <module> Main() File "osxauditor.py", line 1651, in Main ParseStartup() File "osxauditor.py", line 550, in ParseStartup if "SessionItems" in LoginItemsPlist: TypeError: argument of type 'bool' is not iterable
- arm 12y ago
  I’m guessing that’s probably happening because of the double slash after /Users.
  - entelechy0 12y ago
    nono, that was originally asterisks but apparently hacker news replaces those with an empty string. that is weird, but unrelated to my problem
w0rd-driven 12y ago
I'd say this isn't quite ready for prime time. The errors I received were annoying and you're not going to get an end user or most IT admins to parse the python just to find the errors. I've only had 3 major errors but they were significant: 1) I'm on Yosemite so GetAuditedSystemVersion() looks for a PatchVersion variable that simply isn't there. The header reads 10.10 not 10.10.patch as expected. 2) The Safari parsing snafu listed in my previous comment. Opening Safari isn't enough, you have to use the browser quite a bit. The same could likely be said for all browser tests and it would be a good idea to outline precisely what this needs to be. Hint: A new system or install of Yosemite for instance will produce the errors I saw. 3) There's a parsing bug in ParseMailAppAccount() and I just commented out the call completely. Any number of these could just be Yosemite related but I don't think so. All of the bugs I ran into are variations on index out of bounds due to some hardcoded assumption that mostly works, except in this instance apparently. I'm not the only one with these nagging bugs based on the issues list but mine do seem very specific to Yosemite or how I do(n't) use my system.
makeset 12y ago
Application path scanning follows symlinks recursively. One upwards link, and it never terminates.

348 points by evandrix 12y ago

Great idea. Love to run it, but... crash and burn: ~/Library/Safari/LastSession.plist Traceback (most recent call last): File "osxauditor.py", line 1702, in <module> Main() File "osxauditor.py", line 1663, in Main ParseBrowsers() File "osxauditor.py", line 808, in ParseBrowsers ParseSafari() File "osxauditor.py", line 745, in ParseSafari ParseSafariProfile(User, UserSafariProfilePath) File "osxauditor.py", line 717, in ParseSafariProfile LastSession = LastSessionPlist["SessionWindows"][0]["TabStates"][0] File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC/objc/_convenience.py", line 451, in __getitem__objectAtIndex_ return container_unwrap(self.objectAtIndex_(idx), RuntimeError) IndexError: NSRangeException - -[__NSCFArray objectAtIndex:]: index (0) beyond bounds (0)

bsmartt 12y ago
I can't even get pyobjc running on 10.9..
- rmetzler 12y ago
  I also have this problem. I'm not sure, but it might be caused by using brew python instead of the systems original python
yohanatan 12y ago
Looks like you have an empty session list. Have you ever used Safari? Quick workaround might be to pop open Safari once (and browse a little) and then try again.
- mwfunk 12y ago
  Why on earth would anyone downvote this comment? The lack of a Safari user profile is the most obvious potential issue here, based on the backtrace.
  - w0rd-driven 12y ago
    That's the precise problem I had. Opening Safari got me one step closer and then another failure. I wanted to run all options but got tired of all these errors so I commented out ParseSafari, ParseFirefox, and ParseChrome calls inside the ParseBrowsers function. I could've left Firefox or Chrome but not wanting to shut Chrome down or not using Firefox much, I didn't want to go through this again for each browser.
mdaniel 12y ago
And that, folks, is why we don't do chained expressions. Variables increase readability and also offer one the opportunity to check a condition that should be true. This, KeyError, NoneType exceptions and its Java friend NullPointerException drive me batty. They are so simple to defend against, if the author just took the time.
- nsxwolf 12y ago
  I love Optionals for this.
- chc 12y ago
  Chaining itself isn't really the problem here IMO. Creating a variable for each level of the nested data structure and manually checking for the validity of each subsequent one would really, really not produce more readable code. It would turn a "get this value out of a data structure" expression into about 15 lines of code that each need to be individually read and understood. The problem is that Cocoa doesn't provide a great way to say "Access this stuff in this data structure unless it isn't there." An option type is really the best way to kill this breed of problem, and thankfully Apple is including one in Swift, though it's still somewhat green.
  - photojosh 12y ago
    That's a Obj-C problem, because it's being fixed with what Swift does with it's '?' on variables. :)
    - chc 12y ago
      Not exactly. This program would have the same problem in Swift, because rather than returning optionals, Swift arrays have the same explode-on-invalid-accesses behavior as NSArray.
- arianvanp 12y ago
  Maybe monads make this kind of chaining safe.
- 12y ago

mcescalante 12y ago

Very interested to try this out on my own machines to see the results. I know somebody who does computer security at a University and the staff there has been frustrated with the lack of available forensics tools for OSX, so this may nicely fill an empty niche for some industry people as well.

c0wl 12y ago

OSXAuditor is pretty dope and we used it a bunch at Yelp. Over time, we created what we think is an inspired next version - https://github.com/Yelp/osxcollector https://github.com/Yelp/osxcollector

deleted 12y ago
[deleted]
traveller_57681 12y ago
The passive scan approach has the single weakness that it appears to be effective, but mostly is security theatre. Take for example, when we travelling into Australia, and they asked if we had any criminal convictions - I didn't know that a conviction was still an entry requirement for the island!
crag 12y ago
I run OSXCollector and I get missing modules error. I'm not a python guy, but looking at the code, it requires: import Foundation, import calendar, import os, import sys, import shutil I assuming os, sys, and shutil are part of the system (are they?). What about the other two? Also, which version of Python do I run this under?
- ddp 12y ago
  ...not a python user either. The README mentions 'pip' but I don't seem to have one of those either.
  - texuf 12y ago
    Here's a "non-magical introduction to pip" http://www.dabapps.com/blog/introduction-to-pip-and-virtualenv-python/ http://www.dabapps.com/blog/introduction-to-pip-and-virtuale... Pip is an excellent package management system.
  - deleted 12y ago
    [deleted]
- 12y ago

korzun 12y ago

Cool idea. But using raw md5 hashes to verify against a blacklist is kind of useless. Especially now. You should be using smarter file signatures: http://hooked-on-mnemonics.blogspot.com/2011/01/intro-to-creating-anti-virus-signatures.html http://hooked-on-mnemonics.blogspot.com/2011/01/intro-to-cre...

jnazario 12y ago
so, if i understand it correctly you're saying create AV signatures (the link you point to is how to create ClamAV signatures). to do so in a worthwhile, comprehensive way would required the maintainer to basically become an AV company. possible, but requires a lot of effort to scale. by calling out to various services, the creator of this tool is essentially outsourcing maintaining a file reputation blacklist. MD5s are, despite their limitations, the lingua franca of the security industry. nearly everyone who provides a file reputation query service supports them (as opposed to SHA1s or other hashes like ssdeep). so, i think i get what you're saying, but i don't think it's a relevant suggestion here.
- korzun 12y ago
  I'm pointing out that this tool is essentially useless from security perspective. Most of the payloads will have different hash results due to extensive use of packers. If you are going to get hit with variant #11929 before the online databases obtains a hash of it, this tool is not going to pick it up but it will tell you that you are secure.
- hueving 12y ago
  >so, i think i get what you're saying, but i don't think it's a relevant suggestion here. It's pretty relevant. Without sending the whole file to the third party, the file reputation service isn't 'outsourced' as you suggest. Sending the MD5 will not do any good if the program makes non-deterministic modifications to its binary.

SmileyKeith 12y ago

This looks nice but there are a ton of issues around pyobjc. I've never been able to successfully install it on 10.9.

pudquick 12y ago
This is because you don't need to. Especially for this particular script. If you're using the natively included python, pyobjc comes free. As soon as I saw that mentioned in the documentation, I knew this project was going to have issues.
- johndoze 12y ago
  I'm familiar with python bridges at all - how would someone transition something with pyobjc as a dependency to the native alternative you mention?
  - pudquick 12y ago
    You misunderstand. OS X has shipped with python for quite some time. In addition, OS X python started including a build of pyobjc since OS X 10.5 You literally can fire up python from the terminal on OS X 10.5 and later and type: import Foundation and it will "just work". No additional installs required.

spiralganglion 12y ago

This looks like an interesting project. But, there's one bit of information, which is missing from the description and this discussion, that would help a lot with my understanding. Is this a tool for a user who wants to learn more about their own machine, or a non-user who wants to know how a given machine has been used? More information about the use cases of such a tool would be most helpful.

thinkling 12y ago
Ditto, in particular would be good to know whether this is only for post--intrusion forensics or also useful ahead of time.

entelechy0 12y ago

I'm getting an error after running it: [INFO] Users' LoginItems [INFO] 's LoginItems [INFO] /Users//Library/Preferences/com.apple.loginitems.plist [INFO] Cannot parse /Users//Library/Preferences/com.apple.loginitems.plist (Binary or JSON plist may FAIL) Traceback (most recent call last): File "osxauditor.py", line 1702, in <module> Main() File "osxauditor.py", line 1651, in Main ParseStartup() File "osxauditor.py", line 550, in ParseStartup if "SessionItems" in LoginItemsPlist: TypeError: argument of type 'bool' is not iterable

arm 12y ago
I’m guessing that’s probably happening because of the double slash after /Users.
- entelechy0 12y ago
  nono, that was originally asterisks but apparently hacker news replaces those with an empty string. that is weird, but unrelated to my problem

w0rd-driven 12y ago

I'd say this isn't quite ready for prime time. The errors I received were annoying and you're not going to get an end user or most IT admins to parse the python just to find the errors. I've only had 3 major errors but they were significant: 1) I'm on Yosemite so GetAuditedSystemVersion() looks for a PatchVersion variable that simply isn't there. The header reads 10.10 not 10.10.patch as expected. 2) The Safari parsing snafu listed in my previous comment. Opening Safari isn't enough, you have to use the browser quite a bit. The same could likely be said for all browser tests and it would be a good idea to outline precisely what this needs to be. Hint: A new system or install of Yosemite for instance will produce the errors I saw. 3) There's a parsing bug in ParseMailAppAccount() and I just commented out the call completely. Any number of these could just be Yosemite related but I don't think so. All of the bugs I ran into are variations on index out of bounds due to some hardcoded assumption that mostly works, except in this instance apparently. I'm not the only one with these nagging bugs based on the issues list but mine do seem very specific to Yosemite or how I do(n't) use my system.

makeset 12y ago

Application path scanning follows symlinks recursively. One upwards link, and it never terminates.