6 ms·

How does Google build such forsenics? Do they have spyware monitoring all their company laptops?

by tajen 10y ago

vmarsy 10y ago
I would assume that the servers that host big trade secrets would log everything happening. There wouldn't be a need to log the laptop itself.
clay_to_n 10y ago
What? Most version control systems have authentication systems, I'm pretty sure Google has good reason to keep audit logs of employee's access to schematics.
aidos 10y ago
Yeah, they definitely have some really specific accusations about what went on: >>> To gain access to Waymo’s design server, Mr. Levandowski searched for and installed specialized software onto his company-issued laptop. Once inside, he downloaded 9.7 GB of Waymo’s highly confidential files and trade secrets, including blueprints, design files and testing documentation. Then he connected an external drive to the laptop. Mr. Levandowski then wiped and reformatted the laptop in an attempt to erase forensic fingerprints. The bit about connecting the external drive is interesting but I guess there's probably a ghost of that action somewhere on a drive (assuming you could more or less restore the drive before being wiped).
- kbenson 10y ago
  It could well be that they have comprehensive debugging enabled for all their employees on work supplied machines. I know they beta test through employees, so that would make sense, to some degree. If the debug log is remotely synced, as an added benefit they get to review actions after the fact, which might be what happened here.
- mahyarm 10y ago
  My guess is he still had a network connection up when he did the copy. These kind of things like JAMF, osquery and so on upload what you do on your computer on a fairly frequent basis. After a certain company size all of them do something like this, and most of them do not spell out the amount of spying they do on their employee's work devices and office space. Or he didn't do a secure wipe when he reformatted the drive and google inspected the computer when he returned it. OR google modifies their laptops and has separate chips logging this stuff, which would be fairly impressive! Or he didn't actually do any of this alledged stuff and it's all innocent. We will find out in the court case either way!
  - jsprogrammer 10y ago
    Could Google's records be fabricated? If there are separate logging chips, one would get access to all schematics, fabrication facilities, etc. in order to even consider any Google claims based on data collected through such methods.
  - x0x0 10y ago
    I would imagine there are IDS systems running, given Google's state level adversaries. Accessing large amounts of files you haven't previously accessed and shortly thereafter attaching an external drive should trip a competent IDS.
  - jlgaddis 10y ago
    Or, you know, syslog.
  - brazzledazzle 10y ago
    Unless it's changed since the last time I played with it osquery doesn't upload anything on its own–it's just a local tool/agent that you can use to gather data. Carbon black is a good example of something like that. JAMF I believe gathers application usage data but nothing as in depth as what's being discussed. It's also comically handicapped. It somehow manages to do a poor job of everything it tries to do so "the world's largest online Mac administrator community" is forum post after forum post of half understood franken-scripts. I used to think it was milquetoast but after sitting through their sales reps crapping all over open source software (despite extensive use of OSS libs in their products) and seeing it fail to do the most basic stuff out of the box my opinion is that it's over priced crap.
- klodolph 10y ago
  Company-issued laptop probably tracks what software is installed anyway to mitigate malware. Services are probably designed not to accept connections from untrusted devices. Hardware 2FA makes it difficult to impersonate someone without their knowledge. I can see the temptation—I've always missed having access to IP after leaving a company.
- rasz_pl 10y ago
  You cant restore SSDs. I would assume Googlers are rolling SATA SSDs, if not M.2 PCIE ones. What Google could be doing is remote logging on the laptops - logs uploaded to ze cloud every time you connect to the mothership. Plugging in USB drive leaves trace with USB ID, volume information etc. Windows also logs this and more http://www.forensicswiki.org/wiki/USB_History_Viewing http://www.forensicswiki.org/wiki/USB_History_Viewing Protip: to exfiltrate data with minimal trace your best bet is taking out the drive and reading it in another computer (using write blocker for best effect), this can still be traced if someone is logging SMART written/read data (I am, but Im paranoid), not all HDD/SSD vendors provide this info. Second best is booting from USB drive so the original OS never sees the plug/unplug event in the first place, I have no idea about current state of UEFI/AMT logging going on tho. Disclaimer: I used to do forensics.
  - aidos 10y ago
    Really interesting! It's sobering to think about the ways that even in a system not set up for logging you can trace back through these actions. I was asked to figure out what had happened on a system where some data had changed and 2 parties were blaming each other. After about a hour digging around I managed to piece together a picture of how Person X had got up on a Monday morning, discovered (on their mobile, home wifi) that they had made a mistake on Friday, then logged in on their desktop to fix it from home (first time they logged in at home), then went to work and blamed someone else. What was remarkable was how many different sources there were to pick up bits and pieces from. In isolation there wasn't much to go on, but once you start the connecting the parts, it's really incriminating.
startupdiscuss 10y ago
This was the noteworthy part for me as well. They had to know that he: 1. modified the software on his laptop 2. logged into an area he should not have had access to (this is probably standard) 3. attached an external drive (possible, but standard?) 4. and they got all this info after he deleted the drive, which means they either went in and found remaining data on the drive or else they captured the info in real time. I suppose if the drive is clean now, and they know he downloaded data, they can infer that he wiped it. I suppose that if they know he accessed it, and there was software on his computer preventing him from doing so, they can infer that he downloaded something to overcome it. But knowing that he connected to an external drive implies active monitoring. That's the part I am most curious about.
- foota 10y ago
  Maybe video footage?
- Jabbles 10y ago
  Just to be clear "his" laptop is "the laptop issued to him by Google". I'd think that standard antivirus software detects and alerts external drives being attached.
- ocdtrekkie 10y ago
  This is all information a rudimentary desktop auditing tool can gather and store on a server. Most collect both hardware (which would include connected devices) and software inventory. Anyone SHOULD be auditing company PCs on a relatively regular basis. It wouldn't surprise me if Google was auditing much more frequently than the average and could catch something like this in the act.
  - 086421357909764 10y ago
    Yup, lots of firms using HIDS that gather all sorts of system data. OSSEC is what I've seen rather frequently for this.
- corvus_sapiens 10y ago
  1) makes sense if Levandowski did it over the company internet connection. There could be a record of his unusual requests (software download, software update). > they can infer that he wiped it For 4), Levandowski reformatted the hard drive before returning it, so there's no inference there.
deejaybog 10y ago
Even if they do, it is very unlikely they were built with provable nonrepudiation requirements. From personal experience designing the security of a PKI CA that passed gov security certifications, the audit subsystem is the most challenging part to do right. Could probably consult for the defense in tearing down the evidence :)
- inlined 10y ago
  I'll accept that it's hard, but why do you think Google didn't do it right?
  - deejaybog 10y ago
    It would require a prohibitive amount of engineering resources to be done right, i.e. a chain of guarantees that from creation time to the moment they are inspected it can be proven that the logs cannot be tampered with by nonauthorized users. There are other requirements e.g. separation of roles that are expected on audit subsystems. I am positive it would not pass an adversary expert analysis.
    - sangnoir 10y ago
      Google's threat models include nation-state adversaries: I suspect the effort that seems "prohibitive" to you was seen as necessary after the infamous smiley on that PRISM slide. Security is an existential threat: if user's don't trust Google, they will fail. Google also has an internal PKI CA - I think they meet and exceed that security baseline for rigor.
      - deejaybog 10y ago
        Yes, for purposes of issuing certificates I'm sure they are OK wrt auditing (I was just establishing my "credentials" with the CA comment). The threat models targeting anti-Google malicious actions obviously worked since they have traces of the Otto guy's activities. What I am asserting is that these forensics logs they use as evidence can be attacked in court as not being sufficiently protected from tampering by an internal Google party interested in fabricating evidence.
dsacco 10y ago
Typical garden-variety endpoint security/IT management solutions such as Crowdstrike or Casper with a kernel module are capable of all of that.
deleted 10y ago
[deleted]
jonstewart 10y ago
Yes! It's open source: https://github.com/google/grr https://github.com/google/grr Google probably has the largest team of internal forensics software developers.

How does Google build such forsenics? Do they have spyware monitoring all their company laptops?

by tajen 10y ago

How does Google build such forsenics? Do they have spyware monitoring all their company laptops?

I would assume that the servers that host big trade secrets would log everything happening. There wouldn't be a need to log the laptop itself.

clay_to_n 10y ago

What? Most version control systems have authentication systems, I'm pretty sure Google has good reason to keep audit logs of employee's access to schematics.

aidos 10y ago

Yeah, they definitely have some really specific accusations about what went on: >>> To gain access to Waymo’s design server, Mr. Levandowski searched for and installed specialized software onto his company-issued laptop. Once inside, he downloaded 9.7 GB of Waymo’s highly confidential files and trade secrets, including blueprints, design files and testing documentation. Then he connected an external drive to the laptop. Mr. Levandowski then wiped and reformatted the laptop in an attempt to erase forensic fingerprints. The bit about connecting the external drive is interesting but I guess there's probably a ghost of that action somewhere on a drive (assuming you could more or less restore the drive before being wiped).

kbenson 10y ago
It could well be that they have comprehensive debugging enabled for all their employees on work supplied machines. I know they beta test through employees, so that would make sense, to some degree. If the debug log is remotely synced, as an added benefit they get to review actions after the fact, which might be what happened here.
mahyarm 10y ago
My guess is he still had a network connection up when he did the copy. These kind of things like JAMF, osquery and so on upload what you do on your computer on a fairly frequent basis. After a certain company size all of them do something like this, and most of them do not spell out the amount of spying they do on their employee's work devices and office space. Or he didn't do a secure wipe when he reformatted the drive and google inspected the computer when he returned it. OR google modifies their laptops and has separate chips logging this stuff, which would be fairly impressive! Or he didn't actually do any of this alledged stuff and it's all innocent. We will find out in the court case either way!
- jsprogrammer 10y ago
  Could Google's records be fabricated? If there are separate logging chips, one would get access to all schematics, fabrication facilities, etc. in order to even consider any Google claims based on data collected through such methods.
- x0x0 10y ago
  I would imagine there are IDS systems running, given Google's state level adversaries. Accessing large amounts of files you haven't previously accessed and shortly thereafter attaching an external drive should trip a competent IDS.
- jlgaddis 10y ago
  Or, you know, syslog.
- brazzledazzle 10y ago
  Unless it's changed since the last time I played with it osquery doesn't upload anything on its own–it's just a local tool/agent that you can use to gather data. Carbon black is a good example of something like that. JAMF I believe gathers application usage data but nothing as in depth as what's being discussed. It's also comically handicapped. It somehow manages to do a poor job of everything it tries to do so "the world's largest online Mac administrator community" is forum post after forum post of half understood franken-scripts. I used to think it was milquetoast but after sitting through their sales reps crapping all over open source software (despite extensive use of OSS libs in their products) and seeing it fail to do the most basic stuff out of the box my opinion is that it's over priced crap.
klodolph 10y ago
Company-issued laptop probably tracks what software is installed anyway to mitigate malware. Services are probably designed not to accept connections from untrusted devices. Hardware 2FA makes it difficult to impersonate someone without their knowledge. I can see the temptation—I've always missed having access to IP after leaving a company.
rasz_pl 10y ago
You cant restore SSDs. I would assume Googlers are rolling SATA SSDs, if not M.2 PCIE ones. What Google could be doing is remote logging on the laptops - logs uploaded to ze cloud every time you connect to the mothership. Plugging in USB drive leaves trace with USB ID, volume information etc. Windows also logs this and more http://www.forensicswiki.org/wiki/USB_History_Viewing http://www.forensicswiki.org/wiki/USB_History_Viewing Protip: to exfiltrate data with minimal trace your best bet is taking out the drive and reading it in another computer (using write blocker for best effect), this can still be traced if someone is logging SMART written/read data (I am, but Im paranoid), not all HDD/SSD vendors provide this info. Second best is booting from USB drive so the original OS never sees the plug/unplug event in the first place, I have no idea about current state of UEFI/AMT logging going on tho. Disclaimer: I used to do forensics.
- aidos 10y ago
  Really interesting! It's sobering to think about the ways that even in a system not set up for logging you can trace back through these actions. I was asked to figure out what had happened on a system where some data had changed and 2 parties were blaming each other. After about a hour digging around I managed to piece together a picture of how Person X had got up on a Monday morning, discovered (on their mobile, home wifi) that they had made a mistake on Friday, then logged in on their desktop to fix it from home (first time they logged in at home), then went to work and blamed someone else. What was remarkable was how many different sources there were to pick up bits and pieces from. In isolation there wasn't much to go on, but once you start the connecting the parts, it's really incriminating.

startupdiscuss 10y ago

This was the noteworthy part for me as well. They had to know that he: 1. modified the software on his laptop 2. logged into an area he should not have had access to (this is probably standard) 3. attached an external drive (possible, but standard?) 4. and they got all this info after he deleted the drive, which means they either went in and found remaining data on the drive or else they captured the info in real time. I suppose if the drive is clean now, and they know he downloaded data, they can infer that he wiped it. I suppose that if they know he accessed it, and there was software on his computer preventing him from doing so, they can infer that he downloaded something to overcome it. But knowing that he connected to an external drive implies active monitoring. That's the part I am most curious about.

foota 10y ago
Maybe video footage?
Jabbles 10y ago
Just to be clear "his" laptop is "the laptop issued to him by Google". I'd think that standard antivirus software detects and alerts external drives being attached.
ocdtrekkie 10y ago
This is all information a rudimentary desktop auditing tool can gather and store on a server. Most collect both hardware (which would include connected devices) and software inventory. Anyone SHOULD be auditing company PCs on a relatively regular basis. It wouldn't surprise me if Google was auditing much more frequently than the average and could catch something like this in the act.
- 086421357909764 10y ago
  Yup, lots of firms using HIDS that gather all sorts of system data. OSSEC is what I've seen rather frequently for this.
corvus_sapiens 10y ago
1) makes sense if Levandowski did it over the company internet connection. There could be a record of his unusual requests (software download, software update). > they can infer that he wiped it For 4), Levandowski reformatted the hard drive before returning it, so there's no inference there.

deejaybog 10y ago

Even if they do, it is very unlikely they were built with provable nonrepudiation requirements. From personal experience designing the security of a PKI CA that passed gov security certifications, the audit subsystem is the most challenging part to do right. Could probably consult for the defense in tearing down the evidence :)

inlined 10y ago
I'll accept that it's hard, but why do you think Google didn't do it right?
- deejaybog 10y ago
  It would require a prohibitive amount of engineering resources to be done right, i.e. a chain of guarantees that from creation time to the moment they are inspected it can be proven that the logs cannot be tampered with by nonauthorized users. There are other requirements e.g. separation of roles that are expected on audit subsystems. I am positive it would not pass an adversary expert analysis.
  - sangnoir 10y ago
    Google's threat models include nation-state adversaries: I suspect the effort that seems "prohibitive" to you was seen as necessary after the infamous smiley on that PRISM slide. Security is an existential threat: if user's don't trust Google, they will fail. Google also has an internal PKI CA - I think they meet and exceed that security baseline for rigor.
    - deejaybog 10y ago
      Yes, for purposes of issuing certificates I'm sure they are OK wrt auditing (I was just establishing my "credentials" with the CA comment). The threat models targeting anti-Google malicious actions obviously worked since they have traces of the Otto guy's activities. What I am asserting is that these forensics logs they use as evidence can be attacked in court as not being sufficiently protected from tampering by an internal Google party interested in fabricating evidence.

dsacco 10y ago

Typical garden-variety endpoint security/IT management solutions such as Crowdstrike or Casper with a kernel module are capable of all of that.

deleted 10y ago

[deleted]

jonstewart 10y ago

Yes! It's open source: https://github.com/google/grr https://github.com/google/grr Google probably has the largest team of internal forensics software developers.