6 ms·

With a vanilla git configuration, it's as easy as: $ vi centrifuge_rpm_controller.cc $ git commit --author "Bob Goodguy <bob@tesla.com>" -a -m "Totally har

by tlb 8y ago

With a vanilla git configuration, it's as easy as: $ vi centrifuge_rpm_controller.cc $ git commit --author "Bob Goodguy <bob@tesla.com>" -a -m "Totally harmless changes" $ git push In a high-stakes environment, it's probably worth signing commits. https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work

prepend 8y ago
My company only allows authenticated users to commit changes. Even pull requests on our public repos get scanned and reviewed before human review. And pull requests are infrequent enough that unknown users usually involve some interaction. It seems like if an organization is allowing randos to push changes, they would also allow randos with valid pgp to push changes as well. So the investigation would change to “Who the fudge is B8956DBFEE7C105C?” instead of “Who the fudge is Bob Goodguy?”
- zxcmx 8y ago
  What you might not realise is that the author of a commit is not related to the (authenticated) user who pushes. You couldn’t merge changes from multiple people otherwise. Signed commits do fix this although you have to be careful about your threat model. Lots of software (e.g. gitlab) will prevent alice from pretending to be bob but will not protect you against server compromise (because public key directory and signing enforcement are done on the server). Working at an org which does signed commits, it doesn’t buy us quite as much as we would like. Also, if you enforce signing you can’t mash a server-side “merge” button anymore because that commit would need to be signed... This complicates the pr/mr workflow for teams and increases resistance from those who don’t see the value in signing.
  - jaggederest 8y ago
    Github signs merge commits FWIW. They're pretty good about it.
  - solatic 8y ago
    > but will not protect you against server compromise (because public key directory and signing enforcement are done on the server) Which is why it's important for the security team to directly manage and be responsible for the code repository server. Usually the server also stores all the git repositories in some filesystem directory, meaning that server compromise would allow an attacker to remove all the company's code, regardless of permissions set for those repositories, and potentially leak it.
  - prepend 8y ago
    What I mean is that making fake author accounts and then merging with an authenticated account shouldn’t help a malicious actor. Any competent project will be interested in the accounts contributing. Maybe my org is just simple but if “John Doe” started submitting pull requests and prepend merged them, it’s me on the hook and there are people interested in who John Doe is. And in my org, even John Doe needs a network credential so even their local git instance (which I’ve never seen anyone use) would need a credential.
flyingswift 8y ago
Are we really saying that Tesla, a company building highly sensitive software, is not employing basic branch policies? How is it that these changes could have made it through a code review process and deployed to 'production'? What I take away from this is that one malicious actor was able to single handedly deploy malicious code and that there were no processes in place to stop this. That is a far more worrying issue
- deleted 8y ago
  [deleted]
- giarc 8y ago
  Not sure if it changes your point at all, but the changes were to the manufacturing operating system, not the car OS. I'm not familiar with manufacturing but I imagine there are a ton of different systems and perhaps not all are as secure as the other. I recall reading they needed some paint improvements, so perhaps they changed something there. I don't imagine they would have software controlling paint totally locked down.
  - gregpilling 8y ago
    Most manufacturing machines run G-code. Simply changing one of the G-code values (text editing) would cause the machine to make parts incorrectly, and this would screw up things down the line. https://en.wikipedia.org/wiki/G-code https://en.wikipedia.org/wiki/G-code on my Haas mill, the code is directly editable by the operator. On my robots, the code is 'taught' by moving the arm and telling the machine that the new location is the updated. It is routine to have to adjust minor things in the robots, the operator would have to have access to do their job. I had a welder put a washer under a sensor on my robot one time, making every weld 1 mm in the wrong spot. We couldn't figure it out and ended up reprogramming the entire setup, 8 hours, to fix it. A week later I noticed the washer, put there by a guy who I fired for bad attitude a week before. Once I had found the washer, I had the choice of going back to the old code, or leaving it with the new code. I left it with the new code. I didn't have another 8 hours to waste to fix the fix. Even tiny companies like mine can have sabotage. Why not Tesla?
- 8y ago

by tlb 8y ago

prepend 8y ago

My company only allows authenticated users to commit changes. Even pull requests on our public repos get scanned and reviewed before human review. And pull requests are infrequent enough that unknown users usually involve some interaction. It seems like if an organization is allowing randos to push changes, they would also allow randos with valid pgp to push changes as well. So the investigation would change to “Who the fudge is B8956DBFEE7C105C?” instead of “Who the fudge is Bob Goodguy?”

zxcmx 8y ago
What you might not realise is that the author of a commit is not related to the (authenticated) user who pushes. You couldn’t merge changes from multiple people otherwise. Signed commits do fix this although you have to be careful about your threat model. Lots of software (e.g. gitlab) will prevent alice from pretending to be bob but will not protect you against server compromise (because public key directory and signing enforcement are done on the server). Working at an org which does signed commits, it doesn’t buy us quite as much as we would like. Also, if you enforce signing you can’t mash a server-side “merge” button anymore because that commit would need to be signed... This complicates the pr/mr workflow for teams and increases resistance from those who don’t see the value in signing.
- jaggederest 8y ago
  Github signs merge commits FWIW. They're pretty good about it.
- solatic 8y ago
  > but will not protect you against server compromise (because public key directory and signing enforcement are done on the server) Which is why it's important for the security team to directly manage and be responsible for the code repository server. Usually the server also stores all the git repositories in some filesystem directory, meaning that server compromise would allow an attacker to remove all the company's code, regardless of permissions set for those repositories, and potentially leak it.
- prepend 8y ago
  What I mean is that making fake author accounts and then merging with an authenticated account shouldn’t help a malicious actor. Any competent project will be interested in the accounts contributing. Maybe my org is just simple but if “John Doe” started submitting pull requests and prepend merged them, it’s me on the hook and there are people interested in who John Doe is. And in my org, even John Doe needs a network credential so even their local git instance (which I’ve never seen anyone use) would need a credential.

flyingswift 8y ago

Are we really saying that Tesla, a company building highly sensitive software, is not employing basic branch policies? How is it that these changes could have made it through a code review process and deployed to 'production'? What I take away from this is that one malicious actor was able to single handedly deploy malicious code and that there were no processes in place to stop this. That is a far more worrying issue

deleted 8y ago
[deleted]
giarc 8y ago
Not sure if it changes your point at all, but the changes were to the manufacturing operating system, not the car OS. I'm not familiar with manufacturing but I imagine there are a ton of different systems and perhaps not all are as secure as the other. I recall reading they needed some paint improvements, so perhaps they changed something there. I don't imagine they would have software controlling paint totally locked down.
- gregpilling 8y ago
  Most manufacturing machines run G-code. Simply changing one of the G-code values (text editing) would cause the machine to make parts incorrectly, and this would screw up things down the line. https://en.wikipedia.org/wiki/G-code https://en.wikipedia.org/wiki/G-code on my Haas mill, the code is directly editable by the operator. On my robots, the code is 'taught' by moving the arm and telling the machine that the new location is the updated. It is routine to have to adjust minor things in the robots, the operator would have to have access to do their job. I had a welder put a washer under a sensor on my robot one time, making every weld 1 mm in the wrong spot. We couldn't figure it out and ended up reprogramming the entire setup, 8 hours, to fix it. A week later I noticed the washer, put there by a guy who I fired for bad attitude a week before. Once I had found the washer, I had the choice of going back to the old code, or leaving it with the new code. I left it with the new code. I didn't have another 8 hours to waste to fix the fix. Even tiny companies like mine can have sabotage. Why not Tesla?
8y ago