12 ms·

I made this same comment on the other discussion. I find it concerning that one person was able to push malicious code to 'production'. To me, this suggests th

I made this same comment on the other discussion. I find it concerning that one person was able to push malicious code to 'production'. To me, this suggests that Tesla, a company building highly sensitive software, does not employ basic branch policies. How is is it that these changes could have made it through a code review process and get deployed? If a company like Microsoft or Google announced that a disgruntled employee was able to push some code that, say, stole user information, I think the general reaction would be "why was this allowed to happen?" I'm not sure why Tesla gets a pass in this regard

SuoDuanDao 8y ago
I would say it's like the difference between China and America having a major environmental accident - we treat developing economies and businesses differently from established ones.
- asfasgasg 8y ago
  Tesla was founded before Facebook, and I'd argue that Facebook would fall into the same category as Google or Microsoft on this question.
cottsak 8y ago
This is a very naive comment. There will always be a small handful of engineers that can push the button to move code into PROD or even change code in PROD live. Ideally, with mature controls, the people in this list is short. But to jump to the conclusion that Tesla doesn't use good practises is very short sighted. Who's to say that external parties didn't target this person specifically because of their role/influence.
- sanderjd 8y ago
  > There will always be a small handful of engineers that can push the button to move code into PROD or even change code in PROD live. There is really no reason for this to be the case. Certainly all code that actually runs on the car can be required to go through review and be verifiably built, even if server code standards are more lax.
  - nickparker 8y ago
    The code in question is for their manufacturing systems, not the cars. Not that that's necessarily better... Manufacturing equipment's at about the same danger tier as cars.
    - flyingswift 8y ago
      True. It's possible that the manufacturing software could be modified in some manner to introduce some fundamental flaw in the final product though. For that reason, I would say the code should also be held to a higher set of standards
  - tehwebguy 8y ago
    > This included making direct code changes to the Tesla Manufacturing Operating System under false usernames If they found a way to use more than one username they may very well have run it through the review process and approved it themselves
  - SolarNet 8y ago
    I think you misunderstand. Even with code review policies, there is still a short list of people who can push to production without going through code review. Not from a policy standpoint but from a security and access perspective.
    - falcolas 8y ago
      Which indicates a different problem if the sabatuer was on that short list. And in this day and age of cryptographically signed commits, the number of people who could do this should go down even further. Given this, I think it’s much more likely there were few or bad controls, than a person in an incredibly privileged position working in the margins.
      - SolarNet 8y ago
        Someone builds the code, it doesn't have to be committed.
- asfasgasg 8y ago
  It seems far more likely that the controls were lax than that the saboteur was one of those privileged few. It's not a naive comment.
- EGreg 8y ago
  Why is it naive to expect there not to be a single point of failure like that? Code reviews are a thing.
  - noobermin 8y ago
    What if they pushed it without asking anyone?
    - davidgay 8y ago
      Then the system that allowed them to do so is badly designed.
      - noobermin 8y ago
        I guess my thinking is if the disgruntled employee was so upset over not getting a promotion to cause so much damage they must have been in a high enough position already to warrant such a disposition, and "high enough" might mean high enough to have push access themselves. Edit: see closeparens comment above. Complicated systems can always be subverted when trust is broken.
      - chillee 8y ago
        At Facebook, you could alter code even after somebody had given the OK for code review. I know some people specifically kept some small commits open after being approved, just so they could quickly make changes without needing approval if they ever needed to.
        sneak 8y ago
        “Dear respected members of Congress, ...”
        skj 8y ago
        At Google you can mark something as TBR (to be reviewed), as long as you're one of the designated owners for that code.
- xiphias 8y ago
  At Google binaries, packages and code are digitally signed by the logged in user and computer
- alxlaz 8y ago
  Obviously, not all details are available, but the wording in the email suggests that the parent comment is anything but naive: > This included making direct code changes to the Tesla Manufacturing Operating System under false usernames and exporting large amounts of highly sensitive Tesla data to unknown third parties. This sounds like something out of the 1990s, that dark and romantic era of version control when we thought CVS was pretty cool actually and we didn't know what key-based authentication and 2FA were. There are volunteer-ran projects that don't have this problem. Edit: to be clear, I presume no one is debating the fact that someone with high enough credentials can push code to production. The questions that the email raises are: 1. Why can anyone, regardless of credentials, push mission-critical code without review (or, alternatively, if the changes did go through review, why did the review process not catch multiple malicious changes?) 2. Why can someone compromise several high-level credentials without anyone figuring it out (the changes were made, apparently, under "false usernames")?
  - alanfranzoni 8y ago
    > 1. Why can anyone, regardless of credentials, push mission-critical code without review (or, alternatively, if the changes did go through review, why did the review process not catch multiple malicious changes?) Why do you suppose the unauthorized party was following the company's development practices? Maybe it was from the sysadmin side, somebody who worked on the toolchain used for reviewing and pushing things to production. So he was able to sidestep the normal review process. This can happen, what is important is that such things are discovered.
    - alxlaz 8y ago
      > So he was able to sidestep the normal review process. He should not have been able to sidestep the normal review process. That's the problem in the first place. Even if you're from the sysadmin side. It should not be possible to do it. You may think that looks exaggerated but I've worked in two places where we implemented such a process, both of them far more boring than Tesla and, I suspect, far less money to burn on infrastructure. > This can happen, what is important is that such things are discovered. No, what is important when working with mission-critical code is that such things are mitigated. Discovering such a problem in production code is already a problem, not a solution.
- _Codemonkeyism 8y ago
  "There will always be a small handful of engineers that can push the button to move code into PROD" I have very different experiences from a SEC regulated company. With SOX there are controls to prevent such a thing to happen. If this is a SOX breakage, Tesla is in deep trouble with the SEC.
  - gnode 8y ago
    Can you explain how Sarbanes-Oxley applies to the situation of an employee sabotaging a production line? Specifically how it would inherently imply wrongdoing on Tesla's part.
    - CaptainZapp 8y ago
      In the financial industry part of SOX is segregation of duty. As a developer I'm not allowed to have write access to any production system, except in an emergency via a break-glass mechanism, which is audited to the hilt and back. It also means we're not allowed to deploy software to production systems. This has to happen via a specific chain development > regression / user acceptance testing > production. All those environments need to be physically seperated with very specific access requirements. The deployment process needed to be signed off by outr auditors. I can't speak for other banks, but they probably need to implement the same -, or a similar system. Neither can I speak for SOX requirements regarding software fo car manufacturing.
      - _Codemonkeyism 8y ago
        Not only banks, every company listed in the US. It was the same where I've worked, and it wasn't a bank (Enron was no bank either).
      - alanfranzoni 8y ago
        > As a developer What if the employee were a sysadmin-level person that sidestepped the normal process?
        8y ago
- cheeze 8y ago
  Fully agree. In an ideal world all of this would be locked down, but you've gotta be prerty naive to assume that everything is perfectly locked down. There are areas that I couldn't push code willy nilly, namely in the security space. But I'd be willing to bet that a majority of teams at any bigN could have a single bad actor cause some damage... That's just the maturity of the industry.
- EnderMB 8y ago
  Occam's Razor applies here. While you're absolutely right, it is far more likely that someone was able to do this because there is a lack of security in their software engineering practices. This isn't aimed at you, but I think a lot of people are blinded by their support of Elon Musk and Tesla to acknowledge that ultimately he's one man, and Tesla are just a company that makes cars. People and companies are fallable.
- rorykoehler 8y ago
  A good system will ensure that others are notified when this happens. For critical stuff you could also require 2 keys to push code to production so no one individual can do it alone.
- finnthehuman 8y ago
  >There will always be a small handful of engineers that can push the button to move code into PROD or even change code in PROD live. What? Why? Nobody on my development team has access to the production code signing keys. And nobody - at all - has the ability to remotely make a production system take an unsigned update.
  - rndgermandude 8y ago
    But somebody has access to the signing keys/signing process. And somebody has access to the production machines. Etc.
    - finnthehuman 8y ago
      Yes, but that changes the scenario from a "small handful of engineers" that can all do it unilaterally, to needing at least one person from N different teams to collaborate. And in my specific case, the group with the singing keys is also the group paid to tell us "no" whenever a release is blocked by process reasons.
      - rndgermandude 8y ago
        My guess is that most people with access to signing keys or prod environments would have enough skills to code in some sabotage bugs before deployment or siphon off some data, so a lone devops person with (physical) access could probably a lot of harm just by themself.
        finnthehuman 8y ago
        Sure, I’m just saying our commitment to process is strong enough that the technical systems are a funnel into following a reasonable process. I got dragged into defending my technical solution, but my point was that if the ability isn’t needed, don’t have it. You can break the glass when when you need to. Good process make deviation from it more visible. Our code signing keys belong to a team we already need wet ink signatures from to release software. I can go into the biohazard labs with shorts on easier than I can leverage our technical disaster recovery. I’ve only ever had to do the latter.
- wslh 8y ago
  In addition, it would be easy to inject obfuscated bugs that are extremely difficult to find.
- havetocharge 8y ago
  I think it's your comment that's naive. Mature organizations have strict separation of duties, and a great deal of oversight over code reviews and code deployment. It is becoming obvious that Tesla's focus is on execution speed and other aspects (in this case internal security) are suffering.
  - manigandham 8y ago
    You mean all the mature organizations that have security breaches announced every day? Most companies do not have good security. Even when they do, it's hard to get it right, especially for internal attacks. Don't underestimate what a single individual can do when they're already inside and well-informed.
    - yread 8y ago
      To announce a breach you first have to find it. It seems Tesla took quite some time to find this one.
- toddh 8y ago
  You're describing how websites work, but that's not how embedded systems work, especially those were lives are it risk. If this happened then it can only be because good practices were not followed.
- HeyLaughingBoy 8y ago
  This is a very naive comment No, not for a company doing what Tesla does. In the environment I used to work in, that would have been flat out impossible. No change would have been allowed that did not follow process unless an explicit and documented exception was made. Even if you did manage to commit code to the release trunk without review, every single change to the codebase was checked before the release process started in order to verify that the right process was followed. If we ever found a change that no one could find paperwork for, it would be reverted. So, yeah, it's possible if you want to do it badly enough.
tslathrowaway31 8y ago
There were no malicious code changes, rather malicious and deliberate misconfiguration of the software.
- flyingswift 8y ago
  That is not at all what the memo says. "Direct code changes"
  - mortehu 8y ago
    Judging from their username, tslathrowaway31, this could be an insider who knows more than what's in the memo. Regardless, I think configuration should be treated as code (use version control and reviews, write tests), since the damage you can do through misconfiguration is often as great as anything you can do with code.
acover 8y ago
I know in Microsoft it would be relatively easy to push destructive code changes. It depends on the product though. I was able to connect to production machines. There was someone running torrents in the past.
- flyingswift 8y ago
  From my experience on the O365 team, it would have been difficult to push code without approval because it would have voilated a number of the compliance requirements that area of the business has to meet. But yeah, I suppose there are other areas of the company that might have more relaxed requirements
  - XorNot 8y ago
    This doesn't sound like a technical control though?
    - mcintyre1994 8y ago
      You can enforce it on your git server. My experience is with Bitbucket but I'm sure Github has the same options. You could set up something like: nobody can push to master except through the PR UI, all PRs must have a reviewer, no PR can be merged to master without being approved by all reviewers. That way you guarantee that everything on master can be tracked to a dev + a reviewer giving sign off.
      - XorNot 8y ago
        Sure but who admins the git server? At work it's actually me (and my team) - we have backend access to the git repos as a necessary result of that. While it would be tricky to hide changes, if people only merge via PR it would be easy to bury a few extra commits no one worries about provided they don't break anything.
        deleted 8y ago
        [deleted]
whalesalad 8y ago
I guess you’ve never worked at a startup? Tesla is still very much a young and rambunctious company.
- flyingswift 8y ago
  Ok, but we are trusting this 'young and rambunctious' company to produce software for self driving cars, where lack of appropriate policy controls could result in serious consequences if another "disgruntled employee" decides to sabotage the codebase
  - gamblor956 8y ago
    Really, the most worrisome aspect of this story isn't the alleged sabotage: it's that a single unauthorized employee was able to make such widespread changes to the software. Says horrible things about Tesla's security policies, especially for a company claiming that it will unleash true self-driving sometime in the next 6 months.
    - coatmatter 8y ago
      Move fast and break things.
taneq 8y ago
> I find it concerning that one person was able to push malicious code to 'production'. Production line software is almost certainly handled separately from the software that runs their vehicles, and isn't "production" in the usual web sense of being customer facing. This sounds more like someone messed with their factory automation setup.
- finnthehuman 8y ago
  Are you trying to us that less stringent controls on manufacturing software is in any way acceptable? Manufacturing process (including mfg software tools) are a huge potential source of product failure. I don't know about automotive QMS specifically, but I work in embedded software for safety critical systems. If an unreleased procedure or unreleased software is used to build the hardware, or unreleased software is run on the hardware, it's not even suitable for QA (let alone going to the field).
  - nordsieck 8y ago
    >> Production line software is almost certainly handled separately from the software that runs their vehicles, and isn't "production" in the usual web sense of being customer facing. > Are you trying to us that less stringent controls on manufacturing software is in any way acceptable? Yes. It is simply uneconomical to develop all software in a no-bugs-allowed NASA style process. One of the classes of software that can easily be placed in a lower scrutiny bucket is industrial software where the the result of the industrial process will be separately checked by a quality control process. This is in dramatic contrast to safety critical software in cars, for example the battery management software or the fly-by-wire systems. Defects in these systems, in contract to industrial software, are much more difficult to discover by inspection, and failures are much more costly.
    - finnthehuman 8y ago
      Yeah, in that case the controls are moved from the development process to the downstream verification. Sorry if I wasn't clear, I originally had some wording about how known unknowns are handled in my post, but it was getting unnecessarily long and felt like a diversion from the hypothetical I was responding to. I was focused more on documentation are repeatability of process than the quality of the tool themselves. >> I find it concerning that one person was able to push malicious code to 'production'. >Production line software is almost certainly handled separately [...] This sounds more like someone messed with their factory automation setup. For a production line, I would still expect a level of control over exactly what software was running and how it was configured, even if it was SOUP. edit: SOUP not SOAP. I need to stop posting in the morning.
mlindner 8y ago
This meaning of production is different than what you're thinking of. In factories "production" is the assembly line itself.
danpalmer 8y ago
> To me, this suggests that Tesla, a company building highly sensitive software, does not employ basic branch policies. Are they building highly sensitive software? I would suggest that the factory automation systems would be sensitive to downtime more than anything else, and the ability to respond quickly with changes would be very valuable. Merging policies would only slow this down. I would also suggest that there are many other internal tools, from machine learning infrastructure to small internal webapps, or supply chain management software that really aren't sensitive, and where the whole dev team having the _ability_ (whether that's used often or not) to push to master is advantageous. Where is it not appropriate? Customer databases, HR tools, the autonomous driving software, perhaps? Different development teams can work in areas with wildly different sensitivities and priorities, and in many cases it creates a better culture, faster delivery and more ownership to have developers able to push to master. Not always appropriate, but worth striving for when it is.
- CaptainZapp 8y ago
  > Are they building highly sensitive software? Yes! > I would suggest that the factory automation systems would be sensitive to downtime more than anything else If somebody pulls some shenanigans on the sofware, which is responsible for actually building the cars I'd wager that such software is even more critical than the software in the cars. Just imagine some twiddling with the break systems being manufactured. So I don't believe downtime is the major concern. Security of the final product, which is directly derived from such software, is.
  - danpalmer 8y ago
    Sure, changing factory automation in a way that might affect the quality of the finished product would be sensitive. I don't know how much of an opportunity there is to do that – I'd expect it's much more "it works or it doesn't" with factory automation, but I honestly don't know.
- amelius 8y ago
  > Are they building highly sensitive software? A car is a potential murder weapon. So yes. Especially considering it may be remote-controlled, and deployed by the millions.
AYBABTME 8y ago
That's all good and nice but in most cases, this is a matter of policy. If someone actively tries to circumvent the policy or the process, odds are that most software shops would fall victim to the same thing. Especially in the case of internal software. Even if you have a system that attempts to enforce the process, odds are that the system isn't without flaw and is not too hard to circumvent. Most businesses who's primary money maker is not defense or security related... are not running their operation with a focus on defending themselves from themselves.
- mieseratte 8y ago
  > If someone actively tries to circumvent the policy or the process, odds are that most software shops would fall victim to the same thing. Too often the focus is entirely on outside attacks, with little consideration given to insider attacks. Previous job was at a cyber security firm. We'd routinely come under attack from criminal and, we believed, occasional nation state attacks as our researchers attributed a few "Axis-of-Evil" nation-state operations. Then you'd come back from lunch and find the mantrap doors propped open, or someone left a workstation unlocked with root access to something important, or random guests just wandering around. It's a miracle we never were compromised by a disgruntled employee (of which there were many). New job, new industry, same behavior. Folks leaving "secure" doors open, workstations unlocked with root access, and all that jazz. They're happy to cite a vague SEC regulation that may have applied at their last job, but doesn't apply to ours and even then, it's not like an attacker would ever give a flying fuck about what the SEC thinks. I'm starting to wonder if this isn't all a comically bad dream.
  - arcticfox 8y ago
    > Then you'd come back from lunch and find the mantrap doors propped open, or someone left a workstation unlocked with root access to something important, or random guests just wandering around. It's a miracle we never were compromised by a disgruntled employee (of which there were many). Maybe not the most technical solution to this, but one of my previous employers had a workplace culture of setting the desktop background to a rainbow with unicorns if someone left their computer unlocked. Sometimes even teams of two would work together to pwn some of the more alert members, by temporarily keeping them distracted while the other "attacker" did the business. 3 or more fails and the background bumped up to something pretty gross that nobody wanted, simply as a practical matter. I actually thought gamifying it made it pretty fun, and at least for workstation locking, we were elite. Even today I'm sharp about it. Maybe it could be extended to physical access to areas as well.
erikb 8y ago
You would be surprised how many companies would not catch something like that.
twayalways 8y ago
>I find it concerning that one person was able to push malicious code to 'production'. On a vehicle that claims to be able to adjust the strength of your brakes over-the-air, no less.
mrep 8y ago
> How is is it that these changes could have made it through a code review process Most companies allow you to add on code and then push it after it has already been approved (mainly to account for small nitpick changes) > get deployed? I would assume through the normal process? You don't have to immediately try and blow up everything causing an immediate rollback. If you are smart, you'll pull a stuxnet [0] and carefully fuck up the machinery to try and make it go unnoticed. [0]: https://news.ycombinator.com/item?id=17099969 https://news.ycombinator.com/item?id=17099969
paulie_a 8y ago
That is most companies. and Microsoft has such a shitty security record so they don't really need any disgruntled employees.
gchadwick 8y ago
As the under-handed C contest (http://www.underhanded-c.org/ http://www.underhanded-c.org/) demonstrates it's perfectly possible to write code that both looks innocent and apparently is innocent when run in a testing environment. You cannot produce a set of tests that show 'this code never causes anything bad to happen' so a sufficiently skilled malicious programmer could certainly write something that could cause issues that would pass stringent code review, testing and other quality control processes (especially when they have an in-depth understanding of those processes themselves). I have no idea what has happened at Tesla, it's perfectly possible they lack processes they really should have, but you cannot prove that from the information available.
- londons_explore 8y ago
  And even if they do lack review processes, it can't be said that putting in even the strictest of review procedures would stop a determined attacker. It's almost childs play to hide a security vulnerability into a large 1000 line PR, down some error path, through a use after free, hidden deep in a library, a buffer overflow, a debug handler, or anything else really.
Hernanpm 8y ago
That happens at G too, I remember the case where all open wifi were tracked by street view software while mapping streets, and it led to G being sued.
londons_explore 8y ago
Because it wasn't code pushed out to users. It was code pushed to the factory. Codereview is a methodology to get high quality code you can stand behind and put your brand to. Maverick engineers making hodgepodge changes and knocking things together quickly with no oversight is the way you get a proof of concept prototype working fast. For the factory, I take it they're still in (or have returned to) the latter methodology.
toddh 8y ago
At the very least you would expect these images to be signed. There's no way code should be injectable on the floor. The boot loader would simply not accept it. And you would expect any signed images to be code reviewed, regression tested, and system tested. Still, no way for sabotage on the floor.
LinuxBender 8y ago
If they really had access to credentials for multiple accounts, then I don't see why they could not +2 their own code commits. Provided the code still builds and passes basic functional tests, I doubt anyone would see this. Could it be that folks started looking at code commits when vehicles started behaving badly?

SuoDuanDao 8y ago

I would say it's like the difference between China and America having a major environmental accident - we treat developing economies and businesses differently from established ones.

asfasgasg 8y ago
Tesla was founded before Facebook, and I'd argue that Facebook would fall into the same category as Google or Microsoft on this question.

cottsak 8y ago

This is a very naive comment. There will always be a small handful of engineers that can push the button to move code into PROD or even change code in PROD live. Ideally, with mature controls, the people in this list is short. But to jump to the conclusion that Tesla doesn't use good practises is very short sighted. Who's to say that external parties didn't target this person specifically because of their role/influence.

sanderjd 8y ago
> There will always be a small handful of engineers that can push the button to move code into PROD or even change code in PROD live. There is really no reason for this to be the case. Certainly all code that actually runs on the car can be required to go through review and be verifiably built, even if server code standards are more lax.
- nickparker 8y ago
  The code in question is for their manufacturing systems, not the cars. Not that that's necessarily better... Manufacturing equipment's at about the same danger tier as cars.
  - flyingswift 8y ago
    True. It's possible that the manufacturing software could be modified in some manner to introduce some fundamental flaw in the final product though. For that reason, I would say the code should also be held to a higher set of standards
- tehwebguy 8y ago
  > This included making direct code changes to the Tesla Manufacturing Operating System under false usernames If they found a way to use more than one username they may very well have run it through the review process and approved it themselves
- SolarNet 8y ago
  I think you misunderstand. Even with code review policies, there is still a short list of people who can push to production without going through code review. Not from a policy standpoint but from a security and access perspective.
  - falcolas 8y ago
    Which indicates a different problem if the sabatuer was on that short list. And in this day and age of cryptographically signed commits, the number of people who could do this should go down even further. Given this, I think it’s much more likely there were few or bad controls, than a person in an incredibly privileged position working in the margins.
    - SolarNet 8y ago
      Someone builds the code, it doesn't have to be committed.
asfasgasg 8y ago
It seems far more likely that the controls were lax than that the saboteur was one of those privileged few. It's not a naive comment.
EGreg 8y ago
Why is it naive to expect there not to be a single point of failure like that? Code reviews are a thing.
- noobermin 8y ago
  What if they pushed it without asking anyone?
  - davidgay 8y ago
    Then the system that allowed them to do so is badly designed.
    - noobermin 8y ago
      I guess my thinking is if the disgruntled employee was so upset over not getting a promotion to cause so much damage they must have been in a high enough position already to warrant such a disposition, and "high enough" might mean high enough to have push access themselves. Edit: see closeparens comment above. Complicated systems can always be subverted when trust is broken.
    - chillee 8y ago
      At Facebook, you could alter code even after somebody had given the OK for code review. I know some people specifically kept some small commits open after being approved, just so they could quickly make changes without needing approval if they ever needed to.
      - sneak 8y ago
        “Dear respected members of Congress, ...”
      - skj 8y ago
        At Google you can mark something as TBR (to be reviewed), as long as you're one of the designated owners for that code.
xiphias 8y ago
At Google binaries, packages and code are digitally signed by the logged in user and computer
alxlaz 8y ago
Obviously, not all details are available, but the wording in the email suggests that the parent comment is anything but naive: > This included making direct code changes to the Tesla Manufacturing Operating System under false usernames and exporting large amounts of highly sensitive Tesla data to unknown third parties. This sounds like something out of the 1990s, that dark and romantic era of version control when we thought CVS was pretty cool actually and we didn't know what key-based authentication and 2FA were. There are volunteer-ran projects that don't have this problem. Edit: to be clear, I presume no one is debating the fact that someone with high enough credentials can push code to production. The questions that the email raises are: 1. Why can anyone, regardless of credentials, push mission-critical code without review (or, alternatively, if the changes did go through review, why did the review process not catch multiple malicious changes?) 2. Why can someone compromise several high-level credentials without anyone figuring it out (the changes were made, apparently, under "false usernames")?
- alanfranzoni 8y ago
  > 1. Why can anyone, regardless of credentials, push mission-critical code without review (or, alternatively, if the changes did go through review, why did the review process not catch multiple malicious changes?) Why do you suppose the unauthorized party was following the company's development practices? Maybe it was from the sysadmin side, somebody who worked on the toolchain used for reviewing and pushing things to production. So he was able to sidestep the normal review process. This can happen, what is important is that such things are discovered.
  - alxlaz 8y ago
    > So he was able to sidestep the normal review process. He should not have been able to sidestep the normal review process. That's the problem in the first place. Even if you're from the sysadmin side. It should not be possible to do it. You may think that looks exaggerated but I've worked in two places where we implemented such a process, both of them far more boring than Tesla and, I suspect, far less money to burn on infrastructure. > This can happen, what is important is that such things are discovered. No, what is important when working with mission-critical code is that such things are mitigated. Discovering such a problem in production code is already a problem, not a solution.
_Codemonkeyism 8y ago
"There will always be a small handful of engineers that can push the button to move code into PROD" I have very different experiences from a SEC regulated company. With SOX there are controls to prevent such a thing to happen. If this is a SOX breakage, Tesla is in deep trouble with the SEC.
- gnode 8y ago
  Can you explain how Sarbanes-Oxley applies to the situation of an employee sabotaging a production line? Specifically how it would inherently imply wrongdoing on Tesla's part.
  - CaptainZapp 8y ago
    In the financial industry part of SOX is segregation of duty. As a developer I'm not allowed to have write access to any production system, except in an emergency via a break-glass mechanism, which is audited to the hilt and back. It also means we're not allowed to deploy software to production systems. This has to happen via a specific chain development > regression / user acceptance testing > production. All those environments need to be physically seperated with very specific access requirements. The deployment process needed to be signed off by outr auditors. I can't speak for other banks, but they probably need to implement the same -, or a similar system. Neither can I speak for SOX requirements regarding software fo car manufacturing.
    - _Codemonkeyism 8y ago
      Not only banks, every company listed in the US. It was the same where I've worked, and it wasn't a bank (Enron was no bank either).
    - alanfranzoni 8y ago
      > As a developer What if the employee were a sysadmin-level person that sidestepped the normal process?
      - 8y ago
cheeze 8y ago
Fully agree. In an ideal world all of this would be locked down, but you've gotta be prerty naive to assume that everything is perfectly locked down. There are areas that I couldn't push code willy nilly, namely in the security space. But I'd be willing to bet that a majority of teams at any bigN could have a single bad actor cause some damage... That's just the maturity of the industry.
EnderMB 8y ago
Occam's Razor applies here. While you're absolutely right, it is far more likely that someone was able to do this because there is a lack of security in their software engineering practices. This isn't aimed at you, but I think a lot of people are blinded by their support of Elon Musk and Tesla to acknowledge that ultimately he's one man, and Tesla are just a company that makes cars. People and companies are fallable.
rorykoehler 8y ago
A good system will ensure that others are notified when this happens. For critical stuff you could also require 2 keys to push code to production so no one individual can do it alone.
finnthehuman 8y ago
>There will always be a small handful of engineers that can push the button to move code into PROD or even change code in PROD live. What? Why? Nobody on my development team has access to the production code signing keys. And nobody - at all - has the ability to remotely make a production system take an unsigned update.
- rndgermandude 8y ago
  But somebody has access to the signing keys/signing process. And somebody has access to the production machines. Etc.
  - finnthehuman 8y ago
    Yes, but that changes the scenario from a "small handful of engineers" that can all do it unilaterally, to needing at least one person from N different teams to collaborate. And in my specific case, the group with the singing keys is also the group paid to tell us "no" whenever a release is blocked by process reasons.
    - rndgermandude 8y ago
      My guess is that most people with access to signing keys or prod environments would have enough skills to code in some sabotage bugs before deployment or siphon off some data, so a lone devops person with (physical) access could probably a lot of harm just by themself.
      - finnthehuman 8y ago
        Sure, I’m just saying our commitment to process is strong enough that the technical systems are a funnel into following a reasonable process. I got dragged into defending my technical solution, but my point was that if the ability isn’t needed, don’t have it. You can break the glass when when you need to. Good process make deviation from it more visible. Our code signing keys belong to a team we already need wet ink signatures from to release software. I can go into the biohazard labs with shorts on easier than I can leverage our technical disaster recovery. I’ve only ever had to do the latter.
wslh 8y ago
In addition, it would be easy to inject obfuscated bugs that are extremely difficult to find.
havetocharge 8y ago
I think it's your comment that's naive. Mature organizations have strict separation of duties, and a great deal of oversight over code reviews and code deployment. It is becoming obvious that Tesla's focus is on execution speed and other aspects (in this case internal security) are suffering.
- manigandham 8y ago
  You mean all the mature organizations that have security breaches announced every day? Most companies do not have good security. Even when they do, it's hard to get it right, especially for internal attacks. Don't underestimate what a single individual can do when they're already inside and well-informed.
  - yread 8y ago
    To announce a breach you first have to find it. It seems Tesla took quite some time to find this one.
toddh 8y ago
You're describing how websites work, but that's not how embedded systems work, especially those were lives are it risk. If this happened then it can only be because good practices were not followed.
HeyLaughingBoy 8y ago
This is a very naive comment No, not for a company doing what Tesla does. In the environment I used to work in, that would have been flat out impossible. No change would have been allowed that did not follow process unless an explicit and documented exception was made. Even if you did manage to commit code to the release trunk without review, every single change to the codebase was checked before the release process started in order to verify that the right process was followed. If we ever found a change that no one could find paperwork for, it would be reverted. So, yeah, it's possible if you want to do it badly enough.

tslathrowaway31 8y ago

There were no malicious code changes, rather malicious and deliberate misconfiguration of the software.

flyingswift 8y ago
That is not at all what the memo says. "Direct code changes"
- mortehu 8y ago
  Judging from their username, tslathrowaway31, this could be an insider who knows more than what's in the memo. Regardless, I think configuration should be treated as code (use version control and reviews, write tests), since the damage you can do through misconfiguration is often as great as anything you can do with code.

acover 8y ago

I know in Microsoft it would be relatively easy to push destructive code changes. It depends on the product though. I was able to connect to production machines. There was someone running torrents in the past.

flyingswift 8y ago
From my experience on the O365 team, it would have been difficult to push code without approval because it would have voilated a number of the compliance requirements that area of the business has to meet. But yeah, I suppose there are other areas of the company that might have more relaxed requirements
- XorNot 8y ago
  This doesn't sound like a technical control though?
  - mcintyre1994 8y ago
    You can enforce it on your git server. My experience is with Bitbucket but I'm sure Github has the same options. You could set up something like: nobody can push to master except through the PR UI, all PRs must have a reviewer, no PR can be merged to master without being approved by all reviewers. That way you guarantee that everything on master can be tracked to a dev + a reviewer giving sign off.
    - XorNot 8y ago
      Sure but who admins the git server? At work it's actually me (and my team) - we have backend access to the git repos as a necessary result of that. While it would be tricky to hide changes, if people only merge via PR it would be easy to bury a few extra commits no one worries about provided they don't break anything.
      - deleted 8y ago
        [deleted]

whalesalad 8y ago

I guess you’ve never worked at a startup? Tesla is still very much a young and rambunctious company.

flyingswift 8y ago
Ok, but we are trusting this 'young and rambunctious' company to produce software for self driving cars, where lack of appropriate policy controls could result in serious consequences if another "disgruntled employee" decides to sabotage the codebase
- gamblor956 8y ago
  Really, the most worrisome aspect of this story isn't the alleged sabotage: it's that a single unauthorized employee was able to make such widespread changes to the software. Says horrible things about Tesla's security policies, especially for a company claiming that it will unleash true self-driving sometime in the next 6 months.
  - coatmatter 8y ago
    Move fast and break things.

taneq 8y ago

> I find it concerning that one person was able to push malicious code to 'production'. Production line software is almost certainly handled separately from the software that runs their vehicles, and isn't "production" in the usual web sense of being customer facing. This sounds more like someone messed with their factory automation setup.

finnthehuman 8y ago
Are you trying to us that less stringent controls on manufacturing software is in any way acceptable? Manufacturing process (including mfg software tools) are a huge potential source of product failure. I don't know about automotive QMS specifically, but I work in embedded software for safety critical systems. If an unreleased procedure or unreleased software is used to build the hardware, or unreleased software is run on the hardware, it's not even suitable for QA (let alone going to the field).
- nordsieck 8y ago
  >> Production line software is almost certainly handled separately from the software that runs their vehicles, and isn't "production" in the usual web sense of being customer facing. > Are you trying to us that less stringent controls on manufacturing software is in any way acceptable? Yes. It is simply uneconomical to develop all software in a no-bugs-allowed NASA style process. One of the classes of software that can easily be placed in a lower scrutiny bucket is industrial software where the the result of the industrial process will be separately checked by a quality control process. This is in dramatic contrast to safety critical software in cars, for example the battery management software or the fly-by-wire systems. Defects in these systems, in contract to industrial software, are much more difficult to discover by inspection, and failures are much more costly.
  - finnthehuman 8y ago
    Yeah, in that case the controls are moved from the development process to the downstream verification. Sorry if I wasn't clear, I originally had some wording about how known unknowns are handled in my post, but it was getting unnecessarily long and felt like a diversion from the hypothetical I was responding to. I was focused more on documentation are repeatability of process than the quality of the tool themselves. >> I find it concerning that one person was able to push malicious code to 'production'. >Production line software is almost certainly handled separately [...] This sounds more like someone messed with their factory automation setup. For a production line, I would still expect a level of control over exactly what software was running and how it was configured, even if it was SOUP. edit: SOUP not SOAP. I need to stop posting in the morning.

mlindner 8y ago

This meaning of production is different than what you're thinking of. In factories "production" is the assembly line itself.

danpalmer 8y ago

> To me, this suggests that Tesla, a company building highly sensitive software, does not employ basic branch policies. Are they building highly sensitive software? I would suggest that the factory automation systems would be sensitive to downtime more than anything else, and the ability to respond quickly with changes would be very valuable. Merging policies would only slow this down. I would also suggest that there are many other internal tools, from machine learning infrastructure to small internal webapps, or supply chain management software that really aren't sensitive, and where the whole dev team having the _ability_ (whether that's used often or not) to push to master is advantageous. Where is it not appropriate? Customer databases, HR tools, the autonomous driving software, perhaps? Different development teams can work in areas with wildly different sensitivities and priorities, and in many cases it creates a better culture, faster delivery and more ownership to have developers able to push to master. Not always appropriate, but worth striving for when it is.

CaptainZapp 8y ago
> Are they building highly sensitive software? Yes! > I would suggest that the factory automation systems would be sensitive to downtime more than anything else If somebody pulls some shenanigans on the sofware, which is responsible for actually building the cars I'd wager that such software is even more critical than the software in the cars. Just imagine some twiddling with the break systems being manufactured. So I don't believe downtime is the major concern. Security of the final product, which is directly derived from such software, is.
- danpalmer 8y ago
  Sure, changing factory automation in a way that might affect the quality of the finished product would be sensitive. I don't know how much of an opportunity there is to do that – I'd expect it's much more "it works or it doesn't" with factory automation, but I honestly don't know.
amelius 8y ago
> Are they building highly sensitive software? A car is a potential murder weapon. So yes. Especially considering it may be remote-controlled, and deployed by the millions.

AYBABTME 8y ago

That's all good and nice but in most cases, this is a matter of policy. If someone actively tries to circumvent the policy or the process, odds are that most software shops would fall victim to the same thing. Especially in the case of internal software. Even if you have a system that attempts to enforce the process, odds are that the system isn't without flaw and is not too hard to circumvent. Most businesses who's primary money maker is not defense or security related... are not running their operation with a focus on defending themselves from themselves.

mieseratte 8y ago
> If someone actively tries to circumvent the policy or the process, odds are that most software shops would fall victim to the same thing. Too often the focus is entirely on outside attacks, with little consideration given to insider attacks. Previous job was at a cyber security firm. We'd routinely come under attack from criminal and, we believed, occasional nation state attacks as our researchers attributed a few "Axis-of-Evil" nation-state operations. Then you'd come back from lunch and find the mantrap doors propped open, or someone left a workstation unlocked with root access to something important, or random guests just wandering around. It's a miracle we never were compromised by a disgruntled employee (of which there were many). New job, new industry, same behavior. Folks leaving "secure" doors open, workstations unlocked with root access, and all that jazz. They're happy to cite a vague SEC regulation that may have applied at their last job, but doesn't apply to ours and even then, it's not like an attacker would ever give a flying fuck about what the SEC thinks. I'm starting to wonder if this isn't all a comically bad dream.
- arcticfox 8y ago
  > Then you'd come back from lunch and find the mantrap doors propped open, or someone left a workstation unlocked with root access to something important, or random guests just wandering around. It's a miracle we never were compromised by a disgruntled employee (of which there were many). Maybe not the most technical solution to this, but one of my previous employers had a workplace culture of setting the desktop background to a rainbow with unicorns if someone left their computer unlocked. Sometimes even teams of two would work together to pwn some of the more alert members, by temporarily keeping them distracted while the other "attacker" did the business. 3 or more fails and the background bumped up to something pretty gross that nobody wanted, simply as a practical matter. I actually thought gamifying it made it pretty fun, and at least for workstation locking, we were elite. Even today I'm sharp about it. Maybe it could be extended to physical access to areas as well.

erikb 8y ago

You would be surprised how many companies would not catch something like that.

twayalways 8y ago

>I find it concerning that one person was able to push malicious code to 'production'. On a vehicle that claims to be able to adjust the strength of your brakes over-the-air, no less.

mrep 8y ago

> How is is it that these changes could have made it through a code review process Most companies allow you to add on code and then push it after it has already been approved (mainly to account for small nitpick changes) > get deployed? I would assume through the normal process? You don't have to immediately try and blow up everything causing an immediate rollback. If you are smart, you'll pull a stuxnet [0] and carefully fuck up the machinery to try and make it go unnoticed. [0]: https://news.ycombinator.com/item?id=17099969 https://news.ycombinator.com/item?id=17099969

paulie_a 8y ago

That is most companies. and Microsoft has such a shitty security record so they don't really need any disgruntled employees.

gchadwick 8y ago

As the under-handed C contest (http://www.underhanded-c.org/ http://www.underhanded-c.org/) demonstrates it's perfectly possible to write code that both looks innocent and apparently is innocent when run in a testing environment. You cannot produce a set of tests that show 'this code never causes anything bad to happen' so a sufficiently skilled malicious programmer could certainly write something that could cause issues that would pass stringent code review, testing and other quality control processes (especially when they have an in-depth understanding of those processes themselves). I have no idea what has happened at Tesla, it's perfectly possible they lack processes they really should have, but you cannot prove that from the information available.

londons_explore 8y ago
And even if they do lack review processes, it can't be said that putting in even the strictest of review procedures would stop a determined attacker. It's almost childs play to hide a security vulnerability into a large 1000 line PR, down some error path, through a use after free, hidden deep in a library, a buffer overflow, a debug handler, or anything else really.

Hernanpm 8y ago

That happens at G too, I remember the case where all open wifi were tracked by street view software while mapping streets, and it led to G being sued.

londons_explore 8y ago

Because it wasn't code pushed out to users. It was code pushed to the factory. Codereview is a methodology to get high quality code you can stand behind and put your brand to. Maverick engineers making hodgepodge changes and knocking things together quickly with no oversight is the way you get a proof of concept prototype working fast. For the factory, I take it they're still in (or have returned to) the latter methodology.

toddh 8y ago

At the very least you would expect these images to be signed. There's no way code should be injectable on the floor. The boot loader would simply not accept it. And you would expect any signed images to be code reviewed, regression tested, and system tested. Still, no way for sabotage on the floor.

LinuxBender 8y ago

If they really had access to credentials for multiple accounts, then I don't see why they could not +2 their own code commits. Provided the code still builds and passes basic functional tests, I doubt anyone would see this. Could it be that folks started looking at code commits when vehicles started behaving badly?