13 ms·
Trains were designed to break down after third-party repairs, hackers find
- Dah00n 3y ago>According to Dragon Sector, Newag entered code into the control systems of Impuls trains to stop them from operating if a GPS tracker indicated that the train was parked for several days at an independent repair shop. Oh, that seems pretty damning. I wonder if this was a lone developer or ordered from above.
- runiq 3y ago> Oh, that seems pretty damning. I wonder if this was a lone developer or ordered from above. Guess. (Did I miss the sarcasm again? I always miss the sarcasm.)
- stavros 3y agoWhat lone developer says "let me find all the GPS coordinates of all the train repair shops and make the trains not work if they've been parked there for days"?
- ykonstant 3y agoDon't underestimate these damned rouge developers!
- grudg3 3y agoI've heard of Khmer Rouge but not of Dev Rouge before.
- dirtyhippiefree 3y agoHumor is so little appreciated these days.
- smolder 3y agoHumor is prevalent in the internets various forums. It's well appreciated generally - easy upvotes on Reddit and elsewhere. But that also means that a serious forum with higher signal-to-noise ratio is something uniquely valuable.
- deleted 3y ago[deleted]
- Yizahi 3y ago"My cat slept on the keyboard and accidentally developed and merged working advertisement feature into a release build of the new AC game. It was totally on accident, happens all the time." (c) Ubisoft
- LegitShady 3y ago"I come to you now with apologies. This is the third time my cat has put paid mods into Skyrim, something for which he is very very sorry you are unhappy about."
- tiborsaas 3y agoYeah, sounds like a secret 20% project :)
- amelius 3y agoManagement: "a 6-pack of beer for the brilliant dev who figures out a solution to this problem of us ..."
- saalweachter 3y agoWe talk on HN a lot about ethics and programming and the importance of engineers refusing to develop unethical software. But I think we need another PSA that, if you are going to write some code that could land you in jail, make sure you get your entire reporting chain in writing telling you to do it, up to and including the CEO and the board of directors.
- nmeofthestate 3y agoAlternatively, if you're going to break the law, don't do that (break the law).
- sgift 3y agoAll of them. Dieselgate? Rogue dev. Trains breaking for independent shops? Rogue dev. Some airplane going down because of bad software? Rogue dev. Repeat after me: The company is never at fault.
- u32480932048 3y agoPerhaps GitHub Autopilot and ChatGPT teamed up to write the code upon noticing that the developer tripped and fell, causing a GSM modem to fall off their desk and into the assembly process.
- soco 3y agoIs this really suggesting some developer, buried under tasks and putting extra hours, thought to implement this feature in his free time for months, slipped it through reviews and scans and pipeline deployments, for... what? Really? I rather prefer to think the comment was sarcasm.
- blackbear_ 3y agoYes it was sarcasm, since it is common for management to shift the blame on individual engineers in these cases (see for example the Volkswagen emissions scandal and the Boeing MAX crash).
- sgerenser 3y agoIIRC Volkswagen actually used the term “rogue engineer” when speaking to the press.
- bluGill 3y agoIn VW's case there was some reason to believe that at first. There are reasons to test an engine in the lab without emissions controls, and so it is believable that a lazy engineer could put in code for the lab that also triggered in real world conditions. Note that the above is only believable before the rest of the investigation. Proper investigation proved the story wrong, but it is just possible enough to believe it could be true if the proper investigation hod come up differently.
- rob74 3y agoThey will surely try that explanation too if the others fail. Currently their stance seems to be "there is no such code on the trains, and if there is, it's not from us".
- prox 3y agoIt’s from those darn third party developers! ;)
- jacquesm 3y agoBlame the hacker, wait for it. Also: I wonder if their management realizes that they probably have a nice trail in the form of a bunch of repositories and commit messages. Would be nice if that leaked.
- meithecatte 3y agoYou're talking about industrial PLCs. They're programmed using a-bit-more-fancy Scratch snappy blocks. There is no version control. The firmware contained paths embedded as strings, so we know that firmware for each model and customer was developed in a separate folder on disk. I wouldn't be surprised if they also had .zip files with backups of previous versions.
- planede 3y agoAs a lone developer there is absolutely no motivation to do this and hide this decision from management. Having said that a lone developer can come up with the idea, propose it to management with the expectation of some fat bonus. But why do this in secret? There is just no possibility where upper management is not involved in this.
- whstl 3y ago100% agree. Doing this in secret without management's express approval means no bonus, working extra for no benefit, compromising your other tasks and exposing yourself to heavy liability for compromising the country's infrastructure. This is something that companies could get away with, but a single developers would not only lose their job, but risk going to jail.
- bluGill 3y agoNote that management's express approval could be verbal conversations that you cannot prove happened or not. With some work management can find some way to get a developer who has some other conflict of interest to do this: if your brother in law works for the maintenance yard situations then you have motive to ensure he has work.
- JumpCrisscross 3y ago“Newag president Zbigniew Konieczek said that ‘no evidence was provided that our company intentionally installed the faulty software. In our opinion, the truth may be completely different—that, for example, the competition interfered with the software.’” The ridiculousness of this defence makes it clear leadership was in the know.
- HelloNurse 3y agoI hope evidence will be provided in court.
- theginger 3y agoI guess they can't claim any copy write / drm violations if they also deny it was their code.
- dcow 3y agoRiiight?! This is a really strong defense for the white hats. Either it’s your code and you are egregious slime balls doing illegal things, or “some third party interfered and hacked your trains like you said and we fixed it you’re welcome”.
- lesuorac 3y agoI'm not sure that's the smoking gun. IIRC, there was some "konami" code to re-enable the train after disablement that was removed in a first-party update after the third-party repair company found the sequence. And not related to the third-party locations, one of the trains had some code where if the year>2021 & month>11 & day>? then the train would disable itself; ultimately doing it in the wrong year because the train was off during november / december of that year. This is a little excessive for somebody random to do.
- hifromwork 3y agoThe 2021-11-?? date was the day the train was scheduled to be serviced (so the intention of the code was to brick the code after the service). Only by random chance it didn't work, because the train was in maintenance a bit longer, and the code didn't trigger in January 2022 because of the date checking bug.
- crest 3y agoHis dog did it after eating the CS homework.
- stby 3y agonpm add block-repair-shop. It can happen so fast.
- figassis 3y agoIf an employee did this without authorization, the company should still be liable because I do not think safety regulations exempt manufacture from responsibility of employee actions. I believe the opposite is true, that they must ensure safety despite rogue employees, with reviews, audits and such.
- bluGill 3y agoThey need to show audits and that the code is obscure enough that it could be missed. I've seen IOCCC type things that do things I wouldn't catch in a code review. (IIRC there is a malicious code contest where the goal is to put backdoors in that others don't see in review, but I can't recall what it is)
- 8372049 3y agoThere's been a decent amount of those in the wild, too. If you're in an environment without unicode auditing you can do all sorts of crazy things without IOCCC levels of code tricks too.
- nneonneo 3y agoYou’re thinking of the Underhanded C contest: http://www.underhanded-c.org/ http://www.underhanded-c.org/. The goal is to write C code which looks completely innocent, but which actually performs some malicious or unwanted behaviour. In the last year they ran the contest (2016), the goal was to write a program that would compare two sets of measurements for similarity (nominally radiation emission spectra) but fail to give a correct answer when given a particular crafted input.
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- Tangurena2 3y agoI suspect this is going to end all further sales of these trains outside of Poland. What railway will ever trust them again?
- DeathArrow 3y ago[flagged]
- bluescrn 3y agoIf they did, the software would be fine. They'd just be held together with adhesives to make repair far harder than necessarily. And nobody would be able to sell spare parts.
- switch007 3y agoYou’d be able to get in to the driver’s cab by just pressing Enter on the keypad
- Yizahi 3y agoNo, until they could make a proprietary tracks, on which no other train would work.
- baz00 3y agoIf they did they'd look nice and run on time.
- fransje26 3y agoHere is a better write-up of the story: https://badcyber.com/dieselgate-but-for-trains-some-heavyweight-hardware-hacking/ https://badcyber.com/dieselgate-but-for-trains-some-heavywei...
- KolmogorovComp 3y agoAnd the relevant HN discussion: https://news.ycombinator.com/item?id=38567687 https://news.ycombinator.com/item?id=38567687 5 days ago, 289 comments
- manuel_w 3y agoWhich again is a successor to: https://news.ycombinator.com/item?id=38530885 https://news.ycombinator.com/item?id=38530885 8 days ago, 357 comments
- Yizahi 3y agoThat headline was so misleading. I'm interested in the topic and saw that link on HN, but scrolled away because dieselgate is "obviously" about emissions and I assumed that nothing new will come from the article about train emissions, we all know already they are big polluters but we kinda have to deal with it. They should have mentioned something about vendor lock or right to repair in the headline.
- rsynnott 3y ago> I assumed that nothing new will come from the article about train emissions, we all know already they are big polluters but we kinda have to deal with it. ... Eh? Trains, to be clear, are not big polluters; not sure what made you think that.
- iamleppert 3y ago[flagged]
- buzzin_ 3y agoI don't think this particular case was a lone wolf problem.
- peteradio 3y agoI think the above is actually well crafted sarcasm. But it does highlight why as an engineer/developer it is important to get requirements in writing and have a process that shows why you made the changes you ultimately made.
- jacquesm 3y agoAbout as likely as dieselgate.
- usrusr 3y agoLone pawn, as in doing directly for the strong person at the top (owners family?) what the top would never dare to order through the regular chain of command would not be quite as impossible.
- p_l 3y agoMultiple people had to be involved, not just programmers.
- roblabla 3y agoSeriously? I find it very unlikely this is the work of a "lone wolf developer". Newag management and executives knew about this. Some of these trains were configured (poorly) to "break" at a scheduled date that just so happen to line up with when those trains were scheduled to go get serviced[0]. Lone wolf developers usually don't have the ability to make builds/configurations for a particular deployment of the software... [0]: https://badcyber.com/dieselgate-but-for-trains-some-heavyweight-hardware-hacking/ https://badcyber.com/dieselgate-but-for-trains-some-heavywei... - Section "Checking the date is hard"
- andirk 3y agoI was expecting some malicious code like "knock 3 times on this door to enter, otherwise the door locks for good". No. Instead it was "if you're geographically parked at our competitor's lat/long then we'll brick the train". There's ways to add back doors and other fun easter eggs in code, but wow they picked a really cumbersome and obvious method of exploitation.
- throw_a_grenade 3y agoYou almost guessed it though. There was a (since removed) secret combination of buttons on driver console that magically unlocked the train.
- moffkalast 3y ago"hesoyam"
- Tade0 3y agoReading their official statement it's hard to not get the impression that it wasn't even consulted with their attorney - they deny specific allegations in a way that just begs further investigation. I mean, I'm Polish and worked for such people in the past. The statement reeks of an approach to business that I wish went extinct already, but apparently you still see pockets of it here and there.
- jijji 3y agoI have seen developers lock/encrypt their code they write and then when they get fired for it, they want the company to pay a fee to get them to unlock it/release source code
- QVVRP4nYz 3y agoThat is textbook https://en.wikipedia.org/wiki/Logic_bomb https://en.wikipedia.org/wiki/Logic_bomb and is very "jailable" - as if plain blackmail laws weren't enough. Edit: someone already added Newag there :D
- jacquesm 3y agoThey better have some code around it that stops that from happening when the train is in service.
- NiloCK 3y agoIs there any conceivable point where people will be charged and jailed for fraud after selling sabotaged goods?
- mouse_ 3y agoWhy would the powerful punish themselves?
- p_l 3y agoBecause more powerful entities might be enraged.
- matkoniecz 3y agoThemself? No. But there is more than one powerful person in Poland, and Newag owner is far from the most powerful.
- p_l 3y agoTheoretically, the recently-revealed prosecution started with notice of two crimes that include 0.5 to 8 years jail time.
- deleted 3y ago[deleted]
- H8crilA 3y agoThe real question is whether or not we will get quality reporting from the courts! Hopefully someone goes to jail over this.
- InsomniacL 3y ago> The code also allegedly bricked the train if "certain components had been replaced without a manufacturer-approved serial number," 404 Media reported. If only commercial airliners had this too... https://news.sky.com/story/fraud-officers-arrest-one-in-dawn-raid-on-uk-engine-parts-provider-13024145 https://news.sky.com/story/fraud-officers-arrest-one-in-dawn...
- p_l 3y agoCommercial airliners have other procedures, including - just like with trains in europe - complete unbundling of maintenance and repair. Because it's pretty normal for an aircraft to be in use longer than the manufacturer exists.
- JonChesterfield 3y agoThis is on the general topic of right to repair and freedom of fixing and so forth so naturally HN hates the train company. On the other hand, trains that refuse to start don't immediately kill everyone on board, and trains that have fall off the tracks do tend to kill people. So when a train fails and people are dead, and people go in search of who is responsible, it seems decently likely that ambiguity between the third party repair shop and the company responsible for the train is a problem. Further, if your company is financially liable for a product failure, and another company is not, that second company can totally fix it more cheaply than you can. As it's not their liability. There should be an opt out system - something where the train operator contacts the train supplier and says "we want to use this cheaper repair shop, and it's now our problem when the budget train fix kills people" - that seems totally legitimate to me. Or to bring it closer to home, say one of us breaks into our tesla to exercise our God given right to change the software stack, and then it kills the driver and various people around it, to what extent is that Tesla's fault? Say Tesla made all reasonable steps they could take to detect third party mods and refuse to start the car, do we hate them too?
- cnity 3y agoThe same viewpoint applies to any motor vehicle too. How many road accidents are due to mechanical error introduced by a mechanic (that does not operate on behalf of the car manufacturer)? What makes an "official" mechanic employed by the manufacturer superior to a third party? This is an interesting discussion, thank you for your nuanced opinion. I don't know where I stand on it but you are spot on about the demographics of HN. I certainly have a knee-jerk reaction in the direction of operator freedom. Edit: Additionally, if something like this goes wrong (say the train derails) it will be a massive blow to right to repair. Even if it is later found that the third party repair is not responsible, the media will scoop up this story and it will forever warp the public's ideas about right to repair.
- clucas 3y agoYou are putting forth some general rules that seem reasonable, but they don't really apply in this situation. From what I've read, the train operator bid out the maintenance for these trains, and the manufacturer lost the bid to a third party. So your "opt-out" system should absolutely have been triggered here. If there was some reason to believe that the low bidder would be incapable of safely performing the maintenance, the train manufacturer should have raised that issue publicly, not silently sabotaged the low bidder's ability to perform the maintenance.
- silexia 3y agoRight to repair for anything we buy is essential and customers need the right to access software they have purchased.
- samhuk 3y agoTL;DR: * Polish train maintenance company, SPS, was getting suspicious as trains made by a company, Newag, kept on "randomly" breaking and couldn't be fixed. They was getting fined millions by Polish government as they had a contract that fined them for being too slow with repairs. * They secretly hired literal hackers (Dragon Sector) for 2 months to dig around Newag train code. * Hackers found out some incredible things, generally that fit under the umbrella of "late-stage capitalism", or more specifically, corporate protectionism, sabotage, ransom, etc. Some examples of the secret code that the hackers found: * Breaks the trains if they go into geo polygons that are right around the warehouses of 5 Polish train maintenance companies, including SPS. * Breaks the trains after 1 million kilometers. * Breaks the trains if they don't move for 10 days. * Secret button press combination (basically Tekken, Street Fighter, etc.) to disable the "malfunctions".
- swagmoney1606 3y agoDo you know if the low-level technical report is available? I love reverse engineering firmware, and this sounds like a holy grail. I mean a freaking train? Ugh someone should drop binaries.
- samhuk 3y agoNot found one yet. The linked article contains a very small amount of detail, such as the lat-long coordinate values they found within the dissassembled code, etc., but not much else unfortunately. I'm waiting for the stuxnet-like report on this as much as anyone.
- PeterisP 3y agoSome info is here https://badcyber.com/dieselgate-but-for-trains-some-heavyweight-hardware-hacking/ https://badcyber.com/dieselgate-but-for-trains-some-heavywei... , but the actual details will be presented at the Chaos Communication Congress at the end of December.
- tstrimple 3y agoWhat noobs. Everyone knows you've got to put that chicanery behind a web service call so they can't find the evidence directly from the client.
- octacat 3y agopretty sure even if they have source code version control, the "funny" patches are outside of the version control and applied on top before making a build.
- eschneider 3y agoThat's almost certainly not the case. In general for vehicles, you're installing signed, verified images. Not only would it be a LOT harder to consistently apply OOVC patches that don't show up anywhere, having signed builds that don't match your source would be a worse red flag than just having the code in version control.
- hgomersall 3y agoYou do get deniability though. Given their line is "this is malicious code that was put on the train. Physical security is the responsibility of the train companies”, they would argue the fact their binaries are different is proof of malicious actors. Essentially, if they're going to do this, a good route is for them to be the malicious actors. Just before shipping, someone trusted surreptitiously switches the binaries to the modded versions.
- octacat 3y agoyes, they probably have a "clean" build and "a build with extra functionality" build. So, they could even install a clean build for some trains. Though, investigators could just compare firmware from all trains made so far. Another question if it would be even be investigated.
- jakub_g 3y agoPrevious threads on same news: - https://news.ycombinator.com/item?id=38628635 https://news.ycombinator.com/item?id=38628635 - https://news.ycombinator.com/item?id=38632033 https://news.ycombinator.com/item?id=38632033
- dang 3y agoThanks! Macroexpanded: Polish Hackers that repaired DRM trains threatened by train company - https://news.ycombinator.com/item?id=38628635 https://news.ycombinator.com/item?id=38628635 - Dec 2023 (137 comments) Polish train maker denies claims its software bricked competitor rolling stock - https://news.ycombinator.com/item?id=38570654 https://news.ycombinator.com/item?id=38570654 - Dec 2023 (2 comments) Dieselgate, but for trains – some heavyweight hardware hacking - https://news.ycombinator.com/item?id=38567687 https://news.ycombinator.com/item?id=38567687 - Dec 2023 (292 comments) Polish trains lock up when serviced in third-party workshops - https://news.ycombinator.com/item?id=38530885 https://news.ycombinator.com/item?id=38530885 - Dec 2023 (359 comments)
- Pesthuf 3y agoI sure hope the dev who was made to implement it kept the paperwork that proves who made them do it. Otherwise, they’ll find out how loyalty is rewarded nowadays.
- Gud 3y agoTo be honest, with or without a paper trail, society need less people who do shit like this. Whoever implemented this, if you read this, you are a very bad person and destructive for society.
- 2358452 3y agoI disagree with the framing "You are a bad person" (although I think I understand the sentiment), because it implies they can't change (or understand the error). It seems better to leave it at "You did something very harmful, destructive for society".
- Gud 3y agoI agree. I understand sometimes a person is under enormous pressure to do something they know is not right.
- banannaise 3y agoPeople respond to incentives. That's the entire premise of capitalism. If you want this to not happen, you need to incentivize refusing to do this more than the existing incentive to do this. Otherwise, punishment only changes who's in the chair next time. (Future possible punishment is not a very good disincentive, which is why increased prison sentences don't deter crime.)
- dreen 3y agoEven if they did keep that paperwork, they should be punished alongside the manager who asked them to do it. They knew it was unethical and did it anyway. They should have refused. Risk of being fired is not as big of a problem for a programmer as it is for almost anyone else.
- ChrisArchitect 3y ago[dupe] Does no one read the site anymore? original news story discussion just over a week ago: https://news.ycombinator.com/item?id=38530885 https://news.ycombinator.com/item?id=38530885 And the followup from the company Polish train maker denies claims its software bricked competitor rolling stock https://news.ycombinator.com/item?id=38570654 https://news.ycombinator.com/item?id=38570654 Not to mention the 404media dupe with a ton of votes just yesterday also! https://news.ycombinator.com/item?id=38628635 https://news.ycombinator.com/item?id=38628635
- gambiting 3y ago>>Does no one read the site anymore? I don't know, is it some kind of requirement to read every piece of news on the main page every single day to keep visiting? Or can I log in every few days, read a few stories, upvote the ones I like, then visit again few days later? Or is that not allowed?
- ChrisArchitect 3y agoMore on the submitter in this case. Upvote what you see/like, sure, all good. But if you're not around, you miss some stories, that's it. (especially when it's come up so many times over weeks) There is a current events aspect to HN.
- matheusmoreira 3y ago> Does no one read the site anymore? I don't read the vast majority of sites linked here. I come here for the comments. I assume anything that's important enough to know will be directly quoted in a comment here.
- deleted 3y ago[deleted]
- jacquesm 3y agoWhat really bugs me about all this: not only were they designed to break down, they messed that up to the point that the trains could have broken down while in service. The fact that a manufacturer would risk the lives of the passengers of the trains should result in personal liability for all of the execs of that company. https://news.ycombinator.com/item?id=38641289 https://news.ycombinator.com/item?id=38641289 I'd love to see that angle researched more because I think it changes the game from something commercial to a far more important level.
- game_the0ry 3y ago[flagged]
- SR2Z 3y agoThe article describes Polish trains.
- stephenitis 3y agoRead the article. Wrong country
- HumblyTossed 3y agoWhen bad things happen, the execs don't know it happened because they tell other people to go off and do stuff. When good things happen, execs take all the credit because the told people to go off and do stuff.
- usea 3y agoWhen my wife was in business school, the teacher of her management class used this idea as an unironic definition of management. They had to memorize the definition, which was essentially "taking credit for the work of other people." Not as a joke.
- jacquesm 3y agoThat's painful. But I totally believe it.
- SenAnder 3y agoWe lament that circumventing DRM is legally perilous, but sabotaging your customer's property is illegal. That you were the one that sold that property makes no difference - you sold it, it's not yours anymore, despite corporate messaging otherwise.
- tamimio 3y ago> Newag has said that "any remote intervention" is "virtually impossible." That’s a very bold statement right there! Also, from my experience, most if not all companies who build automation/robotic systems they maintain “backdoors” of some sort, not to disable them remotely, but to quickly and efficiently access the software remotely upon request to troubleshoot or fix issues, it doesn’t make sense to fly to the client and sometimes the other half of the planet just find out it was a local IP mismatch or a system service is down, so saying “impossible” is not true.
- jcul 3y agoI'm slightly envious of the reverse engineers that got to dig into this. I can imagine the initial disbelief and shock as you start to piece together what the software is doing. So malicious, I wouldn't believe it at first.