6 ms·

There is something wrong with the description of the laptop use (besides calling DOS a language) - on one hand they are running windows 10 with Win98 virtualize

by jalk 4y ago

There is something wrong with the description of the laptop use (besides calling DOS a language) - on one hand they are running windows 10 with Win98 virtualized. Then they mention Thinkpads circa 2000. I'm pretty sure Win10 won't run on those machines, so I guess the old laptops are for the interface to the train computers - RS-232 is my guess (could be the parallel port). If all they are using the DOS software for is to dump a log from the train, then it sound to me, that sniffing the protocol would be fairly simple. Calling people from bay-area hacker spaces to give them a hand in creating something more modern.

RajT88 4y ago
Would agree with all of this. The problem is that BART doesn't employ talented reverse engineers. There is now way they could afford them in that area. Also: Talented reverse engineers are hard to find on a good day in any given area. I used to think that my mentors from the start of my tech career were just the baseline level of sophistication you'd expect of any decent software engineer. 20 years later I have seen how few people have the mindset to get into that sort of thing. (And it is a mindset much more than a level of intellect, IMO)
- mh- 4y ago
  I don't ride Bart anymore, but plenty of us would have done this work for free.. just to see a small chance at improving the system. The dysfunction is broader than them being unable to afford engineering talent.
  - parker_mountain 4y ago
    > plenty of us would have done this work for free This is the interface to the train control system - a primary safety system - responsible for the lives of literally a million people on an average month. Are you prepared to spend months of your lives rigorously testing and phasing this system in? Are you prepared for the intense legal liability if something goes wrong?
    - mh- 4y ago
      Sorry, I should have been clearer. "The work" here referred to assisting with the reverse engineering aspect of this. I, and almost everyone else here, lack the domain expertise to be of any use in that.
      - deleted 4y ago
        [deleted]
    - gridspy 4y ago
      If an individual engineer is put in a position where they can endanger lives by accident, there is something wrong with the management and systems in place around that person. This is a team game. The people figuring out how the comms works and creating tests for those communications then hand their work off to the "rigor" team.
      - Doxin 4y ago
        Or do like what the dutch railways are doing: run the old and new systems in parallel on real inputs and check if the outputs match. Then proceed to only use the output for the old system until you're really really sure there's no bugs.
        jabl 4y ago
        I don't work in safety-critical hard real-time systems, but isn't this the standard approach when upgrading such a system?
        Doxin 4y ago
        I'd imagine so, I just knew about the dutch railways for certain. IIRC they are doing a mechanized translation from COBOL to C++. The idea being to first test that exhaustively, and then slowly manually translate the COBOL-y-C++ to more idiomatic C++.
        4y ago
- ChuckNorris89 4y ago
  >Also: Talented reverse engineers are hard to find on a good day in any given area Not true. Have you looked in Europe? It's full of them thanks to the tinker culture and the presence all big anti-malware companies (Esset, Avast, Avira, Bit-Defender, Kaspersky, etc). Plus every single cybersecurity consulting company out there in the world, big and small, has guys who can do reverse engineering for you. There's thousands of these companies, just look them up. The reason you're probably not finding them is because you're probably not looking for them or that many juniors who don't have the "RE bug" don't stay in that gig professionally too long because, unless you're a zero-day finding God, cashing out six figure CVE bounties and giving Defcon talks, the pay is worse than a webdev doing CRUD since the demand for reverse engineering work is very low and not valued by customers as it's not the kind of job that impacts their sales or share price so they just look to offshore it to the lowest bidder. I also left the field and moving to cloud engineering since the demand, pay, benefits, respect is so much better.
  - dagw 4y ago
    Not true. Have you looked in Europe? I wonder to what extent that is still the case. When I graduated 15ish years ago it was definitely true that many of my classmates had those low level skills, and serval of them had written their Masters thesis on something related. However looking at people graduating from the same program today, I hardly see anybody doing that sort of work. Everything is either some sort of high level web stuff or machine learning. Even the people I knew who had those skills haven't used them professionally in over decade since, as you say, the pay sucks.
    - ChuckNorris89 4y ago
      >I wonder to what extent that is still the case. It is. The university in my area has a famous master's program for cybersecurity and many grads there know reverse engineering to a degree. And there a re many such universities in Europe, plus all the Eastern European gray beards who work in the industry since they cut their teeth breaking DRM to pirate games in their youth. I personally know at least 5 people who can do this work professionally at a consulting company.
- inferiorhuman 4y ago
  The problem is that BART doesn't employ talented reverse engineers. There is now way they could afford them in that area. Well, no. Money is not the problem and BART doesn't need reverse engineers. They've already thrown away tens of millions of dollars on a replacement train control system (AATCS). Everyone's favorite defense contractor Raytheon was tasked with the project until they realized they couldn't make it work. Eventually Raytheon pawned the project off on Harmon, which was later bought by General Electric. GE shut the whole thing down because it was going nowhere. I think most folks here are drastically underestimating the complexity involved in getting a train control system right (and how badly BART botched the original system). Considering that BART dates back to the 70s, DOS and Windows 98 were upgrades. Meanwhile Muni soldiered on with OS/2 for nearly a couple decades.
  - peteradio 4y ago
    > I think most folks here are drastically underestimating the complexity involved in getting a train control system right The phrase "butt-puckering" comes to mind. Ya some whizkid could come along and put something together but they'd be foolish to attempt, there is 1000 atm of pressure to not get one damn thing wrong. All the things currently wrong with the system are grandfathered. One single wrong thing in your new system and you are toasted.
    - inferiorhuman 4y ago
      Sure there are safety considerations (and plenty of dramatic safety events from BART), but I'd venture to guess even there it's rarely quite so black and white. I was actually thinking more of day-to-day nuisance problems than anything else. Things like getting stuck at the station while the train's computer gizmos reboot, stop annunciators getting confused (or out of sync with each other), having to position the train manually within a station. Or even software failures as hardware wears out – shit like that almost always fails in a safe manner. Unfortunately even a safe failure can have massive consequences which raises the cost of failing fast and breaking thing often.
parker_mountain 4y ago
> that sniffing the protocol would be fairly simple ah yes, "fairly simple". The protocol is not the issue, the data within is. There are hundreds of datapoints, various switching logic, and tons more of 50 years of accumulated cruft. At least. All in a well tested, stable, and understood package. The documentation is probably pretty good but likely has small (but crucial) errors. It's easy to assume that this would be a simple rewrite, but real engineering (not the kind we do on hacker news! lol) requires tons of planning, testing, and phasing in. This is dealing directly with people's safety after all! It's interfacing with a train control system, not a webapp that turns markdown into memes. also > RS-232 is my guess (could be the parallel port) RS-232 is inherently a serial protocol. If you're using RS-232 since basically ever, it's either a built in serial port, some sort of serial headers, or some sort of usb-to-serial adapter. The parallel port has never been directly used for RS-232 - they are fundamentally different technologies.
- midoridensha 4y ago
  In 1990, RS-232 used 25-pin ports (DB25), and the DB9 ports were "new". Why they used 25-pin ports for RS-232 I'm not sure, since only a fraction of the lines (less than 9) were actually used, probably some silly historical reason. 25-to-9-pin adapters were very common then. The DB25 connector was also used for parallel ports as you said, which caused confusion sometimes. But on a standard PC, the printer port (parallel) was a 25-pin female port, whereas the serial ports (25 or 9 pin) were male.
  - parker_mountain 4y ago
    I guess around 1995 then? Whenever they switched to DB25 to DE9 for serial. > probably some silly historical reason original rs232 spec had a lot more signals and capabilities that very few consumer products implemented. So you were expected to have the full connector, even if the lines weren't hooked up.
    - midoridensha 4y ago
      >I guess around 1995 then? Whenever they switched to DB25 to DE9 for serial. There was a transition during the 90s where the DB25s were phased out and replaced with DE9s. Computers became more commonplace and popular, and they also became smaller, so the bulky 25-pin port fell out of favor.
origin_path 4y ago
If you read to the end of the article you see: "Finding replacement parts won’t be a problem and DOS is no longer necessary for Allen and his crew, but it will still be bittersweet to say goodbye to trains that they have lovingly kept alive." So they don't actually use DOS anymore, maybe, or possibly this just means they don't care because the new trains will soon replace the old ones and then DOS will be gone too. It's a poorly written article even allowing for the usual tech mistakes but either interpretation would be reasonable.
honkdaddy 4y ago
>Calling people from bay-area hacker spaces to give them a hand in creating something more modern. Tragically, from the small amount of time I've spent in SF, my understanding is this is unlikely to be their first instinct, even if it's a great idea. Doesn't the SF local government have a testy, if not outright hostile relationship with the tech worker/hacker/startup scene? I've met people from either side of the discussion who pretty much entirely blame the opposite group (hackers or city workers) for why SF is so different now than 'back in the day.'
- inferiorhuman 4y ago
  This is perhaps more specious than the "cops don't want to do their jobs because people are just too mean to them" song and dance. At the risk of putting too fine a point on it BART is not run as part of any city government, San Francisco or otherwise. You're pretty much tilting at windmills here. Besides this sort of thing is well beyond the scope of folks creating subscription based fruit juicers.
  - honkdaddy 4y ago
    Gotcha. BART being funded by the Bay Area Rapid Transit District is something I'd forgotten, and does make my point a lot less salient. It's been a long time since I was in SF, so it's possible my assessments here are way off base. The main observation I was trying to make is that the public/private sector has always seemed to have a harder time working hand-in-hand in the Bay Area than it does in other cities I've lived in, whatever that reason may be.
  - onetokeoverthe 4y ago
Tanoc 4y ago
The big problem it seems like is Windows Vista dropped legacy support drivers and 16-bit support fifteen years ago, and Windows 10 dropped even more legacy support drivers. So they keep the ThinkPads around because they can't find the drivers for some things anymore. I've run into that a few times, where old drivers were made specifically for DOS 7/8 or even DOS 6 to interface with custom fabricated CNC machines or injection mould machines. In the end the company that originally created those few hundred units no longer exists, and nobody has the floppies that held the original drivers or software. So you keep cloning the drives for when they fail and hoping there's enough near-spec hardware spares to hold out until a total system replacement.

by jalk 4y ago

RajT88 4y ago

Would agree with all of this. The problem is that BART doesn't employ talented reverse engineers. There is now way they could afford them in that area. Also: Talented reverse engineers are hard to find on a good day in any given area. I used to think that my mentors from the start of my tech career were just the baseline level of sophistication you'd expect of any decent software engineer. 20 years later I have seen how few people have the mindset to get into that sort of thing. (And it is a mindset much more than a level of intellect, IMO)

mh- 4y ago
I don't ride Bart anymore, but plenty of us would have done this work for free.. just to see a small chance at improving the system. The dysfunction is broader than them being unable to afford engineering talent.
- parker_mountain 4y ago
  > plenty of us would have done this work for free This is the interface to the train control system - a primary safety system - responsible for the lives of literally a million people on an average month. Are you prepared to spend months of your lives rigorously testing and phasing this system in? Are you prepared for the intense legal liability if something goes wrong?
  - mh- 4y ago
    Sorry, I should have been clearer. "The work" here referred to assisting with the reverse engineering aspect of this. I, and almost everyone else here, lack the domain expertise to be of any use in that.
    - deleted 4y ago
      [deleted]
  - gridspy 4y ago
    If an individual engineer is put in a position where they can endanger lives by accident, there is something wrong with the management and systems in place around that person. This is a team game. The people figuring out how the comms works and creating tests for those communications then hand their work off to the "rigor" team.
    - Doxin 4y ago
      Or do like what the dutch railways are doing: run the old and new systems in parallel on real inputs and check if the outputs match. Then proceed to only use the output for the old system until you're really really sure there's no bugs.
      - jabl 4y ago
        I don't work in safety-critical hard real-time systems, but isn't this the standard approach when upgrading such a system?
        Doxin 4y ago
        I'd imagine so, I just knew about the dutch railways for certain. IIRC they are doing a mechanized translation from COBOL to C++. The idea being to first test that exhaustively, and then slowly manually translate the COBOL-y-C++ to more idiomatic C++.
        4y ago
ChuckNorris89 4y ago
>Also: Talented reverse engineers are hard to find on a good day in any given area Not true. Have you looked in Europe? It's full of them thanks to the tinker culture and the presence all big anti-malware companies (Esset, Avast, Avira, Bit-Defender, Kaspersky, etc). Plus every single cybersecurity consulting company out there in the world, big and small, has guys who can do reverse engineering for you. There's thousands of these companies, just look them up. The reason you're probably not finding them is because you're probably not looking for them or that many juniors who don't have the "RE bug" don't stay in that gig professionally too long because, unless you're a zero-day finding God, cashing out six figure CVE bounties and giving Defcon talks, the pay is worse than a webdev doing CRUD since the demand for reverse engineering work is very low and not valued by customers as it's not the kind of job that impacts their sales or share price so they just look to offshore it to the lowest bidder. I also left the field and moving to cloud engineering since the demand, pay, benefits, respect is so much better.
- dagw 4y ago
  Not true. Have you looked in Europe? I wonder to what extent that is still the case. When I graduated 15ish years ago it was definitely true that many of my classmates had those low level skills, and serval of them had written their Masters thesis on something related. However looking at people graduating from the same program today, I hardly see anybody doing that sort of work. Everything is either some sort of high level web stuff or machine learning. Even the people I knew who had those skills haven't used them professionally in over decade since, as you say, the pay sucks.
  - ChuckNorris89 4y ago
    >I wonder to what extent that is still the case. It is. The university in my area has a famous master's program for cybersecurity and many grads there know reverse engineering to a degree. And there a re many such universities in Europe, plus all the Eastern European gray beards who work in the industry since they cut their teeth breaking DRM to pirate games in their youth. I personally know at least 5 people who can do this work professionally at a consulting company.
inferiorhuman 4y ago
The problem is that BART doesn't employ talented reverse engineers. There is now way they could afford them in that area. Well, no. Money is not the problem and BART doesn't need reverse engineers. They've already thrown away tens of millions of dollars on a replacement train control system (AATCS). Everyone's favorite defense contractor Raytheon was tasked with the project until they realized they couldn't make it work. Eventually Raytheon pawned the project off on Harmon, which was later bought by General Electric. GE shut the whole thing down because it was going nowhere. I think most folks here are drastically underestimating the complexity involved in getting a train control system right (and how badly BART botched the original system). Considering that BART dates back to the 70s, DOS and Windows 98 were upgrades. Meanwhile Muni soldiered on with OS/2 for nearly a couple decades.
- peteradio 4y ago
  > I think most folks here are drastically underestimating the complexity involved in getting a train control system right The phrase "butt-puckering" comes to mind. Ya some whizkid could come along and put something together but they'd be foolish to attempt, there is 1000 atm of pressure to not get one damn thing wrong. All the things currently wrong with the system are grandfathered. One single wrong thing in your new system and you are toasted.
  - inferiorhuman 4y ago
    Sure there are safety considerations (and plenty of dramatic safety events from BART), but I'd venture to guess even there it's rarely quite so black and white. I was actually thinking more of day-to-day nuisance problems than anything else. Things like getting stuck at the station while the train's computer gizmos reboot, stop annunciators getting confused (or out of sync with each other), having to position the train manually within a station. Or even software failures as hardware wears out – shit like that almost always fails in a safe manner. Unfortunately even a safe failure can have massive consequences which raises the cost of failing fast and breaking thing often.

parker_mountain 4y ago

> that sniffing the protocol would be fairly simple ah yes, "fairly simple". The protocol is not the issue, the data within is. There are hundreds of datapoints, various switching logic, and tons more of 50 years of accumulated cruft. At least. All in a well tested, stable, and understood package. The documentation is probably pretty good but likely has small (but crucial) errors. It's easy to assume that this would be a simple rewrite, but real engineering (not the kind we do on hacker news! lol) requires tons of planning, testing, and phasing in. This is dealing directly with people's safety after all! It's interfacing with a train control system, not a webapp that turns markdown into memes. also > RS-232 is my guess (could be the parallel port) RS-232 is inherently a serial protocol. If you're using RS-232 since basically ever, it's either a built in serial port, some sort of serial headers, or some sort of usb-to-serial adapter. The parallel port has never been directly used for RS-232 - they are fundamentally different technologies.

midoridensha 4y ago
In 1990, RS-232 used 25-pin ports (DB25), and the DB9 ports were "new". Why they used 25-pin ports for RS-232 I'm not sure, since only a fraction of the lines (less than 9) were actually used, probably some silly historical reason. 25-to-9-pin adapters were very common then. The DB25 connector was also used for parallel ports as you said, which caused confusion sometimes. But on a standard PC, the printer port (parallel) was a 25-pin female port, whereas the serial ports (25 or 9 pin) were male.
- parker_mountain 4y ago
  I guess around 1995 then? Whenever they switched to DB25 to DE9 for serial. > probably some silly historical reason original rs232 spec had a lot more signals and capabilities that very few consumer products implemented. So you were expected to have the full connector, even if the lines weren't hooked up.
  - midoridensha 4y ago
    >I guess around 1995 then? Whenever they switched to DB25 to DE9 for serial. There was a transition during the 90s where the DB25s were phased out and replaced with DE9s. Computers became more commonplace and popular, and they also became smaller, so the bulky 25-pin port fell out of favor.

origin_path 4y ago

If you read to the end of the article you see: "Finding replacement parts won’t be a problem and DOS is no longer necessary for Allen and his crew, but it will still be bittersweet to say goodbye to trains that they have lovingly kept alive." So they don't actually use DOS anymore, maybe, or possibly this just means they don't care because the new trains will soon replace the old ones and then DOS will be gone too. It's a poorly written article even allowing for the usual tech mistakes but either interpretation would be reasonable.

honkdaddy 4y ago

>Calling people from bay-area hacker spaces to give them a hand in creating something more modern. Tragically, from the small amount of time I've spent in SF, my understanding is this is unlikely to be their first instinct, even if it's a great idea. Doesn't the SF local government have a testy, if not outright hostile relationship with the tech worker/hacker/startup scene? I've met people from either side of the discussion who pretty much entirely blame the opposite group (hackers or city workers) for why SF is so different now than 'back in the day.'

inferiorhuman 4y ago
This is perhaps more specious than the "cops don't want to do their jobs because people are just too mean to them" song and dance. At the risk of putting too fine a point on it BART is not run as part of any city government, San Francisco or otherwise. You're pretty much tilting at windmills here. Besides this sort of thing is well beyond the scope of folks creating subscription based fruit juicers.
- honkdaddy 4y ago
  Gotcha. BART being funded by the Bay Area Rapid Transit District is something I'd forgotten, and does make my point a lot less salient. It's been a long time since I was in SF, so it's possible my assessments here are way off base. The main observation I was trying to make is that the public/private sector has always seemed to have a harder time working hand-in-hand in the Bay Area than it does in other cities I've lived in, whatever that reason may be.
- onetokeoverthe 4y ago

Tanoc 4y ago

The big problem it seems like is Windows Vista dropped legacy support drivers and 16-bit support fifteen years ago, and Windows 10 dropped even more legacy support drivers. So they keep the ThinkPads around because they can't find the drivers for some things anymore. I've run into that a few times, where old drivers were made specifically for DOS 7/8 or even DOS 6 to interface with custom fabricated CNC machines or injection mould machines. In the end the company that originally created those few hundred units no longer exists, and nobody has the floppies that held the original drivers or software. So you keep cloning the drives for when they fail and hoping there's enough near-spec hardware spares to hold out until a total system replacement.