6 ms·

I've been trying to crack the encryption on this for a while. I want to make a free public train status api (I built amtrak.io). If anyone would like to help im

I've been trying to crack the encryption on this for a while. I want to make a free public train status api (I built amtrak.io). If anyone would like to help improve public open data around Amtrak, please contact me. Been also digging into making a PDF parser for their very complicated schedule files, not that they follow it anyway :)

thetwentyone 8y ago
I don't have the capacity to do this, but I tried a few years ago: I no longer am in a situation where I need tracking, but a couple of years ago I had the need for tracking my daily commuter train. I had to resort to scraping the website to find the status. If it was late, I had a push notification sent to my phone (a combination of a free app engine deployment and pushbullet). I tried to get integration via Tasker to set my alarm back if the train was more than [x] minutes, but never got to that point. I really hope that you make some progress on this, though I wish you didn't have to and Amtrak shared the data.
jonknee 8y ago
A comment in the source code seems to describe what you need to do: MasterSegment is the length of the string at the end of the encrypted data that contains the secret key To decrypt - we do the following 1. Take masterSegment (88) length - from the right of the data - this has the private key 2. Everything from 0 to the end - master segment is the raw data - that needs to be decrypted 3. Decrypt the 88 characters using the public key - that will give you a pipe separated string of the private key (random guid from MDS) and a time stamp (to scramble it) 4. Now use the private key and decrypt the data stored from step 2. 5. Parse the decrypted data - and rejoice 6. KSUE -means key issue 7. __$$_jmd - the public key that we obtain __$_s1 : variable name for the Security object in the Helper.js file - that file has the calls to actually decrypt the data.
- iancarroll 8y ago
  This is insane. How much time did they put in to this pointless obfuscation, and then why would they go and document it all?
  - jonknee 8y ago
    I would guess "management", probably under the name of Homeland Security. It seems to be some spaghetti code of loops/callbacks written by a frustrated jQuery developer. Here are some more fun tidbits: > /MasterZoom is the sum of the zoom levels from the routes_list.json file. That is the index in the routesList.v.json -> arr array where we have the public key stored. IF THE ROUTES_LIST CHANGES, REMEMBER TO CHANGE THE INDEX TO BE CORRECT / > /Salt Value - the element is at the 8th position. So we can essentially pick any number from 0-100 (length of the s array in the file), get the length of the element, and then go to that index the following funky looking code will evaluate to 8. Salt has a length of 8 / > /Initialization Vector Value - the element is at the 32th position. So we can essentially pick any number from 0-100 (length of the IV array in the file), get the length of the element, and then go to that index the following funky looking code will evaluate to 32 - IV has a length of 32 / And my favorite: > masterSegment - length of data to be extracted from the encrypted response - 55 is just a fake //FAKE VARIABLES to throw off people hahahahaha
    - AdamJacobMuller 8y ago
      "Homeland Security"... and i'm on another list. Great.
  - kingbirdy 8y ago
    I'd imagine it's something like company policy says it has to obfuscated, but the guy actually programming the site realizes it's pointless and fellow hackers could make good use of the data.
  - gsnedders 8y ago
    Alternatively, all the comments were meant to be stripped out in some build stage.
  - chris0x00 8y ago
    IANAL and things may have changed, but it was my understanding that having a copy protection mechanism, even a simple one, gives them better claim to DMCA protection of their content should someone do something they'd like to discourage. "Your honor they read our source code and stole our private key to break the encryption of our protected data" sounds better than "they downloaded the data we gave them"
- AdamJacobMuller 8y ago
  // cw: FORCE the bloody stations to show at the initial zoom level. If this is not done, there is // a possibility that stations will not show in the initial map load. // I crib this shamelessly from SO...because it is so useful!! // http://stackoverflow.com/questions/3125065/how-do-i-limit-panning-in-google-maps-api-v3 /* __$$_jmd - public key masterSegment - length of data to be extracted from the encrypted response - 55 is just a fake //FAKE VARIABLES to throw off people hahahahaha __$_s - salt value __$_v - iv vale */ This code is hilarious.
maxerickson 8y ago
It's not really "open" data in the sense people expect if a letter or change in the scraped implementation results in it vanishing.
gregsadetsky 8y ago
Alternative to cracking the (useless, goofy, etc.) encryption: scrape the data using an automated browser. Open https://www.amtrak.com/services/maps.trainlocation.html https://www.amtrak.com/services/maps.trainlocation.html with a headless Chrome instance and get all of the data from the global `resolver` JS variable (see: `resolver.routes`, `resolver.trains`, etc.)
AdamJacobMuller 8y ago
I'm not a javascript guy so I'm not going to take this further, but, if you run the following inside the console of the page it loads and decrypts the station data: $.get(_$_sal_5,{ }) .done(function(d){ var x12 = d.trim(); console.log(JSON.parse(__$_s1._$_dcrt(x12.substring(0,x12.length-masterSegment),__$_s1._$_dcrt(x12.substr(x12.length - masterSegment),__$$_jmd).split('|')[masterSegment-88])) )}) I'm sure someone else can take thus further, but, its a mess of javascript. I think its hand-rolled encryption too from a cursory look at the __$_s1._$_dcrt function
gresrun 8y ago
Looks like they're using CryptoJS[0] based on some symbol names in the source[1]. [0]: https://github.com/bianjiang/crypto-js-npm/blob/1762940b9eef076dd20cfab149c9f97975d4b7d4/original/crypto-js-read-only/build/components/cipher-core.js#L850 https://github.com/bianjiang/crypto-js-npm/blob/1762940b9eef... [1]: https://maps.amtrak.com/rttl/js/AS.js https://maps.amtrak.com/rttl/js/AS.js
mef 8y ago
decryption walkthrough https://md.hecke.rs/m87XCW1ZT6i4GaVW--eaog https://md.hecke.rs/m87XCW1ZT6i4GaVW--eaog

thetwentyone 8y ago

I don't have the capacity to do this, but I tried a few years ago: I no longer am in a situation where I need tracking, but a couple of years ago I had the need for tracking my daily commuter train. I had to resort to scraping the website to find the status. If it was late, I had a push notification sent to my phone (a combination of a free app engine deployment and pushbullet). I tried to get integration via Tasker to set my alarm back if the train was more than [x] minutes, but never got to that point. I really hope that you make some progress on this, though I wish you didn't have to and Amtrak shared the data.

jonknee 8y ago

A comment in the source code seems to describe what you need to do: MasterSegment is the length of the string at the end of the encrypted data that contains the secret key To decrypt - we do the following 1. Take masterSegment (88) length - from the right of the data - this has the private key 2. Everything from 0 to the end - master segment is the raw data - that needs to be decrypted 3. Decrypt the 88 characters using the public key - that will give you a pipe separated string of the private key (random guid from MDS) and a time stamp (to scramble it) 4. Now use the private key and decrypt the data stored from step 2. 5. Parse the decrypted data - and rejoice 6. KSUE -means key issue 7. __$$_jmd - the public key that we obtain __$_s1 : variable name for the Security object in the Helper.js file - that file has the calls to actually decrypt the data.

iancarroll 8y ago
This is insane. How much time did they put in to this pointless obfuscation, and then why would they go and document it all?
- jonknee 8y ago
  I would guess "management", probably under the name of Homeland Security. It seems to be some spaghetti code of loops/callbacks written by a frustrated jQuery developer. Here are some more fun tidbits: > /MasterZoom is the sum of the zoom levels from the routes_list.json file. That is the index in the routesList.v.json -> arr array where we have the public key stored. IF THE ROUTES_LIST CHANGES, REMEMBER TO CHANGE THE INDEX TO BE CORRECT / > /Salt Value - the element is at the 8th position. So we can essentially pick any number from 0-100 (length of the s array in the file), get the length of the element, and then go to that index the following funky looking code will evaluate to 8. Salt has a length of 8 / > /Initialization Vector Value - the element is at the 32th position. So we can essentially pick any number from 0-100 (length of the IV array in the file), get the length of the element, and then go to that index the following funky looking code will evaluate to 32 - IV has a length of 32 / And my favorite: > masterSegment - length of data to be extracted from the encrypted response - 55 is just a fake //FAKE VARIABLES to throw off people hahahahaha
  - AdamJacobMuller 8y ago
    "Homeland Security"... and i'm on another list. Great.
- kingbirdy 8y ago
  I'd imagine it's something like company policy says it has to obfuscated, but the guy actually programming the site realizes it's pointless and fellow hackers could make good use of the data.
- gsnedders 8y ago
  Alternatively, all the comments were meant to be stripped out in some build stage.
- chris0x00 8y ago
  IANAL and things may have changed, but it was my understanding that having a copy protection mechanism, even a simple one, gives them better claim to DMCA protection of their content should someone do something they'd like to discourage. "Your honor they read our source code and stole our private key to break the encryption of our protected data" sounds better than "they downloaded the data we gave them"
AdamJacobMuller 8y ago
// cw: FORCE the bloody stations to show at the initial zoom level. If this is not done, there is // a possibility that stations will not show in the initial map load. // I crib this shamelessly from SO...because it is so useful!! // http://stackoverflow.com/questions/3125065/how-do-i-limit-panning-in-google-maps-api-v3 /* __$$_jmd - public key masterSegment - length of data to be extracted from the encrypted response - 55 is just a fake //FAKE VARIABLES to throw off people hahahahaha __$_s - salt value __$_v - iv vale */ This code is hilarious.

maxerickson 8y ago

It's not really "open" data in the sense people expect if a letter or change in the scraped implementation results in it vanishing.

gregsadetsky 8y ago

Alternative to cracking the (useless, goofy, etc.) encryption: scrape the data using an automated browser. Open https://www.amtrak.com/services/maps.trainlocation.html https://www.amtrak.com/services/maps.trainlocation.html with a headless Chrome instance and get all of the data from the global `resolver` JS variable (see: `resolver.routes`, `resolver.trains`, etc.)

AdamJacobMuller 8y ago

I'm not a javascript guy so I'm not going to take this further, but, if you run the following inside the console of the page it loads and decrypts the station data: $.get(_$_sal_5,{ }) .done(function(d){ var x12 = d.trim(); console.log(JSON.parse(__$_s1._$_dcrt(x12.substring(0,x12.length-masterSegment),__$_s1._$_dcrt(x12.substr(x12.length - masterSegment),__$$_jmd).split('|')[masterSegment-88])) )}) I'm sure someone else can take thus further, but, its a mess of javascript. I think its hand-rolled encryption too from a cursory look at the __$_s1._$_dcrt function

gresrun 8y ago

Looks like they're using CryptoJS[0] based on some symbol names in the source[1]. [0]: https://github.com/bianjiang/crypto-js-npm/blob/1762940b9eef076dd20cfab149c9f97975d4b7d4/original/crypto-js-read-only/build/components/cipher-core.js#L850 https://github.com/bianjiang/crypto-js-npm/blob/1762940b9eef... [1]: https://maps.amtrak.com/rttl/js/AS.js https://maps.amtrak.com/rttl/js/AS.js

mef 8y ago

decryption walkthrough https://md.hecke.rs/m87XCW1ZT6i4GaVW--eaog https://md.hecke.rs/m87XCW1ZT6i4GaVW--eaog