4 ms·

If you can tell me where to get train timetable information for Australia, I can try! I'll definitely be in Taiwan until the end of 2018. After that, I might b

If you can tell me where to get train timetable information for Australia, I can try! I'll definitely be in Taiwan until the end of 2018. After that, I might be moving somewhere else (possibly New Zealand, Australia, or Canada). Depending on my future job's policies about side work, then I might be able to redraw a map like this one for wherever I'm living in future. Another possibility is to clean up my GUI, write a tutorial, and use it as a lesson plan for kids. I'd need to talk to some teachers first though.

exikyut 9y ago
Ah, a well-researched answer. I'm genuinely curious how much digging was needed to find my location. It's not on page 1 of my profile, at least. (I like doing the same kind of digging... but then people are like "how did you find this old email address?!" and I can't win.) I actually wound up having a look through your blog after your comment on the SD card factory (https://news.ycombinator.com/item?id=15890198 https://news.ycombinator.com/item?id=15890198) which was super interesting as I'm VERY fascinated with chip design at the moment. Would love to see how the photolithography machines work... As for mapping data... [Clarifying disclaimer: the following wall of text conveys no imposition you have to actually do this nor expectation that you will. This wall of text just wants to be helpful. :P] Australia doesn't have a countrywide system; each state manages its own public transport infrastructure. I live in NSW (very possibly where you'd end up visiting if you came here looking for a job, as a FWIW aside), and this state happens to provide full info through https://opendata.transport.nsw.gov.au/ https://opendata.transport.nsw.gov.au/. This portal is... to be honest: on the one hand, it's an utterly confusing mess you could very reasonably and accurately sell as a technical IQ test, it's structured so confusingly. On the other hand, if you succeed at getting an OAuth2 key and figuring out the right URL to poke, it'll spit back little ~20-40KB protobuf-compressed blob files containing the live GPS coordinates for every train, bus, tram and ferry in NSW. And on top of that there's stuff like route shape information, realtime timetable data, etc. So, figuring the site out will be a multi-day project of its own. As far as I can figure out, you basically need a list of stops and the locations. Methinks the GTFS Static feed for stop information may be a workable start: https://opendata.transport.nsw.gov.au/node/334/exploreapi https://opendata.transport.nsw.gov.au/node/334/exploreapi To get this, I a) logged in, b) made an app (just enable all the APIs), c) went to the above page, left "type" as the default "API key", hit "expand", scrolled down to the "try it!" button, opened the network tab in the devtools, clicked the "try it" button, right-clicked the in-flight download, went to "copy" > "copy as curl", reloaded the page to stop the download, then just ran the curl command. Yep, a giant hack, but it worked. The site's basically broken, but it works.™ The result was a 100MB ZIP file (remember to add "-o gtfs.zip" or something to the curl command) with a bunch of CSV text files in it that conform to this format: https://developers.google.com/transit/gtfs/examples/gtfs-feed https://developers.google.com/transit/gtfs/examples/gtfs-fee... The above URL may be helpful for basic familiarization but I had a look at the data from the standpoint of "how would I turn this into a proportionally accurate map". stops.txt has all the train stations buried in it. I particularly like that each platform at a station is listed separately, each with its own (ever so slightly different) GPS coordinate :D This is the second approach to look at everything that occurred to me, but I've kept the first one below. Each station is noted in stops.txt both as an actual station as well as a series of platforms. (I'm guessing the GPS coordinates in the station reference are for the actual building.) The actual station entry IDs seem to all start with PST (the IDs are allowed to be random FWIW). The platform entries are listed as "___ Station, Platform _" and have a numeric ID, and you'll see the PST___ identifier in the "parent" column off toward the right. If you grab the ID for a station platform, you'll find a ton of references to the ID in stop_times.txt, which represent a train stopping at the platform as part of a route. This is useful because of the "shape_dist_traveled" column, which records how far the train traveled as part of the route it happens to be on. If you crossreference all the platforms against all the routes in stop_times.txt, you'll be able to derive the distance between each stop by just looking at the route as it travels between stations. -- This was where I started: the routes are structured by startpoint and an endpoint. If you ^F for "Train" inside routes.txt to find a train route and grab the ID, and you'll find a ton of matches for that ID inside shapes.txt. This describes every single little twist and turn in the track as a discrete GPS coordinate. You may find it fun to feed this to your Logo implementation - but you don't _have_ to, thanks to the "dist_traveled" column (which is not a standard GDFS column but got added here). Searching trips.txt for the same ID will yield a ton of matches that represent all the stations associated with a particular route. (The GPS locations for the station platforms are of course back in stops.txt.) Each route is distinguished by a unique ID which explains the extra characters. I don't think there's a pattern to these. -- I unfortunately don't know whether the ___dist_measured columns measure the distance in feet, miles, meters, kilometers, or what. It might be meters. You could probably derive it from a pair of GPS positions (I have no idea how to). I don't mind seeing if I can figure out the distance information between the stops. It sounds like a fun project, in theory... :P
- peterburkimsher 9y ago
  Wow! Thank you for all that. I stalked you via the data URI on your profile to get your email address, then I Googled that to find your DeviantArt, which had your location. I'll look in the details of NSW transport when I get back to my computer. It sounds complicated, but similar to the way I derived the Kaohsiung bus network from the real-time GPS coordinates. I didn't make a map for that yet though, just KML files.
  - exikyut 9y ago
    It was fun to figure it out! The offer to headscratch through the actual data stands :) Ah, deviantart, haven't seriously used that for a few years :) heh It unfortunately sounds complicated-ish to me too. Interesting to hear that real-time route data is a worldwide thing nowadays. It makes sense, really, now that M2M over 3G/4G is pretty widespread.

exikyut 9y ago

Ah, a well-researched answer. I'm genuinely curious how much digging was needed to find my location. It's not on page 1 of my profile, at least. (I like doing the same kind of digging... but then people are like "how did you find this old email address?!" and I can't win.) I actually wound up having a look through your blog after your comment on the SD card factory (https://news.ycombinator.com/item?id=15890198 https://news.ycombinator.com/item?id=15890198) which was super interesting as I'm VERY fascinated with chip design at the moment. Would love to see how the photolithography machines work... As for mapping data... [Clarifying disclaimer: the following wall of text conveys no imposition you have to actually do this nor expectation that you will. This wall of text just wants to be helpful. :P] Australia doesn't have a countrywide system; each state manages its own public transport infrastructure. I live in NSW (very possibly where you'd end up visiting if you came here looking for a job, as a FWIW aside), and this state happens to provide full info through https://opendata.transport.nsw.gov.au/ https://opendata.transport.nsw.gov.au/. This portal is... to be honest: on the one hand, it's an utterly confusing mess you could very reasonably and accurately sell as a technical IQ test, it's structured so confusingly. On the other hand, if you succeed at getting an OAuth2 key and figuring out the right URL to poke, it'll spit back little ~20-40KB protobuf-compressed blob files containing the live GPS coordinates for every train, bus, tram and ferry in NSW. And on top of that there's stuff like route shape information, realtime timetable data, etc. So, figuring the site out will be a multi-day project of its own. As far as I can figure out, you basically need a list of stops and the locations. Methinks the GTFS Static feed for stop information may be a workable start: https://opendata.transport.nsw.gov.au/node/334/exploreapi https://opendata.transport.nsw.gov.au/node/334/exploreapi To get this, I a) logged in, b) made an app (just enable all the APIs), c) went to the above page, left "type" as the default "API key", hit "expand", scrolled down to the "try it!" button, opened the network tab in the devtools, clicked the "try it" button, right-clicked the in-flight download, went to "copy" > "copy as curl", reloaded the page to stop the download, then just ran the curl command. Yep, a giant hack, but it worked. The site's basically broken, but it works.™ The result was a 100MB ZIP file (remember to add "-o gtfs.zip" or something to the curl command) with a bunch of CSV text files in it that conform to this format: https://developers.google.com/transit/gtfs/examples/gtfs-feed https://developers.google.com/transit/gtfs/examples/gtfs-fee... The above URL may be helpful for basic familiarization but I had a look at the data from the standpoint of "how would I turn this into a proportionally accurate map". stops.txt has all the train stations buried in it. I particularly like that each platform at a station is listed separately, each with its own (ever so slightly different) GPS coordinate :D This is the second approach to look at everything that occurred to me, but I've kept the first one below. Each station is noted in stops.txt both as an actual station as well as a series of platforms. (I'm guessing the GPS coordinates in the station reference are for the actual building.) The actual station entry IDs seem to all start with PST (the IDs are allowed to be random FWIW). The platform entries are listed as "___ Station, Platform _" and have a numeric ID, and you'll see the PST___ identifier in the "parent" column off toward the right. If you grab the ID for a station platform, you'll find a ton of references to the ID in stop_times.txt, which represent a train stopping at the platform as part of a route. This is useful because of the "shape_dist_traveled" column, which records how far the train traveled as part of the route it happens to be on. If you crossreference all the platforms against all the routes in stop_times.txt, you'll be able to derive the distance between each stop by just looking at the route as it travels between stations. -- This was where I started: the routes are structured by startpoint and an endpoint. If you ^F for "Train" inside routes.txt to find a train route and grab the ID, and you'll find a ton of matches for that ID inside shapes.txt. This describes every single little twist and turn in the track as a discrete GPS coordinate. You may find it fun to feed this to your Logo implementation - but you don't _have_ to, thanks to the "dist_traveled" column (which is not a standard GDFS column but got added here). Searching trips.txt for the same ID will yield a ton of matches that represent all the stations associated with a particular route. (The GPS locations for the station platforms are of course back in stops.txt.) Each route is distinguished by a unique ID which explains the extra characters. I don't think there's a pattern to these. -- I unfortunately don't know whether the ___dist_measured columns measure the distance in feet, miles, meters, kilometers, or what. It might be meters. You could probably derive it from a pair of GPS positions (I have no idea how to). I don't mind seeing if I can figure out the distance information between the stops. It sounds like a fun project, in theory... :P

peterburkimsher 9y ago
Wow! Thank you for all that. I stalked you via the data URI on your profile to get your email address, then I Googled that to find your DeviantArt, which had your location. I'll look in the details of NSW transport when I get back to my computer. It sounds complicated, but similar to the way I derived the Kaohsiung bus network from the real-time GPS coordinates. I didn't make a map for that yet though, just KML files.
- exikyut 9y ago
  It was fun to figure it out! The offer to headscratch through the actual data stands :) Ah, deviantart, haven't seriously used that for a few years :) heh It unfortunately sounds complicated-ish to me too. Interesting to hear that real-time route data is a worldwide thing nowadays. It makes sense, really, now that M2M over 3G/4G is pretty widespread.