3 ms·
For a while I've been running a Twitter bot that tracks the comings and goings of major ships. I did not expect it to become extremely topical!
by wpietri 7y ago
For a while I've been running a Twitter bot that tracks the comings and goings of major ships. I did not expect it to become extremely topical!
- gus_massa 7y agoCan you make a technical post about how you get the info or the arrivals? How you take the photos?[1] Anything unexpected running the bot? It is a real bot, or you just type the message? Problems with name collisions? Duplications when a ship get delayed and arrive the next day? [1] With the camera, from the shore. :) I guess there are a few technical details and anecdotes here.
- wpietri 7y agoHi! It is a real bot. All major ships transmit AIS [1] messages that can be received with a cheap antenna and receiver. I wrote a parser and tools [2] for the protocol. I have pals [3] with an office near the water [4]. That receiver is part of a data commons [5], giving me access to data on other ships. The bot then filters that feed (50m messages/day), looking for major ships passing through the Golden Gate [6]. When something goes through, it tweets an announcement. Photos are all contributed by fans. I've thought about setting up an automated camera, but that requires an actual good view, which is trickier to acquire. The big surprise for me was how much garbage data there is flowing through that system. Poorly maintained transmitters. Badly configured devices. And merchant marine sailors typing stuff into bridge systems definitely don't share my fussiness when it comes to getting data right. A lot of the code is just a huge set of regexs to turn what people type in the destination field into clean, consistent geographical names. (And if you scroll back some, it's definitely not perfect yet.) But since most of the data is intended to provide human-viewed radar-like displays, it works out fine in practice. Another thing I learned is that marine geography is just different than how landlubbers see the world. In the same way we see the land as full of detail and the water as a vague blue area on the map, they have a lot of specificity for the water and its edge. But documentation of their notion of geography is much less available. So I've had to do a lot of manual work to figure out what a sailor means by, say, SFO, and there's still more to go. Is that helpful? [1] https://en.wikipedia.org/wiki/Automatic_identification_system https://en.wikipedia.org/wiki/Automatic_identification_syste... [2] https://github.com/wpietri/simpleais https://github.com/wpietri/simpleais [3] http://longnow.org/ http://longnow.org/ [4] https://fortmason.org/ https://fortmason.org/ [5] http://www.aishub.net/stations/2161 http://www.aishub.net/stations/2161 [6] https://en.wikipedia.org/wiki/Golden_Gate https://en.wikipedia.org/wiki/Golden_Gate
- gus_massa 7y agoThanks for the details. This looks like a nice subject for a blog post with the technical details and some photos if you ever have time to write it. I think it will get some traction here, but I can't guaranty it. Anyway, I can guaranty at least one upvote.
- wpietri 7y agoThat's a good idea. Thanks!