5 ms·
I used to have a job where I analyzed global ship tracking data (AIS and other less-open (i.e. more classified) data streams) for law enforcement and environmen
by tony_cannistra 7y ago
I used to have a job where I analyzed global ship tracking data (AIS and other less-open (i.e. more classified) data streams) for law enforcement and environmental conservation purposes, and learned one thing: man is AIS data a bitch.
This is especially true when you're dealing with potential bad actors who can spoof their transmissions, combined with incredibly poor vessel metadata databases and strange satellite AIS coverage issues.
If there are any data scientists out there looking for a real head-basher, start an AIS project.
- iooi 7y agoWhy do bad actors spoof transmissions? Smuggling?
- wpietri 7y agoI'm sure Tony knows more, but one I've heard about is fishing. It's not legal to just fish anything anywhere. Poachers have an incentive to avoid monitoring and enforcement.
- jandrewrogers 7y agoMany reasons, and a significant percentage of all AIS transmissions are dodgy. Smuggling is just one reason. Other common reasons include illegal fishing in protected waters, covert meetings (e.g. a couple yachts coming together in the middle of the ocean), spy ships by state actors trying to look like ordinary commercial traffic, and misrepresenting state of shipping assets for commercial leverage in contract negotiations.
- wpietri 7y agoI would love to read more about this. Any suggestions?
- jandrewrogers 7y agoThese are things you learn by doing in-depth analysis of AIS and related data sources. As far as I know, no one writes about it. Physical world data sources are full of interesting patterns and anomalies. Few people ever look because the data sources are challenging to work with and very different from Internet data, which is the only kind of data most people are used to.
- wpietri 7y agoWell at this point I have terabytes of AIS data from the last few years, so if anybody is looking to dig into this, I'm glad to share.
- tony_cannistra 7y agoMuch of this kind of thing is written up in pretty terribly-written documents that come from government/law enforcement. Here's [0] a report I helped to put together which circles around some of these issues. [0]: https://ocean.csis.org/spotlights/illuminating-the-south-china-seas-dark-fishing-fleets/ https://ocean.csis.org/spotlights/illuminating-the-south-chi...
- tony_cannistra 7y agoYes, these are all things we observed in our data.
- wpietri 7y agoThat sounds really interesting. And if anybody is doing an AIS project, feel free to get in touch, whether or not you're using my library. I have soaked up all this esoteric knowledge that I'm eager to pass on.