Oh, that’s my thread. It’s not written for the HN crowd; my followers on Twitter are a very different demographic. It’s written to help non-technical folks star

Oh, that’s my thread. It’s not written for the HN crowd; my followers on Twitter are a very different demographic. It’s written to help non-technical folks start to consider the possibilities of the technology that exists in their lives now. They’re just getting a carefully curated peek behind the curtain. The HN crowd knows what’s behind the curtain already. A couple things: there’s a lot more interesting stuff going on than I’m explaining in my thread. As syllogism noticed, I’m using spaCy for the NLP part of the analysis (and displaCy was the best visual way to explain that to non technical folks!). The approach I’m using to dereference relative dates could be useful for its own spaCy post on its NER, for example. Same with dereferencing last names to the full names they’re referring to. Those two things alone are immensely useful for journalists and data viz designers. Plus, I love the hell of spaCy. I’ve done a ton of interesting stuff with it that I should write about—like discovering sock puppet accounts by comparing simple grammar signatures (relative ratios of parts of speech used in their text, along with other features like where adverbs and prepositions are used). Two more things: the flight data is the sleeping dragon of that thread. Really. And I’ve abandoned the idea of “unredacting” with predictions. For some technical reasons, sure (the redactions, for example, actually displace or edit other text, so it’s impossible to know the correct box size)…but mostly for ethical concerns. It’s both cool and weird to see my thread posted here! I hope my actual technical post gets a shot when I’m finished with it.