4 ms·

your problem sounds like https://developers.google.com/places/place-id https://developers.google.com/places/place-id

wpietri 8y ago
Yes, except that I'm doing it for a class of places that isn't covered by any open dataset I've found. What I have is a lot of data on ship movements, which includes destination strings hand-entered by sailors (mostly merchant marine sailors). The data is messy, and many of the strings can be obscure. For example, consider this guide for Tokyo's ports: https://www.kaiho.mlit.go.jp/03kanku/h22houkaisei/sozai/guide3_e.pdf https://www.kaiho.mlit.go.jp/03kanku/h22houkaisei/sozai/guid... This one happens to be well documented, but most don't seem to be, and in any case many of the port labels in the movement data bear a somewhat hazy relationship to official labels. I could do it all manually, of course: look at the data, figure out where ships stop, collect all the labels they apply, build a regex pachinko machine. But I'm wondering if I can bootstrap my way to a good text location classifier.
- ethbro 8y ago
  There are two approaches I can think of, depending on your gut about your data set. If single or minimal-word phrases are enough (e.g. 'kws' 'kei' 'hei' 'keih' etc), something like word2vec. Except instead of training to predict distance between words, you're generating an embedding between words and ports. If more complex modeling is required (e.g. 'tokyo tuesday kei then chiba'), something like LSTM [1]. The lat-long data you'd want to squish into a closest-port training set via geospacial math. Haven't done anything in the space, but I'd imagine chewing through lat-long legs and computing closest-approach distances to various ports (that is, to canonical port lat-long). Optimized for computation and space. Probably compressed down into a "visit / not-visit" binary feature. Might take awhile, but the math seems straightforward for this bit. [1] http://colah.github.io/posts/2015-08-Understanding-LSTMs/ http://colah.github.io/posts/2015-08-Understanding-LSTMs/

Yes, except that I'm doing it for a class of places that isn't covered by any open dataset I've found. What I have is a lot of data on ship movements, which includes destination strings hand-entered by sailors (mostly merchant marine sailors). The data is messy, and many of the strings can be obscure. For example, consider this guide for Tokyo's ports: https://www.kaiho.mlit.go.jp/03kanku/h22houkaisei/sozai/guide3_e.pdf https://www.kaiho.mlit.go.jp/03kanku/h22houkaisei/sozai/guid... This one happens to be well documented, but most don't seem to be, and in any case many of the port labels in the movement data bear a somewhat hazy relationship to official labels. I could do it all manually, of course: look at the data, figure out where ships stop, collect all the labels they apply, build a regex pachinko machine. But I'm wondering if I can bootstrap my way to a good text location classifier.

ethbro 8y ago
There are two approaches I can think of, depending on your gut about your data set. If single or minimal-word phrases are enough (e.g. 'kws' 'kei' 'hei' 'keih' etc), something like word2vec. Except instead of training to predict distance between words, you're generating an embedding between words and ports. If more complex modeling is required (e.g. 'tokyo tuesday kei then chiba'), something like LSTM [1]. The lat-long data you'd want to squish into a closest-port training set via geospacial math. Haven't done anything in the space, but I'd imagine chewing through lat-long legs and computing closest-approach distances to various ports (that is, to canonical port lat-long). Optimized for computation and space. Probably compressed down into a "visit / not-visit" binary feature. Might take awhile, but the math seems straightforward for this bit. [1] http://colah.github.io/posts/2015-08-Understanding-LSTMs/ http://colah.github.io/posts/2015-08-Understanding-LSTMs/