3 ms·
That's what using difflib does. Plaza vs pl vs place is just one example of many, though. There are about 1k different street names in chicago and about 18k dif
by bpchaps 10y ago
That's what using difflib does. Plaza vs pl vs place is just one example of many, though. There are about 1k different street names in chicago and about 18k different street names in the data. Many are missing street type and there are many street names which have different street types. So that tends to open up cans of worms. That, and street names that have 'street' in their names. Also, 'avenue' is a street name which screws things up left and right.
It's not a simple problem, and none of the popular machine learning libraries which use string distance for this sort of problem have been very effective.
- maxerickson 10y agoHave you tried https://github.com/openvenues/libpostal https://github.com/openvenues/libpostal?
- bpchaps 10y agoYep - it's actually been pretty invaluable in getting this project to where it is now. :) It definitely has its own flaws, but those are easy to work around. Edit - looks like they have a similar problem from dealing with hand typed data. From their road map: Ambiguous token classification (coming soon): e.g. "dr" => "doctor" or "drive" for an English address depending on the context. Multiclass logistic regression trained on OSM addresses, where abbreviations are discouraged, giving us many examples of fully qualified addresses on which to train