3 ms·
I worked on getting an address database into elasticsearch years ago when it was still using modified tf-idf. Customers wanted FTS where a lot of the queries wo
by josh_p 1mo ago
I worked on getting an address database into elasticsearch years ago when it was still using modified tf-idf. Customers wanted FTS where a lot of the queries would be something like "100 First Ave, NY" or "200 2nd St, MN".
It was one of the most fun projects I've worked on in my career so far. I got a learn a lot about how US and international addresses worked, so many edge cases, and got to really understand how customers were using the existing search to make sure they weren't adding any duplicates to the database. Token filters and synonyms were neat and figuring out the right indexing strategy was a lot of fun.
It was a lot more work to get it right for most of the use-cases our customers had than just "throw it into ES and be done". That would probably have been fine for the 80/20 case, like you said, but I agree that the bulk of the work is going to be fine-tuning the search solution, whatever technology you're using.
- oever 1mo agoWhat's your opinion on nominatim? I find that it gives up quickly when there's one or two typos in an address. It nails your examples.
- josh_p 29d agoI didn't know this was a thing until you mentioned it. I'm glad there are more options today. I have since left the company and haven't gotten to work in address-related problems since. We were pretty invested in using Google at the time for their Geocoding API. The questions that needed answering for our use-case were: - Does this place actually exist? - Can it be found on Google Maps? - Do we (the company/researchers) have an entry for this place already?
- chorizo 1mo agoWhy not use what USPS Addresses API? That what it’s there for. It will validate and standardize addresses including ZIP+4. Wonder if other countries have similar API’s.
- polishdude20 1mo agoTheir API doesn't allow bulk conversions.
- chorizo 1mo agoBelieve they have a paid access tier with much higher rate limits.
- josh_p 29d agoI honestly can't remember. I don't think it was an option for our company at the time, or I was too inexperienced to evaluate that as a solution. As fun as it was, I was also flying by the seat of my pants there.