3 ms·

I think fundamentally, no parsing/normalizing library can be effective for addresses. A much better approach is to have a search library which finds the address

I think fundamentally, no parsing/normalizing library can be effective for addresses. A much better approach is to have a search library which finds the address you're looking for within a dataset of all the addresses in the world. Addresses are fundamentally unstructured data. You can't validate them structurally. It's trivial to create nonexistent addresses which any parsing library will parse just fine. On the flipside, there's enough variety in real addresses that your parser has to be extremely tolerant in what it accepts--so tolerant that it basically tolerates everything. The entire purpose of a parser for addresses is to reject invalid addresses, so if your parser tolerates everything it's pointless. The only validation that makes any sense is "does this address exist in the real world?". And the way to do that is not parsing, it's by comparing to a dataset of all the addresses in the world. I haven't evaluated this project enough to understand confidently what they're doing, but I hope they're approaching this as a search engine for address datasets, and not as a parsing/normalizing library.

vidarh 1y ago
And keeping such datasets up to date is another matter entirely, because clearly a lot of companies rely datasets that were outdated before their company even existed. A trivially simple example of just how messy this is when people try to constrain it is that it's nearly random whether or not a given carrier would insist on me giving an incorrect address for my previous place, seemingly because traditionally and prior to 1965 the address was in Surrey, England. The "postcode area name" for my old house is Croydon, and Croydon has legally been in London since 1965, and was allocated it's own postcode area in 1966. "Surrey" hasn't been correct for addresses in Croydon since then. But at least one delivery company insisted my old address was invalid unless I changed the town/postcode area to "Surrey", and refused to even attempt a delivery. Never mind they had my house number and postcode, which was sufficient to uniquely identify my house.
- kerkeslager 1y ago
  Agreed. Keeping an up-to-date dataset of addresses is enormously hard. It's impossible to do perfectly, and only a few companies are capable of doing it passably, while the rest of us have no choice but to buy from them. But notably, to validate a parser/normalizer, you need this dataset anyway, so creating a parser/normalizer isn't even saving you that work. It's just giving you a worse result for more work.
derdi 1y ago
> real world [...] dataset You are equating two things that are not equatable.
- kerkeslager 1y ago
  I don't think I am. And despite your mangling what I said beyond recognition, even your quotation of me doesn't make it look like I am.
  - derdi 1y ago
    I know that you are equating things that are not equatable, since I have personally been affected by businesses relying on "datasets" to claim that my real world address, which definitely existed, did not exist. Data is not the same as reality.
    - kerkeslager 1y ago
      > I know that you are equating things that are not equatable, since I have personally been affected by businesses relying on "datasets" to claim that my real world address, which definitely existed, did not exist. It sounds like people at those businesses equated a dataset to the real world, not me. You're an adult, direct your frustrations appropriately. > Data is not the same as reality. That glosses over a lot of nuance. Obviously, no dataset perfectly represents reality. But, this fact is often used to dismiss data entirely, resulting in people making decisions with absolutely no evidence whatsoever. An appropriate use of an address database might be: when the user enters an address not in the database, do a fuzzy search and suggest the best match you can find, asking "Did you mean X?" At that point, if the user says, "No, I really meant what I put in," then you accept the data they gave you. This catches most mistakes while allowing users to put in addresses that aren't in your dataset.
      - derdi 1y ago
        That last suggestion makes a lot of sense. It makes a lot of sense specifically because it is the opposite of what you suggested above: > The entire purpose of a parser for addresses is to reject invalid addresses, so if your parser tolerates everything it's pointless.

vidarh 1y ago

And keeping such datasets up to date is another matter entirely, because clearly a lot of companies rely datasets that were outdated before their company even existed. A trivially simple example of just how messy this is when people try to constrain it is that it's nearly random whether or not a given carrier would insist on me giving an incorrect address for my previous place, seemingly because traditionally and prior to 1965 the address was in Surrey, England. The "postcode area name" for my old house is Croydon, and Croydon has legally been in London since 1965, and was allocated it's own postcode area in 1966. "Surrey" hasn't been correct for addresses in Croydon since then. But at least one delivery company insisted my old address was invalid unless I changed the town/postcode area to "Surrey", and refused to even attempt a delivery. Never mind they had my house number and postcode, which was sufficient to uniquely identify my house.

kerkeslager 1y ago
Agreed. Keeping an up-to-date dataset of addresses is enormously hard. It's impossible to do perfectly, and only a few companies are capable of doing it passably, while the rest of us have no choice but to buy from them. But notably, to validate a parser/normalizer, you need this dataset anyway, so creating a parser/normalizer isn't even saving you that work. It's just giving you a worse result for more work.

derdi 1y ago

> real world [...] dataset You are equating two things that are not equatable.

kerkeslager 1y ago
I don't think I am. And despite your mangling what I said beyond recognition, even your quotation of me doesn't make it look like I am.
- derdi 1y ago
  I know that you are equating things that are not equatable, since I have personally been affected by businesses relying on "datasets" to claim that my real world address, which definitely existed, did not exist. Data is not the same as reality.
  - kerkeslager 1y ago
    > I know that you are equating things that are not equatable, since I have personally been affected by businesses relying on "datasets" to claim that my real world address, which definitely existed, did not exist. It sounds like people at those businesses equated a dataset to the real world, not me. You're an adult, direct your frustrations appropriately. > Data is not the same as reality. That glosses over a lot of nuance. Obviously, no dataset perfectly represents reality. But, this fact is often used to dismiss data entirely, resulting in people making decisions with absolutely no evidence whatsoever. An appropriate use of an address database might be: when the user enters an address not in the database, do a fuzzy search and suggest the best match you can find, asking "Did you mean X?" At that point, if the user says, "No, I really meant what I put in," then you accept the data they gave you. This catches most mistakes while allowing users to put in addresses that aren't in your dataset.
    - derdi 1y ago
      That last suggestion makes a lot of sense. It makes a lot of sense specifically because it is the opposite of what you suggested above: > The entire purpose of a parser for addresses is to reject invalid addresses, so if your parser tolerates everything it's pointless.