23 ms·

We need to liberate the Postcode Address File

356 points by edward 2y ago

jimbob45 2y ago
I can respect the arguments for making it public but there are strong arguments also to raise a high barrier of entry to discourage abuse. Further, the fewer users of the list, they easier they are to police.
- xnorswap 2y ago
  It's a lookup between postcode and address, what is the abuse cases you're worried about?
  - lnxg33k1 2y ago
    Considering that in UK if you live in a building, the door next to you can have a different postcode, I wouldn't worry at all
    - xnorswap 2y ago
      If that weren't true, you'd have entire cities in the same postcode. There has to be a boundary somewhere.
      - nly 2y ago
        Odd numbered homes on one side of the street and evens on the other often have different postcodes
        willvarfar 2y ago
        Postcodes are about sorting mail to match the delivery rounds.
        ooklala 2y ago
        Many buildings also have their own postcode! (The second half of the postcode represents the 'delivery point' which is basically limited by the amount of post that the postman/woman can physically carry...)
      - lnxg33k1 2y ago
        Well, in Italy postcodes define city areas, and cities, for example for my city the main postcode is 80100, but my area is 80142, and it contains few buildings, so it's different from UK, UK was the first time I saw such specific postcodes, and I've lived also in Germany and Netherlands
        Muromec 2y ago
        Netherlands had a postcode per street
        lnxg33k1 2y ago
        Oh yeah, I remember being able to insert just postcode and street number in forms, but it's not as specific as UK, I think
    - ben_w 2y ago
      I've lived in a one bed apartment where the front and back doors had different postcodes. IIRC, the neighbours to one side in the same building had a third postcode for their front, but shared mine for the back.
  - pjc50 2y ago
    Crucially, it doesn't have people's names in it.
    - xnorswap 2y ago
      Indeed. If it's an issue that someone would know your address, then it's an issue that they would know your postcode. If it's an issue that someone would know your postcode, then it's an issue that they would know your address. I'm struggling to think of a scenario where you'd be fine with someone knowing one of those pieces of information without knowing the other. It's not therefore an issue that there's a lookup between the two. Indeed you can do it trivially with google maps, or the plenty of other services that expose this database through their operation. Any safety concerns aren't at the layer of translation between postcode and address, they're how someone tied either of those pieces of information to a given person.
- secretsatan 2y ago
  You miss the point that it was once freely accessible, and now it is not.
  - scraplab 2y ago
    I don’t believe it’s ever been accessible for free. It’s just that ownership has moved from the state to a private company and now it’s difficult to make it open.
- nottorp 2y ago
  Yeah, maybe you should pay a subscription to know your own post code...
- IneffablePigeon 2y ago
  What nonsense. Are you worried about physical spam mail? That ship has already sailed. I genuinely can’t think of any other abuse vector for a dataset like this.
- andrewjl 2y ago
  > Would open address data create privacy risks? No. Unlike opening up more sensitive datasets such as personal location, releasing address data - a list of the physical places recognised by the government - carries few new legal or ethical risks. Many other countries are doing this, including those with strong privacy regimes. Open address data could only create new risks if it were linked and used with other datasets, and these risks should be managed in that context. The harms created by the lack of access to address data are more pressing. https://static1.squarespace.com/static/5ee7a7d964aeed7e5c507900/t/655dd5d4d1c0f4729bca69c9/1700648428211/Briefing%3A+Address+data https://static1.squarespace.com/static/5ee7a7d964aeed7e5c507...
- mrweasel 2y ago
  How exactly would that be abused? Denmark have a website where you can enter any address, or an address close to where you want to be and then let you select the right house on a map. The same site will show you the owners, the purchase price the taxable value, size, number of bathrooms, stuff like that. I used it to find the address of a friend when I needed to ship him a present and I only roughly knew where he lives.
shakna 2y ago
Australia also has ours locked away privately. You can purchase access, but... You also need to sign a contract that you won't make the PDF, or anything you derive from it, publicly accessible. (At least, that was the case the ladt time I did). [0] https://auspost.com.au/business/marketing-and-communications/access-data-and-insights/address-data/postcode-data https://auspost.com.au/business/marketing-and-communications...
- tim-- 2y ago
  Isn't this G-NAF? https://geoscape.com.au/solutions/g-naf/ https://geoscape.com.au/solutions/g-naf/
  - shakna 2y ago
    Not quite. G-NAF is a government owned enterprise, separate to the privitised but government body of Australia Post. G-NAF is the equivalent to the UK's National Address Gazette. It's a separate body of data, that sometimes disagrees with the "source of truth" that is Australia Post, and all the post systems that rely upon them. For example, it took two years for G-NAF to notice that Winter Valley, Victoria, is not within 3356, but actually has its own brand new post code of 3358.
- memorylane 2y ago
  I think g-naf is freely available…
samwillis 2y ago
Sadly I don't think this would happen, particularly if Ordnance Survey is responsible, all their data is paid for access. We have a very different model for access to data produced by government agency use to that in the US. USGS Topographic maps: public domain / free UK OS Topographic maps: paid access, and it's not cheep US National Weather Service: Public domain / free commercial use UK MetOffie: Payed access for commercial use
- scraplab 2y ago
  OS does release a large volume of open data, but yes, the vast majority of the good stuff is not open. https://osdatahub.os.uk/downloads/open https://osdatahub.os.uk/downloads/open
- normangray 2y ago
  I remember asking a USGS person about this. They remarked that the other difference was that, compared with the OS, the USGS data was a bit rubbish (I may be paraphrasing). The USGS is funded by some shard of the US federal budget, and does commendably good stuff with the budget it gets; it's there for both high-minded and commerce-supporting reasons. The OS is now (in a sequence of reorganisations from 1990 to 2015) a private company with a government-owned golden share, and is expected to be revenue-positive. The fact that it has more money per square metre of country, means that it's able to be _very_ thorough, mapping down to the level of individual bits of street furniture. Sidenote: the context I was hearing this included a talk by someone from OS describing using reasoning software to do consistency checking of their GIS: for example, if you find a river bank in the middle of a field, something has been mislabelled. I thought that was cute. When you buy a data product from OS, you're buying some subset of the layers of the database. As the other reply pointed out, some of these layers are available for free, and in the last few years there's been some review/churn/debate in the data subsets made available that way (I see there are more details on the Wikipedia page). One can form a variety of opinions on whether those subsets are as big as they could or should be, but there does seem to be a substantial point that the level of the detail in the master map is there because it's profitable for the company (and thus income-generating for the government) to develop it from surveys, and it wouldn't exist otherwise. I think the Met Office is organised in a similar way. There are a number of questions of principle and practice here, but the OS seems to me to be claimable as an example (rare, in my opinion) of a privatisation which has produced net positive outcomes.
cuonic 2y ago
On the other side of the Channel, the French government has managed to create the "BAN" (Base Adresse Nationale - National Address Database), a database of detailed postal addresses in the country along with precise GPS coordinates: https://adresse.data.gouv.fr/base-adresse-nationale https://adresse.data.gouv.fr/base-adresse-nationale On top of the database they have provided an interface to view the data, interfaces for towns and cities to keep the data up-to-date, free APIs to search addresses and performing geocoding or reverse geocoding (https://adresse.data.gouv.fr/api-doc/adresse https://adresse.data.gouv.fr/api-doc/adresse) and the data is openly licensed and available to download. Feeding the BAN has been enforced by law, localities are required to put together and upload their "Base Adresse Locale" (Local Address Database) The original data was obtained from multiple sources, including "La Poste", the French Royal Mail equivalent, and OpenStreetMap !
- nottorp 2y ago
  Even in the far right US postcodes are public info :)
  - crote 2y ago
    The big difference is that US postcodes describe very large areas. A 5-digit US ZIP code describes a town or neighborhood, with on average 8200 people living in each ZIP code. Most European postcodes are far more precise, often describing a single street, part of a street, or even part of a building. Postcode + house number is usually enough to uniquely identify a mailbox. For example, in The Netherlands on average only 40 people live in each postcode. That makes the dataset far more valuable for geolocation.
    - dmurray 2y ago
      The US also has 9-digit postcodes which usually map to a single building or smaller: aren't they public too?
      - bluGill 2y ago
        They are public, but the post office changes the last 4 digits every few months so there is no point in telling anyone what yours is. These days the post office can look up your street address and give you all the information they need - which is an 11 digit bar code good for the next week.
b112 2y ago
Sad to see a reasonable article with a "This one weird trick could save..." as an ad inline, pointing back to his own page. I tend to think of such ad tactics and wordage to be associated with used car salesmen. Certainly, with scams.
- tomstuart 2y ago
  That’s the joke.
  - b112 2y ago
    If I call some place I've never heard of before, know nothing about, my first interaction with them on the phone shouldn't result in "Oh my god, these people seem like scammy used car salespeople!" If your assertion is true, that it's a joke, it's going to backfire. That's because that call is the equivalent of what's happening here. I called, and the person on the other end ... thinking it a joke, funny, did their best to convince me that they're scam artists. That's what's happened here. I know nothing about this website, and this was my first impression. And no... my initial reaction isn't "Hmm. This website seems scammy and lame. Maybe I should spend my time investigating to determine if I'm right or wrong!". If I did that, I'd spend my entire life looking at scammy websites... I have better things to do. Like I said, it's a shame to see this on what seems to be reputable website. But I literally stopped reading, and moved on to other things when I saw it. The website owner should take that into account. (And indeed, I may be some small ratio, 2% of users, but it could be higher. It could be a lot higher. Or it could obviously be 0.2%. But that's a bold move, putting a big "I'm a scam artist!" sign on a website, first engagement is going to bite.) Heck... if I was Google, any page with "One * trick" on it would be downranked. TL;DR don't put a massive sign on your website that reads "I'm a scam artist, clickbait website!"
    - jstanley 2y ago
      It pattern-matched "scam" so you classified it as "scam" and absolved yourself of doing any further thinking. If something pattern-matches "legit" are you equally blase about sticking with your snap judgment and absolving yourself of doing any further thinking?
robin_reala 2y ago
For non-GB people, a postcode gets you to ~1-15 buildings, not (for example) a town or region.
- mrweasel 2y ago
  The British mail addresses are pretty interesting. We quickly learned that, as you say some postcode have just one or two houses, which may not have numbers, but names. I'm sure there is a "falsehoods programmers believe about addresses" somewhere.
  - darrenf 2y ago
    https://www.mjt.me.uk/posts/falsehoods-programmers-believe-about-addresses/ https://www.mjt.me.uk/posts/falsehoods-programmers-believe-a...
    - mrweasel 2y ago
      Amazing. For a e-commerce site I argued that we would save ourself a lot of trouble by simply making the address field one large text field, rather than attempt to making a form that would work for every country and city (looking at you Mannheim). But apparently that would make data analysis to complicated.
      - bojanz 2y ago
        There is a middle ground and some common patterns that can help. The address field names are fairly standardized[0] and Google has an open dataset (used by Chrome and Android) describing which countries need which fields[1]. I have an older PHP library[2] and a newer Go library[3] that build upon this, while crowdsourcing fixes (since Google hasn't updated their dataset in a while). The Go library allows me to serve all address formats and state lists in a single HTTP request, which can then power a very fast JS widget. [0] Initially by the OASIS eXtensible Address Language (xAL) which trickled down into everything from maps to HTML5 autocomplete. [1] https://chromium-i18n.appspot.com/ssl-address https://chromium-i18n.appspot.com/ssl-address [2] https://github.com/commerceguys/addressing https://github.com/commerceguys/addressing [3] https://github.com/bojanz/address https://github.com/bojanz/address
cjs_ac 2y ago
> The upshot of the research then, is that building an accurate database is really hard. OS concludes that it would have to check the 4.2m bad addresses manually to make its PAF-less database a viable dataset that would actually be useful. The secret to the Royal Mail's success with the PAF, and the reason why only the Royal Mail can maintain the PAF, is that the Royal Mail has people walking and driving to all those delivery points six days every week. Compare the Freedom of Information requests to Royal Mail from OpenStreetMap contributors concerning the locations of post boxes, which were refused ultimately because that information was handled only by local sorting offices.
- chihuahua 2y ago
  Maybe they can use the TV detector vans used for TV license enforcement to collect the data, if they're already surveilling every single building in the country on a daily basis!
  - dambi0 2y ago
    Checking buildings you already know to have a license seems wasteful, but perhaps I missed something?
Normal_gaussian 2y ago
As censorship for FOIA requests is done manually, it may be beneficial to request the missing figures directly without noting you have them in a censored context. Censoring is subjective, so that would at least draw out either the figures or a justification.
nly 2y ago
Unfortunately the British mindset these days is to either rent it out or sell it but, whatever the hell you do, don't grow it. Somehow these idiots managed to strike a deal to keep the sovereigns figurehead on stamps (which has no economic value whatsoever, and actually the Crown should be compensated for this) but, in this data age, didn't safeguard such a critically important database to e-commerce It's like selling off the Tower of London because you can't afford to repair the roof and forgetting you left the crown jewels inside
TechTechTech 2y ago
For comparison, in the Netherlands all postcode data is open data, including detailed building outlines as well as almost all other related information. See https://app.pdok.nl/viewer https://app.pdok.nl/viewer for most datasets.
- Muromec 2y ago
  This creates a very special Dutch thing —- my neighborhood had the roads on the map before the map itself was updated to show landmass instead of the body of water.
  - anticensor 2y ago
    Same in Turkey, except the map data is subject to certain limitations.
  - DonHopkins 2y ago
    I wonder if all the houses on disconnected long islands without roads in Vinkeveense Plassen have postal codes? It's hard to get a pizza delivered there. https://www.google.com/maps/@52.2307079,4.9365182,1869m/data=!3m1!1e3?hl=en&entry=ttu&g_ep=EgoyMDI0MDgyMC4xIKXMDSoASAFQAw%3D%3D https://www.google.com/maps/@52.2307079,4.9365182,1869m/data...
    - jorams 2y ago
      In the PDOK viewer linked above you can enable the "Adressen" layer[1] and it will show markers on everything that has an address. Everything that has an address has a postal code, which is listed in the details if you click the address. (There might be an exception with an address but no postal code somewhere, I'm not sure, but not here.) [1]: https://app.pdok.nl/viewer/#x=124175.54&y=471068.96&z=11.2907&background=BRT-A%20standaard&layers=4074b3c3-ca85-45ad-bc0d-b5fca8540z0b;adressen https://app.pdok.nl/viewer/#x=124175.54&y=471068.96&z=11.290...
    - Muromec 2y ago
      That area looks so weird on a map and so cool in person. I never kinda understood what is going on there except the whole having a lake and being the Netherlands.
agolio 2y ago
I am a bit surprised by how hard this article makes out the problem to be. Crowdsourcing should make short work of the problem, with the right incentives, which the government will be able to offer. Additionally private map providers (e.g. Google, Apple) must surely have this data (since they are able to route navigation to private addresses). Why not just negotiate with them?
- darrenf 2y ago
  How would crowdsourcing solve this problem? > Oh, and it wouldn’t even be legally allowed to include, er, postcodes, as they are specifically owned by Royal Mail
  - moring 2y ago
    How does OpenStreetMap solve it? OSM, more specifically OSM Nominatim, shows postcodes. Example: https://www.openstreetmap.org/search?lat=53.151778&lon=-1.168003#map=19/53.151779/-1.168003 https://www.openstreetmap.org/search?lat=53.151778&lon=-1.16...
    - darrenf 2y ago
      I didn't know the answer so I looked it up. Nominatim gets postcodes from Ordnance Survey: https://nominatim.org/release-docs/3.4/data-sources/GB-Postcodes/ https://nominatim.org/release-docs/3.4/data-sources/GB-Postc... Specifically Code-Point Open: https://www.ordnancesurvey.co.uk/products/code-point-open https://www.ordnancesurvey.co.uk/products/code-point-open which is updated quarterly, and in turn gets the postcodes from Royal Mail.
      - moring 2y ago
        It seems to me that you can download the postcode list freely: https://osdatahub.os.uk/downloads/open/CodePointOpen https://osdatahub.os.uk/downloads/open/CodePointOpen Something is missing here. If OS already has that data from RM and can make it available freely, why would they need to build another database?
andrewjl 2y ago
There's a writeup linked to in the OP comments about how this can happen. https://www.centreforpublicdata.org/blog/freeing-the-paf-our-meeting-with-jonathan-reynolds-on-address-data https://www.centreforpublicdata.org/blog/freeing-the-paf-our...
IshKebab 2y ago
Off topic but this is a bizarrely weird take: > Sadly because of the NIMBYs, this map doesn’t include a London version of The Sphere. "NIMBY" implies they're objecting to something useful and not actually that bad, like a solar farm or a mobile phone mast or a housing estate. Not a giant advertising billboard.
- lol768 2y ago
  I think it's a joke.
robinhouston 2y ago
This is a long-running battle. Those with long memories may remember the skirmish 15 years ago, when a small group of developer-activists set up a website that allowed free access to postcode data (ernestmarples.com, named after the inventor of the modern British postcode system). Needless to say, it was rapidly shut down following threats of legal action by Royal Mail. https://www.theguardian.com/technology/2009/oct/05/ernest-marples-royal-mail-postcodes-legal-threat https://www.theguardian.com/technology/2009/oct/05/ernest-ma... https://blog.okfn.org/2009/10/05/ernest-marples-uk-postcode-site-has-been-taken-down/ https://blog.okfn.org/2009/10/05/ernest-marples-uk-postcode-...
- michaelt 2y ago
  The postcode-to-coordinate data is now freely available as "CodePoint Open" So there's already data for people who want to know postcode AB10 1JL corresponds to the area around 57.14677,-2.09873 The PAF is a more detailed data source, as seen in https://www.royalmail.com/find-a-postcode https://www.royalmail.com/find-a-postcode which can tell you that AB10 1JL specifically covers the addresses 102-104, Union Street, Aberdeen 82, Union Street, Aberdeen Timpson Shoe Repairs Ltd, 86 Union Street, Aberdeen Smart Mobile, 88 Union Street, Aberdeen 92 Union Street, Aberdeen 98 Union Street, Aberdeen The PAF is useful if you want to provide a "quick address entry" option on your website - and to validate address data. But if you just want postcode-to-location conversion, that info is already available.
  - robinhouston 2y ago
    Thanks for the clarification. I’d forgotten that ernestmarples only offered postcode-to-location lookup: it was a long time ago. I suppose this is encouraging! It shows that the forces of openness are gaining ground in this battle.
  - sahmeepee 2y ago
    Given that OSM has a lot of building addresses and locations (at least it has building names and numbers and the streets they are on, plus their outlines) I wonder how much of the PAF could be recreated from the data in CodePoint Open combined with OSM. If adding more house numbers etc to OSM would help improve that then it could be crowdsourced quite quickly via Street-Complete or similar. For the purposes of the quick address lookup it doesn't even need to be 100% accurate. If a property is incorrectly listed against two postcodes it won't really cause problems, so the lookup could err on the generous side
NeoTar 2y ago
Some context, for people not located in the UK - A full British postcode typically aims to cover around 15 buildings (sometime a single building, sometimes a street of 50 houses). This is in contrast to many other postal code systems which cover relatively broad areas). Or to put it another way - UK - 2,643,732 codes, 1 code per 25 people, USA - 41,700 codes, 1 code per 8000 people, Germany - 8,200 codes, 1 code per 10000 people, This means that post-codes are often used as a proxy for an exact location - e.g. if I am going to visit a relative, I can enter their postcode into my sat-nav, and be confident that most of the time I'll get to within a hundred meters of their location. This doesn't work so well in rural area or on large estates where the access point may different from the location, leading to places sometimes advertising a different postcode to put into your sat-nav (e.g. of where the site entrance is) to that of the location itself.
- simonbarker87 2y ago
  Hence why a house number and postcode constitutes a complete address in the UK, we’ve sort of already got What Three Words with “a number and 5-7 characters” - not quite as catchy though
  - beardyw 2y ago
    We needed an ambulance off road in the middle of Richmond Park where a postcode would also not help. We didn't have WTW either, which they asked for and would have helped immensely.
    - sideshowb 2y ago
      If only we had a system of national grid references since, say, 1936
      - beardyw 2y ago
        They were only interested in What Three Words. Didn't want anything else, sadly.
maccard 2y ago
At least the Uk has the the defence that postcodes are 60 years old and that the legacy cruft that comes with that is part of life. Meanwhile Ireland introduced Éircodes less than 10 years ago, chose an opaque format that uses a central database that you have to pay for access to for anything more than a handful of lookups, only covers homes (so you can’t give an eircode of a park, or a walk). It’s pretty much what you’d expect to be designed by a modern government.
- chgs 2y ago
  I’d expect a modern government to design something as clear and well regarded as the GDS stuff in the U.K. I’d expect a corporation like ibm etc to design the total mess we see with any large project
  - jetbooster 2y ago
    Sadly I feel GDS is more of an outlier than the rule.
    - willyt 2y ago
      Transport for London is a pretty tightly run ship. Only capital city in the world that doesn't receive operating subsidy for its public transport. Not that that is a good thing necessarily as the tube is expensive to use relative to Paris or Berlin but a pretty impressive achievement considering the ancient complexity of the whole thing. Scotrail is run by the Scottish government and has been steadily electrifying the Scottish rail network and because of the slow and steady nature of the work, between them, Network rail and the OHLE contractors they have got the cost for this down to 5 times less per km than typical UK costs previously e.g. the great western main line. The moral of the story is get good people, give them stability and a clear goal and they will do great work. It doesn't really matter if they are working for the government or the private sector.
- 7952 2y ago
  Postcode data is already freely available. You can even get coordinates of where the properties are. What is lacking is the actual addresses within a postcode.
wiredfool 2y ago
Essentially the same deal in Ireland, with Eircodes. They were originally created as private dataset with ownership, and now you have to license access to it to use it. Eircodes are better than postcodes, in that there's 1 per building/address/apartment, however they're discontinuous, so adjacent buildings will have distinctly different eircodes. The article highlighted the difficulty of shopping centers and apartment buildings, from my experience trying to validate a large number of Eircode <-> addresses for a project, this is definitely an issue. The worse issue is that there's no way to just send someone out to check, because the eircode isn't like a house number that's posted somewhere. (Leaving aside the problem that valid Irish addresses can have no numbers outside of the eircode, and eircodes are a recent, and therefore non-traditional addition)
- NeoTar 2y ago
  I was impressed when I first heard about the objectives of the Eircode system, but it seems the implementation is lacking.
  - wiredfool 2y ago
    The implementation was captured by a private party.
- closewith 2y ago
  Eircodes also aren't used by An Post, to add insult to injury. > The worse issue is that there's no way to just send someone out to check, because the eircode isn't like a house number that's posted somewhere. (Leaving aside the problem that valid Irish addresses can have no numbers outside of the eircode, and eircodes are a recent, and therefore non-traditional addition) The HSE National Ambulance Service (NAS) National Emergency Operations Centres (NEOCs) have a GIS package that resolves Eircodes (and other traditional and colloquial addresses) to actual buildings and building entrances in real-time, which actually quite impressive. The directions can be transmitted to ambulances and other assets in real-time and has reduces delays in clinical services due to address confusion enormously since 2016. So the country is capable. Eircode is what we chose as a country, not what we were limited to.
ascorbic 2y ago
As he points out, this was a profoundly stupid mistake made when privatising Royal Mail. It would have been trivially easy to do at that point, but now it's a lot harder. If the government decided that it does want to do this, it can't just pass a law that says "the PAF is now free" without paying hundreds of millions of pounds in compensation to Royal Mail. That's quite apart from the ongoing costs of maintaining the data. At a time of cuts of budgets this would be a hard sell.
- knallfrosch 2y ago
  > If the government decided that it does want to do this, it can't just pass a law that says "the PAF is now free" without paying hundreds of millions of pounds in compensation to Royal Mail. You can pass the law, get sued and pay whatever the PAF is worth. But that's just.. fair? The govnerment spent 5 mio just for a survey concluding that it's impossible to recreate the PAF. So hundreds of millions sounds like a good deal.
  - scott_w 2y ago
    Just to be clear: UK Parliament is sovereign. If it passes a law forcibly legalising it, the privatised Royal Mail can sue the government but would need to find an international treaty obligation to win. Even then, if Parliament flagged it and said "we're ignoring this treaty in this case" then the courts are bound to the law, not treaty obligations. If it has knock-on impacts in other areas, it's hard to say, but that's separate to the law.
    - ascorbic 2y ago
      Article 1 of Protocol 1 of the ECHR covers exactly this, so the Supreme Court (and ECtHR if it came to that) would probably find in favour of Royal Mail if this were to be done without compensation.
      - jjmarr 2y ago
        The court would make a "declaration of incompatibility" with the ECHR which leaves it up to Parliament to change the law. https://en.wikipedia.org/wiki/Declaration_of_incompatibility https://en.wikipedia.org/wiki/Declaration_of_incompatibility
intellix 2y ago
it would be nice if the postal system in the UK and anywhere in the world supported what3words to be honest
- NeoTar 2y ago
  Replacing one proprietary database with another? Is that truly useful?
- manarth 2y ago
  Replace one proprietary format owned by a private organisation with another proprietary format owned by a different private organisation?
  - duncans 2y ago
    Plus, fraught with usability issues https://cybergibbons.com/security-2/why-what3words-is-not-suitable-for-safety-critical-applications/ https://cybergibbons.com/security-2/why-what3words-is-not-su...
askvictor 2y ago
Our company started operating in the UK recently, and some of our customers were very surprised we didn't charge for a subscription for part of our product. The idea would have no legs in Australia (our homeland) but is completely normal in the UK. So, new revenue stream for us, and some learnings about the UK culture.
- lewispollard 2y ago
  It is, but at least in my experience, we do it for the 10% discount and then immediately cancel the subscription every time we want to make a purchase.
londons_explore 2y ago
The simple solution here is a threat from the government to Royal Mail. Give us your postcode file for free, or we will simply make up a new numbering scheme, send an address card to every house telling them of their new number with their next council tax bill, and postcodes will become a thing of the past. The new numbering scheme will be unique to each house too, and have a check digit so the number alone is sufficient for 3rd party logistics companies like Amazon to use it for deliveries.
- n4r9 2y ago
  The idea of the UK government attempting to do such a thing fills me with the utmost dread.
  - DaiPlusPlus 2y ago
    When did this almost Reaganite sentiment ("I'm from the government and I'm here to help") make home in the UK? I know it's not recent: I remember similar arguments coming from the No2ID camp in 2005 at-least.
    - pjc50 2y ago
      Quite a lot of it is Reaganism, via Thatcher. Probably dates from the Winter of Discontent. It's not entirely without merit, but only because there's a tendency to drastically underfund and micromanage state services. And things like the Post Office Horizon fiasco do not make the government look good here. On the other hand GDS is excellent - but that's almost entirely as a result of staff professionalism, rather than being driven by whichever ministers had the leadership of the civil service. An odd outcome of the ID discourse is that we now have an extremely high tech biometric identity system .. but only for immigrants.
      - willyt 2y ago
        'The Post Office' is a private company. Wasn't the Horizon system implemented after privatisation?
darau1 2y ago
I hope this happens. I can only dream of the day when my country gets something like this.
librasteve 2y ago
great article, this demonstrates just how bad the civil service & politicians are when it comes to negotiating contracts with private investors… or trade deals, or brexit if it comes to that
deleted 2y ago
[deleted]
sschueller 2y ago
In Switzerland anyone can use the national database of addresses. https://www.swisstopo.admin.ch/en/official-directory-of-building-addresses https://www.swisstopo.admin.ch/en/official-directory-of-buil...
justinclift 2y ago
This doesn't seem correct: The problem is it’s not an easy dataset to get hold of, as it cost a lot of money. This is because the data has to be licensed from Royal Mail ... It seems to be talking about the National Statistics Postcode Lookup UK, which is officially published here: https://www.data.gov.uk/dataset/7ec10db7-c8f4-4a40-8d82-8921935b4865/national-statistics-postcode-lookup-uk https://www.data.gov.uk/dataset/7ec10db7-c8f4-4a40-8d82-8921... It's been there from at least 2017, which is when I first came across it. There are later version of the data set online too: https://open-geography-portalx-ons.hub.arcgis.com/datasets/ons::national-statistics-postcode-lookup-2021-census-may-2024-for-the-uk/about https://open-geography-portalx-ons.hub.arcgis.com/datasets/o... The license: https://www.ons.gov.uk/methodology/geography/licences https://www.ons.gov.uk/methodology/geography/licences Under the terms of the Open Government Licence and UK Government Licensing Framework (launched 30 September 2010), if you wish to use or re-use ONS material, whether commercially or privately, you may do so freely without a specific application for a licence, subject to the conditions of the Open Government Licence and the Framework. If you are reproducing ONS content you must include a source accreditation to ONS. If the article is talking about a different postcode address file though, then the above doesn't apply. ;)
- jokethrowaway 2y ago
  That's not the https://www.poweredbypaf.com/product/paf/ https://www.poweredbypaf.com/product/paf/
  - justinclift 2y ago
    No worries. :)
jokethrowaway 2y ago
Or we can just start using https://what3words.com/ https://what3words.com/ and geolocation. I disagree with the report, I think it's feasible with a bit of creativity. The government also has this: https://www.data.gov.uk/dataset/091feb1c-aea6-45c9-82bf-768a15c65307/open-postcode-geo https://www.data.gov.uk/dataset/091feb1c-aea6-45c9-82bf-768a... We could also start with an imperfect solution, offer it as a free API (maybe even self-hosted and communicating with other services p2p) and wait for users to select or insert missing addresses, until we eventually converge to a good OSS database. If it's a single service being shared by everyone, you would need to insert your address once and then it would be part of the database forever, and you would get the right result at any other time in the future. There is also a dirty but hard to attack option: - Start from the NAG - Build an opaque AI process which is hard to audit and that is tuned until it produces a result close to PAF but with a few extra errors - Sell the new database to the government, government open sources that - Directors get paid their share - Company get sued out of existance by RoyalMail - Government pays a few millions in 20 years, if the RoyalMail experts can prove anything in court
MSFT_Edging 2y ago
It blows my mind how many public services have been privatized in the UK. It just feels like they're selling off the shoes they're standing in. When their railways got privatized, the service didn't improve, the price just ballooned. Even in the states, the USPS has resisted privatization this far. For the love of god I hope it continues to. Protect our boys n girls in blue and tell your congressman you want postal banking.
- normangray 2y ago
  Yup. The Post Office, the railways, the water system, for heavens' sake! The tories, as a matter of religious faith, see privatised => efficient, whilst being unclear on the difference between 'efficient at creating shareholder value' and 'efficient at serving the public good'. The political mood music, over the last few decades, has meant that the Labour party has repeatedly found itself obliged to say positive things about privatisation, as part of the process of Being Sensible About The Economy (there is a much longer alternative version of this comment!). The US -- the world temple of capitalism -- seems to be oddly principled (viewed from outside) about keeping certain things such as the postal service, or USGS, as part of the service to the public realm. The one service probably immune from privatisation is the Health Service. It's only the most frothing-at-the-mouth right-wingers, the provocateurs just one step away from a rabies injection, who'd even admit out loud to a desire to do that. A politician talking about privatising the NHS would I think be pretty much equivalent to a US libertarian politician talking about privatising the armed forces. (there's a longer version of that comment, as well...)
  - johneth 2y ago
    Minor pedantic correction: The Post Office (shops, services, government forms, etc.) is still fully government-owned. It's the Royal Mail (delivery) that was privatised. They used to be the same company but split in two before Royal Mail was privatised.
    - normangray 2y ago
      Thank you! – yes, it's a private company, fully owned by the Government. I think I sort-of knew that, but it hadn't properly registered; I may indeed have been confusing it with Royal Mail. It means that the PO is quite closely analogous to the Ordnance Survey, in organisational terms. It's maybe describable as a quasi-privatisation, in that the company is run on a fully commercial basis, with correspondingly narrow goals in principle, but with the profits (or, in the case of the PO, the losses) going to the Exchequer, and the relevant minister (presumably) having some say about the appointment of board members. It's organisationally tidy, I suppose, and manages to fit in with the long-standing 'private=efficient' doctrine. It still feels vaguely off, to me; not quite cricket.
alexchamberlain 2y ago
To be clear, is the National Address Gazetteer open? As far as I can tell, it isn't, but I don't know if that's because they're trying to obfuscate it.
nickdothutton 2y ago
Recent history teaches that the Post Office should be the last company on earth to be anywhere near creating a nationally important IT system. Their technology team have been useless for decades.
- dan-robertson 2y ago
  The PAF is maintained by Royal Mail, a different company from the Post Office.
frereubu 2y ago
I've used https://postcodes.io/ https://postcodes.io/ on a number of projects. There's information about it here - https://postcodes.io/about https://postcodes.io/about - which doesn't fully answer where the data comes from, although it mentions OS, so I presume it's based on the OS AddressBase product? I also wonder how complete it is now, although the sites we've built haven't had any issues as far as I'm aware.
gorgoiler 2y ago
As the tenant, once, of a new build home in the UK it’s not just the file that’s important: it’s the channels to patch the file too. I spent almost a year having to enter my address manually because the postcode DB — or whatever old version pets.com, cameras.com, and looroll.com had — lagged behind the reality of my infill bungalow for seemingly forever. I’m 8A godamnit, not 8. (Thank you Mrs. No8 for accepting my packages throughout those dark months btw.) It’s just like tzdata. A precious resource not just because it compiles the history of geopolitical wallclock settings, but also because it is meticulously updated, on time and on budget. It’s all very well liberating PAF.v2024_08finalfinal_v3_final.doc, but who is also going to keep it up to date? I’m not a hater, just a realist. TFA is spot on: we’ll never be able out compete with or recreate or leak a sufficient version of the PAF. It, and it’s entire infrastructure, needs to be wrested into public hands ASAP. And we should be prepared to fund the updates.
- sahmeepee 2y ago
  I've been a consumer of the PAF for many years through work. In the bad old days (10 years ago), when we got it from a well known credit check company, it came on CDs. We then moved to "automatic" updates via their rubbish, buggy software to our on-premise install. That wasn't such a huge upgrade, because as soon as you looked away from that software it would stop working. Now we pay a different, altogether better company to access it via a web service. I would expect that the speed of updates through to websites is now mostly as good as the frequency of PAF releases. Unless this has changed recently, the PAF isn't created by Royal Mail itself, it's outsourced to a company called AFD based on the Isle of Man. My only complaint with the service we get now is that the value is quite poor - our total cost is somewhere around 30p per lookup on average for addresses which I suspect are rarely even used in our line of work.
mjevans 2y ago
They could also link tax filings and a Plus Code for physical location: https://maps.google.com/pluscodes/ https://maps.google.com/pluscodes/ https://github.com/google/open-location-code https://github.com/google/open-location-code
gregsadetsky 2y ago
Canada is in the same spot with the postal code data being for sale only. Relatedly and famously, Canada Post sued [0] geocoder.ca [1] years ago because it made available a crowdsourced database of postal codes... Years later, the case was settled [2] but yeah. You still have to pay for the "real" data. [0] https://www.michaelgeist.ca/2012/04/canada-post-geocoder-suit/ https://www.michaelgeist.ca/2012/04/canada-post-geocoder-sui... [1] https://geocoder.ca/ https://geocoder.ca/ [2] https://geocoder.ca/?sued=1 https://geocoder.ca/?sued=1 (nice url...)
ss64 2y ago
The PAF file only returns a 9% profit (based on a 2009 report) https://web.archive.org/web/20090324084021/http://www.psc.gov.uk/postcomm/live/policy-and-consultations/consultations/postcode-address-file--management-of-information/PAF_decision_doc.pdf https://web.archive.org/web/20090324084021/http://www.psc.go...
kwhitefoot 2y ago
Why bother paying attention to all this legal mumbo-jumbo? Just have someone exfiltrate the file and post it on Anna's Archive. Or extract them from Open Street Map. Just bypass the Royal Mail altogether. Or just ignore the postcodes. For most private individuals an ordinary street address works perfectly well and is needed anyway even if the postcode is provided.
- sahmeepee 2y ago
  It's mostly used to get clean address data on form inputs. The user thinks they are being given an easier way to enter their address, but really it's so that the PAF can be used to ensure the address lines are used consistently and there are no typos or other rubbish. If the person has to pick their address from a list after entering the post code it's also a data input check of sorts.

356 points by edward 2y ago

I can respect the arguments for making it public but there are strong arguments also to raise a high barrier of entry to discourage abuse. Further, the fewer users of the list, they easier they are to police.

xnorswap 2y ago
It's a lookup between postcode and address, what is the abuse cases you're worried about?
- lnxg33k1 2y ago
  Considering that in UK if you live in a building, the door next to you can have a different postcode, I wouldn't worry at all
  - xnorswap 2y ago
    If that weren't true, you'd have entire cities in the same postcode. There has to be a boundary somewhere.
    - nly 2y ago
      Odd numbered homes on one side of the street and evens on the other often have different postcodes
      - willvarfar 2y ago
        Postcodes are about sorting mail to match the delivery rounds.
      - ooklala 2y ago
        Many buildings also have their own postcode! (The second half of the postcode represents the 'delivery point' which is basically limited by the amount of post that the postman/woman can physically carry...)
    - lnxg33k1 2y ago
      Well, in Italy postcodes define city areas, and cities, for example for my city the main postcode is 80100, but my area is 80142, and it contains few buildings, so it's different from UK, UK was the first time I saw such specific postcodes, and I've lived also in Germany and Netherlands
      - Muromec 2y ago
        Netherlands had a postcode per street
        lnxg33k1 2y ago
        Oh yeah, I remember being able to insert just postcode and street number in forms, but it's not as specific as UK, I think
  - ben_w 2y ago
    I've lived in a one bed apartment where the front and back doors had different postcodes. IIRC, the neighbours to one side in the same building had a third postcode for their front, but shared mine for the back.
- pjc50 2y ago
  Crucially, it doesn't have people's names in it.
  - xnorswap 2y ago
    Indeed. If it's an issue that someone would know your address, then it's an issue that they would know your postcode. If it's an issue that someone would know your postcode, then it's an issue that they would know your address. I'm struggling to think of a scenario where you'd be fine with someone knowing one of those pieces of information without knowing the other. It's not therefore an issue that there's a lookup between the two. Indeed you can do it trivially with google maps, or the plenty of other services that expose this database through their operation. Any safety concerns aren't at the layer of translation between postcode and address, they're how someone tied either of those pieces of information to a given person.
secretsatan 2y ago
You miss the point that it was once freely accessible, and now it is not.
- scraplab 2y ago
  I don’t believe it’s ever been accessible for free. It’s just that ownership has moved from the state to a private company and now it’s difficult to make it open.
nottorp 2y ago
Yeah, maybe you should pay a subscription to know your own post code...
IneffablePigeon 2y ago
What nonsense. Are you worried about physical spam mail? That ship has already sailed. I genuinely can’t think of any other abuse vector for a dataset like this.
andrewjl 2y ago
> Would open address data create privacy risks? No. Unlike opening up more sensitive datasets such as personal location, releasing address data - a list of the physical places recognised by the government - carries few new legal or ethical risks. Many other countries are doing this, including those with strong privacy regimes. Open address data could only create new risks if it were linked and used with other datasets, and these risks should be managed in that context. The harms created by the lack of access to address data are more pressing. https://static1.squarespace.com/static/5ee7a7d964aeed7e5c507900/t/655dd5d4d1c0f4729bca69c9/1700648428211/Briefing%3A+Address+data https://static1.squarespace.com/static/5ee7a7d964aeed7e5c507...
mrweasel 2y ago
How exactly would that be abused? Denmark have a website where you can enter any address, or an address close to where you want to be and then let you select the right house on a map. The same site will show you the owners, the purchase price the taxable value, size, number of bathrooms, stuff like that. I used it to find the address of a friend when I needed to ship him a present and I only roughly knew where he lives.

shakna 2y ago

Australia also has ours locked away privately. You can purchase access, but... You also need to sign a contract that you won't make the PDF, or anything you derive from it, publicly accessible. (At least, that was the case the ladt time I did). [0] https://auspost.com.au/business/marketing-and-communications/access-data-and-insights/address-data/postcode-data https://auspost.com.au/business/marketing-and-communications...

tim-- 2y ago
Isn't this G-NAF? https://geoscape.com.au/solutions/g-naf/ https://geoscape.com.au/solutions/g-naf/
- shakna 2y ago
  Not quite. G-NAF is a government owned enterprise, separate to the privitised but government body of Australia Post. G-NAF is the equivalent to the UK's National Address Gazette. It's a separate body of data, that sometimes disagrees with the "source of truth" that is Australia Post, and all the post systems that rely upon them. For example, it took two years for G-NAF to notice that Winter Valley, Victoria, is not within 3356, but actually has its own brand new post code of 3358.
memorylane 2y ago
I think g-naf is freely available…

samwillis 2y ago

Sadly I don't think this would happen, particularly if Ordnance Survey is responsible, all their data is paid for access. We have a very different model for access to data produced by government agency use to that in the US. USGS Topographic maps: public domain / free UK OS Topographic maps: paid access, and it's not cheep US National Weather Service: Public domain / free commercial use UK MetOffie: Payed access for commercial use

scraplab 2y ago
OS does release a large volume of open data, but yes, the vast majority of the good stuff is not open. https://osdatahub.os.uk/downloads/open https://osdatahub.os.uk/downloads/open
normangray 2y ago
I remember asking a USGS person about this. They remarked that the other difference was that, compared with the OS, the USGS data was a bit rubbish (I may be paraphrasing). The USGS is funded by some shard of the US federal budget, and does commendably good stuff with the budget it gets; it's there for both high-minded and commerce-supporting reasons. The OS is now (in a sequence of reorganisations from 1990 to 2015) a private company with a government-owned golden share, and is expected to be revenue-positive. The fact that it has more money per square metre of country, means that it's able to be _very_ thorough, mapping down to the level of individual bits of street furniture. Sidenote: the context I was hearing this included a talk by someone from OS describing using reasoning software to do consistency checking of their GIS: for example, if you find a river bank in the middle of a field, something has been mislabelled. I thought that was cute. When you buy a data product from OS, you're buying some subset of the layers of the database. As the other reply pointed out, some of these layers are available for free, and in the last few years there's been some review/churn/debate in the data subsets made available that way (I see there are more details on the Wikipedia page). One can form a variety of opinions on whether those subsets are as big as they could or should be, but there does seem to be a substantial point that the level of the detail in the master map is there because it's profitable for the company (and thus income-generating for the government) to develop it from surveys, and it wouldn't exist otherwise. I think the Met Office is organised in a similar way. There are a number of questions of principle and practice here, but the OS seems to me to be claimable as an example (rare, in my opinion) of a privatisation which has produced net positive outcomes.

cuonic 2y ago

On the other side of the Channel, the French government has managed to create the "BAN" (Base Adresse Nationale - National Address Database), a database of detailed postal addresses in the country along with precise GPS coordinates: https://adresse.data.gouv.fr/base-adresse-nationale https://adresse.data.gouv.fr/base-adresse-nationale On top of the database they have provided an interface to view the data, interfaces for towns and cities to keep the data up-to-date, free APIs to search addresses and performing geocoding or reverse geocoding (https://adresse.data.gouv.fr/api-doc/adresse https://adresse.data.gouv.fr/api-doc/adresse) and the data is openly licensed and available to download. Feeding the BAN has been enforced by law, localities are required to put together and upload their "Base Adresse Locale" (Local Address Database) The original data was obtained from multiple sources, including "La Poste", the French Royal Mail equivalent, and OpenStreetMap !

nottorp 2y ago
Even in the far right US postcodes are public info :)
- crote 2y ago
  The big difference is that US postcodes describe very large areas. A 5-digit US ZIP code describes a town or neighborhood, with on average 8200 people living in each ZIP code. Most European postcodes are far more precise, often describing a single street, part of a street, or even part of a building. Postcode + house number is usually enough to uniquely identify a mailbox. For example, in The Netherlands on average only 40 people live in each postcode. That makes the dataset far more valuable for geolocation.
  - dmurray 2y ago
    The US also has 9-digit postcodes which usually map to a single building or smaller: aren't they public too?
    - bluGill 2y ago
      They are public, but the post office changes the last 4 digits every few months so there is no point in telling anyone what yours is. These days the post office can look up your street address and give you all the information they need - which is an 11 digit bar code good for the next week.

b112 2y ago

Sad to see a reasonable article with a "This one weird trick could save..." as an ad inline, pointing back to his own page. I tend to think of such ad tactics and wordage to be associated with used car salesmen. Certainly, with scams.

tomstuart 2y ago
That’s the joke.
- b112 2y ago
  If I call some place I've never heard of before, know nothing about, my first interaction with them on the phone shouldn't result in "Oh my god, these people seem like scammy used car salespeople!" If your assertion is true, that it's a joke, it's going to backfire. That's because that call is the equivalent of what's happening here. I called, and the person on the other end ... thinking it a joke, funny, did their best to convince me that they're scam artists. That's what's happened here. I know nothing about this website, and this was my first impression. And no... my initial reaction isn't "Hmm. This website seems scammy and lame. Maybe I should spend my time investigating to determine if I'm right or wrong!". If I did that, I'd spend my entire life looking at scammy websites... I have better things to do. Like I said, it's a shame to see this on what seems to be reputable website. But I literally stopped reading, and moved on to other things when I saw it. The website owner should take that into account. (And indeed, I may be some small ratio, 2% of users, but it could be higher. It could be a lot higher. Or it could obviously be 0.2%. But that's a bold move, putting a big "I'm a scam artist!" sign on a website, first engagement is going to bite.) Heck... if I was Google, any page with "One * trick" on it would be downranked. TL;DR don't put a massive sign on your website that reads "I'm a scam artist, clickbait website!"
  - jstanley 2y ago
    It pattern-matched "scam" so you classified it as "scam" and absolved yourself of doing any further thinking. If something pattern-matches "legit" are you equally blase about sticking with your snap judgment and absolving yourself of doing any further thinking?

robin_reala 2y ago

For non-GB people, a postcode gets you to ~1-15 buildings, not (for example) a town or region.

mrweasel 2y ago
The British mail addresses are pretty interesting. We quickly learned that, as you say some postcode have just one or two houses, which may not have numbers, but names. I'm sure there is a "falsehoods programmers believe about addresses" somewhere.
- darrenf 2y ago
  https://www.mjt.me.uk/posts/falsehoods-programmers-believe-about-addresses/ https://www.mjt.me.uk/posts/falsehoods-programmers-believe-a...
  - mrweasel 2y ago
    Amazing. For a e-commerce site I argued that we would save ourself a lot of trouble by simply making the address field one large text field, rather than attempt to making a form that would work for every country and city (looking at you Mannheim). But apparently that would make data analysis to complicated.
    - bojanz 2y ago
      There is a middle ground and some common patterns that can help. The address field names are fairly standardized[0] and Google has an open dataset (used by Chrome and Android) describing which countries need which fields[1]. I have an older PHP library[2] and a newer Go library[3] that build upon this, while crowdsourcing fixes (since Google hasn't updated their dataset in a while). The Go library allows me to serve all address formats and state lists in a single HTTP request, which can then power a very fast JS widget. [0] Initially by the OASIS eXtensible Address Language (xAL) which trickled down into everything from maps to HTML5 autocomplete. [1] https://chromium-i18n.appspot.com/ssl-address https://chromium-i18n.appspot.com/ssl-address [2] https://github.com/commerceguys/addressing https://github.com/commerceguys/addressing [3] https://github.com/bojanz/address https://github.com/bojanz/address

cjs_ac 2y ago

> The upshot of the research then, is that building an accurate database is really hard. OS concludes that it would have to check the 4.2m bad addresses manually to make its PAF-less database a viable dataset that would actually be useful. The secret to the Royal Mail's success with the PAF, and the reason why only the Royal Mail can maintain the PAF, is that the Royal Mail has people walking and driving to all those delivery points six days every week. Compare the Freedom of Information requests to Royal Mail from OpenStreetMap contributors concerning the locations of post boxes, which were refused ultimately because that information was handled only by local sorting offices.

chihuahua 2y ago
Maybe they can use the TV detector vans used for TV license enforcement to collect the data, if they're already surveilling every single building in the country on a daily basis!
- dambi0 2y ago
  Checking buildings you already know to have a license seems wasteful, but perhaps I missed something?

Normal_gaussian 2y ago

As censorship for FOIA requests is done manually, it may be beneficial to request the missing figures directly without noting you have them in a censored context. Censoring is subjective, so that would at least draw out either the figures or a justification.

nly 2y ago

Unfortunately the British mindset these days is to either rent it out or sell it but, whatever the hell you do, don't grow it. Somehow these idiots managed to strike a deal to keep the sovereigns figurehead on stamps (which has no economic value whatsoever, and actually the Crown should be compensated for this) but, in this data age, didn't safeguard such a critically important database to e-commerce It's like selling off the Tower of London because you can't afford to repair the roof and forgetting you left the crown jewels inside

TechTechTech 2y ago

For comparison, in the Netherlands all postcode data is open data, including detailed building outlines as well as almost all other related information. See https://app.pdok.nl/viewer https://app.pdok.nl/viewer for most datasets.

Muromec 2y ago
This creates a very special Dutch thing —- my neighborhood had the roads on the map before the map itself was updated to show landmass instead of the body of water.
- anticensor 2y ago
  Same in Turkey, except the map data is subject to certain limitations.
- DonHopkins 2y ago
  I wonder if all the houses on disconnected long islands without roads in Vinkeveense Plassen have postal codes? It's hard to get a pizza delivered there. https://www.google.com/maps/@52.2307079,4.9365182,1869m/data=!3m1!1e3?hl=en&entry=ttu&g_ep=EgoyMDI0MDgyMC4xIKXMDSoASAFQAw%3D%3D https://www.google.com/maps/@52.2307079,4.9365182,1869m/data...
  - jorams 2y ago
    In the PDOK viewer linked above you can enable the "Adressen" layer[1] and it will show markers on everything that has an address. Everything that has an address has a postal code, which is listed in the details if you click the address. (There might be an exception with an address but no postal code somewhere, I'm not sure, but not here.) [1]: https://app.pdok.nl/viewer/#x=124175.54&y=471068.96&z=11.2907&background=BRT-A%20standaard&layers=4074b3c3-ca85-45ad-bc0d-b5fca8540z0b;adressen https://app.pdok.nl/viewer/#x=124175.54&y=471068.96&z=11.290...
  - Muromec 2y ago
    That area looks so weird on a map and so cool in person. I never kinda understood what is going on there except the whole having a lake and being the Netherlands.

agolio 2y ago

I am a bit surprised by how hard this article makes out the problem to be. Crowdsourcing should make short work of the problem, with the right incentives, which the government will be able to offer. Additionally private map providers (e.g. Google, Apple) must surely have this data (since they are able to route navigation to private addresses). Why not just negotiate with them?

darrenf 2y ago
How would crowdsourcing solve this problem? > Oh, and it wouldn’t even be legally allowed to include, er, postcodes, as they are specifically owned by Royal Mail
- moring 2y ago
  How does OpenStreetMap solve it? OSM, more specifically OSM Nominatim, shows postcodes. Example: https://www.openstreetmap.org/search?lat=53.151778&lon=-1.168003#map=19/53.151779/-1.168003 https://www.openstreetmap.org/search?lat=53.151778&lon=-1.16...
  - darrenf 2y ago
    I didn't know the answer so I looked it up. Nominatim gets postcodes from Ordnance Survey: https://nominatim.org/release-docs/3.4/data-sources/GB-Postcodes/ https://nominatim.org/release-docs/3.4/data-sources/GB-Postc... Specifically Code-Point Open: https://www.ordnancesurvey.co.uk/products/code-point-open https://www.ordnancesurvey.co.uk/products/code-point-open which is updated quarterly, and in turn gets the postcodes from Royal Mail.
    - moring 2y ago
      It seems to me that you can download the postcode list freely: https://osdatahub.os.uk/downloads/open/CodePointOpen https://osdatahub.os.uk/downloads/open/CodePointOpen Something is missing here. If OS already has that data from RM and can make it available freely, why would they need to build another database?

andrewjl 2y ago

There's a writeup linked to in the OP comments about how this can happen. https://www.centreforpublicdata.org/blog/freeing-the-paf-our-meeting-with-jonathan-reynolds-on-address-data https://www.centreforpublicdata.org/blog/freeing-the-paf-our...

IshKebab 2y ago

Off topic but this is a bizarrely weird take: > Sadly because of the NIMBYs, this map doesn’t include a London version of The Sphere. "NIMBY" implies they're objecting to something useful and not actually that bad, like a solar farm or a mobile phone mast or a housing estate. Not a giant advertising billboard.

lol768 2y ago
I think it's a joke.

robinhouston 2y ago

This is a long-running battle. Those with long memories may remember the skirmish 15 years ago, when a small group of developer-activists set up a website that allowed free access to postcode data (ernestmarples.com, named after the inventor of the modern British postcode system). Needless to say, it was rapidly shut down following threats of legal action by Royal Mail. https://www.theguardian.com/technology/2009/oct/05/ernest-marples-royal-mail-postcodes-legal-threat https://www.theguardian.com/technology/2009/oct/05/ernest-ma... https://blog.okfn.org/2009/10/05/ernest-marples-uk-postcode-site-has-been-taken-down/ https://blog.okfn.org/2009/10/05/ernest-marples-uk-postcode-...

michaelt 2y ago
The postcode-to-coordinate data is now freely available as "CodePoint Open" So there's already data for people who want to know postcode AB10 1JL corresponds to the area around 57.14677,-2.09873 The PAF is a more detailed data source, as seen in https://www.royalmail.com/find-a-postcode https://www.royalmail.com/find-a-postcode which can tell you that AB10 1JL specifically covers the addresses 102-104, Union Street, Aberdeen 82, Union Street, Aberdeen Timpson Shoe Repairs Ltd, 86 Union Street, Aberdeen Smart Mobile, 88 Union Street, Aberdeen 92 Union Street, Aberdeen 98 Union Street, Aberdeen The PAF is useful if you want to provide a "quick address entry" option on your website - and to validate address data. But if you just want postcode-to-location conversion, that info is already available.
- robinhouston 2y ago
  Thanks for the clarification. I’d forgotten that ernestmarples only offered postcode-to-location lookup: it was a long time ago. I suppose this is encouraging! It shows that the forces of openness are gaining ground in this battle.
- sahmeepee 2y ago
  Given that OSM has a lot of building addresses and locations (at least it has building names and numbers and the streets they are on, plus their outlines) I wonder how much of the PAF could be recreated from the data in CodePoint Open combined with OSM. If adding more house numbers etc to OSM would help improve that then it could be crowdsourced quite quickly via Street-Complete or similar. For the purposes of the quick address lookup it doesn't even need to be 100% accurate. If a property is incorrectly listed against two postcodes it won't really cause problems, so the lookup could err on the generous side

NeoTar 2y ago

Some context, for people not located in the UK - A full British postcode typically aims to cover around 15 buildings (sometime a single building, sometimes a street of 50 houses). This is in contrast to many other postal code systems which cover relatively broad areas). Or to put it another way - UK - 2,643,732 codes, 1 code per 25 people, USA - 41,700 codes, 1 code per 8000 people, Germany - 8,200 codes, 1 code per 10000 people, This means that post-codes are often used as a proxy for an exact location - e.g. if I am going to visit a relative, I can enter their postcode into my sat-nav, and be confident that most of the time I'll get to within a hundred meters of their location. This doesn't work so well in rural area or on large estates where the access point may different from the location, leading to places sometimes advertising a different postcode to put into your sat-nav (e.g. of where the site entrance is) to that of the location itself.

simonbarker87 2y ago
Hence why a house number and postcode constitutes a complete address in the UK, we’ve sort of already got What Three Words with “a number and 5-7 characters” - not quite as catchy though
- beardyw 2y ago
  We needed an ambulance off road in the middle of Richmond Park where a postcode would also not help. We didn't have WTW either, which they asked for and would have helped immensely.
  - sideshowb 2y ago
    If only we had a system of national grid references since, say, 1936
    - beardyw 2y ago
      They were only interested in What Three Words. Didn't want anything else, sadly.

maccard 2y ago

At least the Uk has the the defence that postcodes are 60 years old and that the legacy cruft that comes with that is part of life. Meanwhile Ireland introduced Éircodes less than 10 years ago, chose an opaque format that uses a central database that you have to pay for access to for anything more than a handful of lookups, only covers homes (so you can’t give an eircode of a park, or a walk). It’s pretty much what you’d expect to be designed by a modern government.

chgs 2y ago
I’d expect a modern government to design something as clear and well regarded as the GDS stuff in the U.K. I’d expect a corporation like ibm etc to design the total mess we see with any large project
- jetbooster 2y ago
  Sadly I feel GDS is more of an outlier than the rule.
  - willyt 2y ago
    Transport for London is a pretty tightly run ship. Only capital city in the world that doesn't receive operating subsidy for its public transport. Not that that is a good thing necessarily as the tube is expensive to use relative to Paris or Berlin but a pretty impressive achievement considering the ancient complexity of the whole thing. Scotrail is run by the Scottish government and has been steadily electrifying the Scottish rail network and because of the slow and steady nature of the work, between them, Network rail and the OHLE contractors they have got the cost for this down to 5 times less per km than typical UK costs previously e.g. the great western main line. The moral of the story is get good people, give them stability and a clear goal and they will do great work. It doesn't really matter if they are working for the government or the private sector.
7952 2y ago
Postcode data is already freely available. You can even get coordinates of where the properties are. What is lacking is the actual addresses within a postcode.

wiredfool 2y ago

Essentially the same deal in Ireland, with Eircodes. They were originally created as private dataset with ownership, and now you have to license access to it to use it. Eircodes are better than postcodes, in that there's 1 per building/address/apartment, however they're discontinuous, so adjacent buildings will have distinctly different eircodes. The article highlighted the difficulty of shopping centers and apartment buildings, from my experience trying to validate a large number of Eircode <-> addresses for a project, this is definitely an issue. The worse issue is that there's no way to just send someone out to check, because the eircode isn't like a house number that's posted somewhere. (Leaving aside the problem that valid Irish addresses can have no numbers outside of the eircode, and eircodes are a recent, and therefore non-traditional addition)

NeoTar 2y ago
I was impressed when I first heard about the objectives of the Eircode system, but it seems the implementation is lacking.
- wiredfool 2y ago
  The implementation was captured by a private party.
closewith 2y ago
Eircodes also aren't used by An Post, to add insult to injury. > The worse issue is that there's no way to just send someone out to check, because the eircode isn't like a house number that's posted somewhere. (Leaving aside the problem that valid Irish addresses can have no numbers outside of the eircode, and eircodes are a recent, and therefore non-traditional addition) The HSE National Ambulance Service (NAS) National Emergency Operations Centres (NEOCs) have a GIS package that resolves Eircodes (and other traditional and colloquial addresses) to actual buildings and building entrances in real-time, which actually quite impressive. The directions can be transmitted to ambulances and other assets in real-time and has reduces delays in clinical services due to address confusion enormously since 2016. So the country is capable. Eircode is what we chose as a country, not what we were limited to.

ascorbic 2y ago

As he points out, this was a profoundly stupid mistake made when privatising Royal Mail. It would have been trivially easy to do at that point, but now it's a lot harder. If the government decided that it does want to do this, it can't just pass a law that says "the PAF is now free" without paying hundreds of millions of pounds in compensation to Royal Mail. That's quite apart from the ongoing costs of maintaining the data. At a time of cuts of budgets this would be a hard sell.

knallfrosch 2y ago
> If the government decided that it does want to do this, it can't just pass a law that says "the PAF is now free" without paying hundreds of millions of pounds in compensation to Royal Mail. You can pass the law, get sued and pay whatever the PAF is worth. But that's just.. fair? The govnerment spent 5 mio just for a survey concluding that it's impossible to recreate the PAF. So hundreds of millions sounds like a good deal.
- scott_w 2y ago
  Just to be clear: UK Parliament is sovereign. If it passes a law forcibly legalising it, the privatised Royal Mail can sue the government but would need to find an international treaty obligation to win. Even then, if Parliament flagged it and said "we're ignoring this treaty in this case" then the courts are bound to the law, not treaty obligations. If it has knock-on impacts in other areas, it's hard to say, but that's separate to the law.
  - ascorbic 2y ago
    Article 1 of Protocol 1 of the ECHR covers exactly this, so the Supreme Court (and ECtHR if it came to that) would probably find in favour of Royal Mail if this were to be done without compensation.
    - jjmarr 2y ago
      The court would make a "declaration of incompatibility" with the ECHR which leaves it up to Parliament to change the law. https://en.wikipedia.org/wiki/Declaration_of_incompatibility https://en.wikipedia.org/wiki/Declaration_of_incompatibility

intellix 2y ago

it would be nice if the postal system in the UK and anywhere in the world supported what3words to be honest

NeoTar 2y ago
Replacing one proprietary database with another? Is that truly useful?
manarth 2y ago
Replace one proprietary format owned by a private organisation with another proprietary format owned by a different private organisation?
- duncans 2y ago
  Plus, fraught with usability issues https://cybergibbons.com/security-2/why-what3words-is-not-suitable-for-safety-critical-applications/ https://cybergibbons.com/security-2/why-what3words-is-not-su...

askvictor 2y ago

Our company started operating in the UK recently, and some of our customers were very surprised we didn't charge for a subscription for part of our product. The idea would have no legs in Australia (our homeland) but is completely normal in the UK. So, new revenue stream for us, and some learnings about the UK culture.

lewispollard 2y ago
It is, but at least in my experience, we do it for the 10% discount and then immediately cancel the subscription every time we want to make a purchase.

londons_explore 2y ago

The simple solution here is a threat from the government to Royal Mail. Give us your postcode file for free, or we will simply make up a new numbering scheme, send an address card to every house telling them of their new number with their next council tax bill, and postcodes will become a thing of the past. The new numbering scheme will be unique to each house too, and have a check digit so the number alone is sufficient for 3rd party logistics companies like Amazon to use it for deliveries.

n4r9 2y ago
The idea of the UK government attempting to do such a thing fills me with the utmost dread.
- DaiPlusPlus 2y ago
  When did this almost Reaganite sentiment ("I'm from the government and I'm here to help") make home in the UK? I know it's not recent: I remember similar arguments coming from the No2ID camp in 2005 at-least.
  - pjc50 2y ago
    Quite a lot of it is Reaganism, via Thatcher. Probably dates from the Winter of Discontent. It's not entirely without merit, but only because there's a tendency to drastically underfund and micromanage state services. And things like the Post Office Horizon fiasco do not make the government look good here. On the other hand GDS is excellent - but that's almost entirely as a result of staff professionalism, rather than being driven by whichever ministers had the leadership of the civil service. An odd outcome of the ID discourse is that we now have an extremely high tech biometric identity system .. but only for immigrants.
    - willyt 2y ago
      'The Post Office' is a private company. Wasn't the Horizon system implemented after privatisation?

darau1 2y ago

I hope this happens. I can only dream of the day when my country gets something like this.

librasteve 2y ago

great article, this demonstrates just how bad the civil service & politicians are when it comes to negotiating contracts with private investors… or trade deals, or brexit if it comes to that

deleted 2y ago

[deleted]

sschueller 2y ago

In Switzerland anyone can use the national database of addresses. https://www.swisstopo.admin.ch/en/official-directory-of-building-addresses https://www.swisstopo.admin.ch/en/official-directory-of-buil...

justinclift 2y ago

This doesn't seem correct: The problem is it’s not an easy dataset to get hold of, as it cost a lot of money. This is because the data has to be licensed from Royal Mail ... It seems to be talking about the National Statistics Postcode Lookup UK, which is officially published here: https://www.data.gov.uk/dataset/7ec10db7-c8f4-4a40-8d82-8921935b4865/national-statistics-postcode-lookup-uk https://www.data.gov.uk/dataset/7ec10db7-c8f4-4a40-8d82-8921... It's been there from at least 2017, which is when I first came across it. There are later version of the data set online too: https://open-geography-portalx-ons.hub.arcgis.com/datasets/ons::national-statistics-postcode-lookup-2021-census-may-2024-for-the-uk/about https://open-geography-portalx-ons.hub.arcgis.com/datasets/o... The license: https://www.ons.gov.uk/methodology/geography/licences https://www.ons.gov.uk/methodology/geography/licences Under the terms of the Open Government Licence and UK Government Licensing Framework (launched 30 September 2010), if you wish to use or re-use ONS material, whether commercially or privately, you may do so freely without a specific application for a licence, subject to the conditions of the Open Government Licence and the Framework. If you are reproducing ONS content you must include a source accreditation to ONS. If the article is talking about a different postcode address file though, then the above doesn't apply. ;)

jokethrowaway 2y ago
That's not the https://www.poweredbypaf.com/product/paf/ https://www.poweredbypaf.com/product/paf/
- justinclift 2y ago
  No worries. :)

jokethrowaway 2y ago

Or we can just start using https://what3words.com/ https://what3words.com/ and geolocation. I disagree with the report, I think it's feasible with a bit of creativity. The government also has this: https://www.data.gov.uk/dataset/091feb1c-aea6-45c9-82bf-768a15c65307/open-postcode-geo https://www.data.gov.uk/dataset/091feb1c-aea6-45c9-82bf-768a... We could also start with an imperfect solution, offer it as a free API (maybe even self-hosted and communicating with other services p2p) and wait for users to select or insert missing addresses, until we eventually converge to a good OSS database. If it's a single service being shared by everyone, you would need to insert your address once and then it would be part of the database forever, and you would get the right result at any other time in the future. There is also a dirty but hard to attack option: - Start from the NAG - Build an opaque AI process which is hard to audit and that is tuned until it produces a result close to PAF but with a few extra errors - Sell the new database to the government, government open sources that - Directors get paid their share - Company get sued out of existance by RoyalMail - Government pays a few millions in 20 years, if the RoyalMail experts can prove anything in court

MSFT_Edging 2y ago

It blows my mind how many public services have been privatized in the UK. It just feels like they're selling off the shoes they're standing in. When their railways got privatized, the service didn't improve, the price just ballooned. Even in the states, the USPS has resisted privatization this far. For the love of god I hope it continues to. Protect our boys n girls in blue and tell your congressman you want postal banking.

normangray 2y ago
Yup. The Post Office, the railways, the water system, for heavens' sake! The tories, as a matter of religious faith, see privatised => efficient, whilst being unclear on the difference between 'efficient at creating shareholder value' and 'efficient at serving the public good'. The political mood music, over the last few decades, has meant that the Labour party has repeatedly found itself obliged to say positive things about privatisation, as part of the process of Being Sensible About The Economy (there is a much longer alternative version of this comment!). The US -- the world temple of capitalism -- seems to be oddly principled (viewed from outside) about keeping certain things such as the postal service, or USGS, as part of the service to the public realm. The one service probably immune from privatisation is the Health Service. It's only the most frothing-at-the-mouth right-wingers, the provocateurs just one step away from a rabies injection, who'd even admit out loud to a desire to do that. A politician talking about privatising the NHS would I think be pretty much equivalent to a US libertarian politician talking about privatising the armed forces. (there's a longer version of that comment, as well...)
- johneth 2y ago
  Minor pedantic correction: The Post Office (shops, services, government forms, etc.) is still fully government-owned. It's the Royal Mail (delivery) that was privatised. They used to be the same company but split in two before Royal Mail was privatised.
  - normangray 2y ago
    Thank you! – yes, it's a private company, fully owned by the Government. I think I sort-of knew that, but it hadn't properly registered; I may indeed have been confusing it with Royal Mail. It means that the PO is quite closely analogous to the Ordnance Survey, in organisational terms. It's maybe describable as a quasi-privatisation, in that the company is run on a fully commercial basis, with correspondingly narrow goals in principle, but with the profits (or, in the case of the PO, the losses) going to the Exchequer, and the relevant minister (presumably) having some say about the appointment of board members. It's organisationally tidy, I suppose, and manages to fit in with the long-standing 'private=efficient' doctrine. It still feels vaguely off, to me; not quite cricket.

alexchamberlain 2y ago

To be clear, is the National Address Gazetteer open? As far as I can tell, it isn't, but I don't know if that's because they're trying to obfuscate it.

nickdothutton 2y ago

Recent history teaches that the Post Office should be the last company on earth to be anywhere near creating a nationally important IT system. Their technology team have been useless for decades.

dan-robertson 2y ago
The PAF is maintained by Royal Mail, a different company from the Post Office.

frereubu 2y ago

I've used https://postcodes.io/ https://postcodes.io/ on a number of projects. There's information about it here - https://postcodes.io/about https://postcodes.io/about - which doesn't fully answer where the data comes from, although it mentions OS, so I presume it's based on the OS AddressBase product? I also wonder how complete it is now, although the sites we've built haven't had any issues as far as I'm aware.

gorgoiler 2y ago

As the tenant, once, of a new build home in the UK it’s not just the file that’s important: it’s the channels to patch the file too. I spent almost a year having to enter my address manually because the postcode DB — or whatever old version pets.com, cameras.com, and looroll.com had — lagged behind the reality of my infill bungalow for seemingly forever. I’m 8A godamnit, not 8. (Thank you Mrs. No8 for accepting my packages throughout those dark months btw.) It’s just like tzdata. A precious resource not just because it compiles the history of geopolitical wallclock settings, but also because it is meticulously updated, on time and on budget. It’s all very well liberating PAF.v2024_08finalfinal_v3_final.doc, but who is also going to keep it up to date? I’m not a hater, just a realist. TFA is spot on: we’ll never be able out compete with or recreate or leak a sufficient version of the PAF. It, and it’s entire infrastructure, needs to be wrested into public hands ASAP. And we should be prepared to fund the updates.

sahmeepee 2y ago
I've been a consumer of the PAF for many years through work. In the bad old days (10 years ago), when we got it from a well known credit check company, it came on CDs. We then moved to "automatic" updates via their rubbish, buggy software to our on-premise install. That wasn't such a huge upgrade, because as soon as you looked away from that software it would stop working. Now we pay a different, altogether better company to access it via a web service. I would expect that the speed of updates through to websites is now mostly as good as the frequency of PAF releases. Unless this has changed recently, the PAF isn't created by Royal Mail itself, it's outsourced to a company called AFD based on the Isle of Man. My only complaint with the service we get now is that the value is quite poor - our total cost is somewhere around 30p per lookup on average for addresses which I suspect are rarely even used in our line of work.

mjevans 2y ago

They could also link tax filings and a Plus Code for physical location: https://maps.google.com/pluscodes/ https://maps.google.com/pluscodes/ https://github.com/google/open-location-code https://github.com/google/open-location-code

gregsadetsky 2y ago

Canada is in the same spot with the postal code data being for sale only. Relatedly and famously, Canada Post sued [0] geocoder.ca [1] years ago because it made available a crowdsourced database of postal codes... Years later, the case was settled [2] but yeah. You still have to pay for the "real" data. [0] https://www.michaelgeist.ca/2012/04/canada-post-geocoder-suit/ https://www.michaelgeist.ca/2012/04/canada-post-geocoder-sui... [1] https://geocoder.ca/ https://geocoder.ca/ [2] https://geocoder.ca/?sued=1 https://geocoder.ca/?sued=1 (nice url...)

ss64 2y ago

The PAF file only returns a 9% profit (based on a 2009 report) https://web.archive.org/web/20090324084021/http://www.psc.gov.uk/postcomm/live/policy-and-consultations/consultations/postcode-address-file--management-of-information/PAF_decision_doc.pdf https://web.archive.org/web/20090324084021/http://www.psc.go...

kwhitefoot 2y ago

Why bother paying attention to all this legal mumbo-jumbo? Just have someone exfiltrate the file and post it on Anna's Archive. Or extract them from Open Street Map. Just bypass the Royal Mail altogether. Or just ignore the postcodes. For most private individuals an ordinary street address works perfectly well and is needed anyway even if the postcode is provided.

sahmeepee 2y ago
It's mostly used to get clean address data on form inputs. The user thinks they are being given an easier way to enter their address, but really it's so that the PAF can be used to ensure the address lines are used consistently and there are no typos or other rubbish. If the person has to pick their address from a list after entering the post code it's also a data input check of sorts.