4 ms·

All of the contact data on this website is fetched from DNS TXT records. It's an example of how NUM [0] data can be used. A small team and I have created NUM b

All of the contact data on this website is fetched from DNS TXT records. It's an example of how NUM [0] data can be used. A small team and I have created NUM because we believe that paid, restricted, rate-limited APIs provided by the giants of the web are holding back developers and we want to change that. NUM provides: a simple way for businesses to provide machine-readable data direct to their users, bypassing data hoarding web giants; a forever free, unlimited and unrestricted source of data to developers. The general-purpose semantic web has failed because the standards (microdata, JSON-LD, Open Graph) are too complicated and difficult for small businesses to adopt and often forgotten about by marketing teams in large businesses because data is too difficult to edit. NUM is different because it can be adopted by any business using a simple online form. The protocol includes two DNS queries – the first to the authoritative DNS zone; the second failover query to our NUM Server, any business can claim their domain on NUM Server using a simple online process. The Company Directory website linked to is just one example of how NUM data can be used. The website fetches all data from DNS using client-side Cloudflare DNS-over-HTTPS calls (see Network tab in Dev Tools). Barclays, or any other company listed can update the data on this site by adopting the NUM protocol [1] independently in their own DNS – for example, to replace the data for the Barclays listing, they can create a TXT record at 1._num.barclays.co.uk or claim the record shown by visiting numserver.com/claim/barclays.co.uk DNS-based protocols like NUM (other examples are SPF and DMARC) typically suffer from the chicken-and-egg problem where no-one looks for this data in DNS because it's not there, and no one stores this data in DNS because no-one is looking for it. To help overcome this, we've pre-populated the DNS with contact data for large UK companies and are in the process of automatically gathering contact data for all *.uk domain names (we expect to complete this in the next 10 days) before moving on to all other UK companies, then companies worldwide (the US, Canada, Australia next). NUM can be adopted / edited with a simple online form. Anyone can create a record for their domain, either independently in their own DNS or using our service at https://app.numserver.com/tools/editor/add https://app.numserver.com/tools/editor/add To ensure that NUM is as efficient as possible, we store all data in DNS using MODL [2] – a compact, DNS-friendly data serialisation format; we make compact DNS objects developer friendly using Unpacker [3] – we have developed both in-house. I have written a very basic Notion document to explain how all of these technologies fit together [4]. To simplify all of this for front-end developers (who by and large don't care about DNS), we've packaged it up in a Typscript library called company-api which you can query for a domain and get a beautiful object back containing company contact data [5]. We are very keen for feedback, good or bad. 0. https://www.num.uk https://www.num.uk 1. https://www.numprotocol.com https://www.numprotocol.com 2. https://www.modl.uk https://www.modl.uk 3. https://www.unpacker.uk https://www.unpacker.uk 4. https://www.notion.so/num/NUM-MODL-and-Unpacker-67d7cd59548d4eafb4852d0d89ff9493 https://www.notion.so/num/NUM-MODL-and-Unpacker-67d7cd59548d... 5. https://www.npmjs.com/package/company-api https://www.npmjs.com/package/company-api Edit: to make it clearer what problem we're trying to solve.

tptacek 5y ago
I guess I don't understand how DNS records work around the problem of metered APIs, since DNS providers are not going to let you store e.g. every item of your inventory in separate TXT records. Also: DNS is much, much slower than an HTTP API, which can fetch arbitrary, specific collections of records for a single query. I'm not trying to shoot this down so much as figure out what I'm missing here.
- nightfly 5y ago
  > since DNS providers are not going to let you store e.g. every item of your inventory in separate TXT records They aren't?
  - bawolff 5y ago
    I can't imagine dns providers would be cool with you storing a couple gb of records. You could of course host your own dns server... but you could also just host your own website.
    - elliottinvent 5y ago
      There are pretty standard industry limits in Cloud DNS (AWS, Microsoft, Google): 10,000 DNS zones per customer 10,000 resource records per zone Both can be increased by request At scale, query costs work out at $200 USD to answer 1 billion uncached queries, depending on TTLs that might actually deliver tens or hundreds of billions of answers to users. It's possible DNS providers and DNS resolvers (like Cloudflare, Google etc) might put limits in place. We're hoping not. Since TXT records are best delivered when below 5kb, it's pretty tough to store a couple of gb of data in your domain using NUM but certainly possible. If NUM is a big success, DNS providers might decide to charge for DNS data transfer rather than individual queries. Edit: line spacing
- elliottinvent 5y ago
  Thanks for your comment. > DNS providers are not going to let you store e.g. every item of your inventory in separate TXT records There are some industry-standard limits in Cloud DNS (10k zones per customer, 10k resource records [e.g. TXT] per zone). These limits would make it difficult to store a large inventory in DNS. NUM could be used for inventory but as you say, an API would be more effective. NUM is designed for standardised use cases like contact data, where all companies store their data in the same way – making it easy for developers to consume. We're trying to standardise this in DNS, instead of an API, for two reasons: 1. Any domain registrant can adopt it, relatively simply (although some providers have atrocious TXT record management) – they can delegate their NUM zone to us for easy management. 2. By using DNS, we can pre-populate huge amounts of this data (we do this from num.net) which makes the protocol useful from launch and helps overcome the chicken-and-egg problem. DNS caching helps us answer these queries, we can answer a billion DNS queries for $200 using standard Cloud DNS pricing. Once caching is factored in it's incredibly efficient. It's possible DNS resolvers might refuse to cache (or even answer) NUM queries but we hope that doesn't happen. > I'm not trying to shoot this down so much as figure out what I'm missing here. I appreciate that, hopefully my answers above have helped

tptacek 5y ago

I guess I don't understand how DNS records work around the problem of metered APIs, since DNS providers are not going to let you store e.g. every item of your inventory in separate TXT records. Also: DNS is much, much slower than an HTTP API, which can fetch arbitrary, specific collections of records for a single query. I'm not trying to shoot this down so much as figure out what I'm missing here.

nightfly 5y ago
> since DNS providers are not going to let you store e.g. every item of your inventory in separate TXT records They aren't?
- bawolff 5y ago
  I can't imagine dns providers would be cool with you storing a couple gb of records. You could of course host your own dns server... but you could also just host your own website.
  - elliottinvent 5y ago
    There are pretty standard industry limits in Cloud DNS (AWS, Microsoft, Google): 10,000 DNS zones per customer 10,000 resource records per zone Both can be increased by request At scale, query costs work out at $200 USD to answer 1 billion uncached queries, depending on TTLs that might actually deliver tens or hundreds of billions of answers to users. It's possible DNS providers and DNS resolvers (like Cloudflare, Google etc) might put limits in place. We're hoping not. Since TXT records are best delivered when below 5kb, it's pretty tough to store a couple of gb of data in your domain using NUM but certainly possible. If NUM is a big success, DNS providers might decide to charge for DNS data transfer rather than individual queries. Edit: line spacing
elliottinvent 5y ago
Thanks for your comment. > DNS providers are not going to let you store e.g. every item of your inventory in separate TXT records There are some industry-standard limits in Cloud DNS (10k zones per customer, 10k resource records [e.g. TXT] per zone). These limits would make it difficult to store a large inventory in DNS. NUM could be used for inventory but as you say, an API would be more effective. NUM is designed for standardised use cases like contact data, where all companies store their data in the same way – making it easy for developers to consume. We're trying to standardise this in DNS, instead of an API, for two reasons: 1. Any domain registrant can adopt it, relatively simply (although some providers have atrocious TXT record management) – they can delegate their NUM zone to us for easy management. 2. By using DNS, we can pre-populate huge amounts of this data (we do this from num.net) which makes the protocol useful from launch and helps overcome the chicken-and-egg problem. DNS caching helps us answer these queries, we can answer a billion DNS queries for $200 using standard Cloud DNS pricing. Once caching is factored in it's incredibly efficient. It's possible DNS resolvers might refuse to cache (or even answer) NUM queries but we hope that doesn't happen. > I'm not trying to shoot this down so much as figure out what I'm missing here. I appreciate that, hopefully my answers above have helped