13 ms·

Show HN: DNS-powered website with no back end

83 points by elliottinvent 5y ago

elliottinvent 5y ago
All of the contact data on this website is fetched from DNS TXT records. It's an example of how NUM [0] data can be used. A small team and I have created NUM because we believe that paid, restricted, rate-limited APIs provided by the giants of the web are holding back developers and we want to change that. NUM provides: a simple way for businesses to provide machine-readable data direct to their users, bypassing data hoarding web giants; a forever free, unlimited and unrestricted source of data to developers. The general-purpose semantic web has failed because the standards (microdata, JSON-LD, Open Graph) are too complicated and difficult for small businesses to adopt and often forgotten about by marketing teams in large businesses because data is too difficult to edit. NUM is different because it can be adopted by any business using a simple online form. The protocol includes two DNS queries – the first to the authoritative DNS zone; the second failover query to our NUM Server, any business can claim their domain on NUM Server using a simple online process. The Company Directory website linked to is just one example of how NUM data can be used. The website fetches all data from DNS using client-side Cloudflare DNS-over-HTTPS calls (see Network tab in Dev Tools). Barclays, or any other company listed can update the data on this site by adopting the NUM protocol [1] independently in their own DNS – for example, to replace the data for the Barclays listing, they can create a TXT record at 1._num.barclays.co.uk or claim the record shown by visiting numserver.com/claim/barclays.co.uk DNS-based protocols like NUM (other examples are SPF and DMARC) typically suffer from the chicken-and-egg problem where no-one looks for this data in DNS because it's not there, and no one stores this data in DNS because no-one is looking for it. To help overcome this, we've pre-populated the DNS with contact data for large UK companies and are in the process of automatically gathering contact data for all *.uk domain names (we expect to complete this in the next 10 days) before moving on to all other UK companies, then companies worldwide (the US, Canada, Australia next). NUM can be adopted / edited with a simple online form. Anyone can create a record for their domain, either independently in their own DNS or using our service at https://app.numserver.com/tools/editor/add https://app.numserver.com/tools/editor/add To ensure that NUM is as efficient as possible, we store all data in DNS using MODL [2] – a compact, DNS-friendly data serialisation format; we make compact DNS objects developer friendly using Unpacker [3] – we have developed both in-house. I have written a very basic Notion document to explain how all of these technologies fit together [4]. To simplify all of this for front-end developers (who by and large don't care about DNS), we've packaged it up in a Typscript library called company-api which you can query for a domain and get a beautiful object back containing company contact data [5]. We are very keen for feedback, good or bad. 0. https://www.num.uk https://www.num.uk 1. https://www.numprotocol.com https://www.numprotocol.com 2. https://www.modl.uk https://www.modl.uk 3. https://www.unpacker.uk https://www.unpacker.uk 4. https://www.notion.so/num/NUM-MODL-and-Unpacker-67d7cd59548d4eafb4852d0d89ff9493 https://www.notion.so/num/NUM-MODL-and-Unpacker-67d7cd59548d... 5. https://www.npmjs.com/package/company-api https://www.npmjs.com/package/company-api Edit: to make it clearer what problem we're trying to solve.
- tptacek 5y ago
  I guess I don't understand how DNS records work around the problem of metered APIs, since DNS providers are not going to let you store e.g. every item of your inventory in separate TXT records. Also: DNS is much, much slower than an HTTP API, which can fetch arbitrary, specific collections of records for a single query. I'm not trying to shoot this down so much as figure out what I'm missing here.
  - nightfly 5y ago
    > since DNS providers are not going to let you store e.g. every item of your inventory in separate TXT records They aren't?
    - bawolff 5y ago
      I can't imagine dns providers would be cool with you storing a couple gb of records. You could of course host your own dns server... but you could also just host your own website.
      - elliottinvent 5y ago
        There are pretty standard industry limits in Cloud DNS (AWS, Microsoft, Google): 10,000 DNS zones per customer 10,000 resource records per zone Both can be increased by request At scale, query costs work out at $200 USD to answer 1 billion uncached queries, depending on TTLs that might actually deliver tens or hundreds of billions of answers to users. It's possible DNS providers and DNS resolvers (like Cloudflare, Google etc) might put limits in place. We're hoping not. Since TXT records are best delivered when below 5kb, it's pretty tough to store a couple of gb of data in your domain using NUM but certainly possible. If NUM is a big success, DNS providers might decide to charge for DNS data transfer rather than individual queries. Edit: line spacing
  - elliottinvent 5y ago
    Thanks for your comment. > DNS providers are not going to let you store e.g. every item of your inventory in separate TXT records There are some industry-standard limits in Cloud DNS (10k zones per customer, 10k resource records [e.g. TXT] per zone). These limits would make it difficult to store a large inventory in DNS. NUM could be used for inventory but as you say, an API would be more effective. NUM is designed for standardised use cases like contact data, where all companies store their data in the same way – making it easy for developers to consume. We're trying to standardise this in DNS, instead of an API, for two reasons: 1. Any domain registrant can adopt it, relatively simply (although some providers have atrocious TXT record management) – they can delegate their NUM zone to us for easy management. 2. By using DNS, we can pre-populate huge amounts of this data (we do this from num.net) which makes the protocol useful from launch and helps overcome the chicken-and-egg problem. DNS caching helps us answer these queries, we can answer a billion DNS queries for $200 using standard Cloud DNS pricing. Once caching is factored in it's incredibly efficient. It's possible DNS resolvers might refuse to cache (or even answer) NUM queries but we hope that doesn't happen. > I'm not trying to shoot this down so much as figure out what I'm missing here. I appreciate that, hopefully my answers above have helped
nopcode 5y ago
Isn’t DNS de-facto centralised too?
- elliottinvent 5y ago
  I think you could argue DNS over HTTPS is centralised (for now) with Cloudflare, Quad9, Google making up the main players. But I think the DNS is the original decentralised system due to the amount of players in the domain registrar / DNS service provider / ISP [DNS resolver] space.
- nine_k 5y ago
  Yes in the sense that DNS is a tree with a single root. No in the sense that you can put your data into leaves of your choice and under your control. You can easily run your own DNS server. Halfway-yes in the sense that this thing uses HTTPS access to DNS which has few providers now, and not the classic DNS because UDP from a browser is hard, and even arbitrary TCP from a browser is hard. I think that the use of DNS is a bit of a gimmick on this demo site; it could be more practically used in more interesting ways. A demo site should be and is very straightforward. But the idea of storing semantic-web-style information in the DNS looks interesting to me, and the suggested simplified format may be worth a look. The service NUM is offering is also interesting, and one of the key parts of it is that it's not storing your data, so you are not beholden to it. It helps make them accessible, but you store the data yourself.
  - tptacek 5y ago
    Ok, but, you can easily run your own Postgres server too, right?
    - nine_k 5y ago
      Its data won't be cached by any number of caching DNS resolvers. If your DNS server has 90% uptime, but your domain if often resolved, clients may not notice anything; if your database has 90% uptime, it's pretty noticeable. It of course only applies if the TTL of your records is long enough, but usually you don't need short TTL for TXT records of such kind.
darkr 5y ago
Sounds similar to Telnic https://datatracker.ietf.org/doc/html/draft-reid-dnsext-nkey-00 https://datatracker.ietf.org/doc/html/draft-reid-dnsext-nkey... https://icannwiki.org/TelNic https://icannwiki.org/TelNic
- elliottinvent 5y ago
  Thanks for your comment and for highlighting this. As a summary for others: Telnic (and Telnames) allowed people to store contact data in the DNS of their .tel domain name. > Sounds similar to Telnic Telnic used NAPTR records and was only available on .tel domains, NUM uses TXT and can be used on any domain name, by publishing a "_num" zone. Telnic was also "just" for contact data. What we're trying to do with NUM is make it possible to publish any kind of data. Contact data as just one "module".
  - fredrivett 5y ago
    The other key difference here as far as I understand, is that Telnic required you to register .tel domains, you couldn't bring your own domain and store contact information against that.
o8r3oFTZPE 5y ago
The idea I had for this over 10 years ago was to modify djb's dnstxt to output raw HTML with a MIME header. tinydns allows one to store arbitrary data so I could put anything in a 512-byte DNS packet, including "text/html" (newlines and carriage returns). I could request tiny web pages from tinydns. There have been other similar things by other folks like putting deCSS code in DNS, Wikipedia data, audio files, etc. Now this might all sound silly, but since then 1. DNS packets are now massive and can carry much more data and 2. big companies are pushing a "next-gen" UDP-based protocol (reminds me of djb's CurveCP) for HTTP, serving HTML and other web junk. They also want to use this UDP-based protocol for other things, like DNS, eventually. Whatever. HTML in DNS worked great. Tiny pages that fit in a CPU cache.
- o8r3oFTZPE 5y ago
  Putting URLs in DNS. Could be useful.
rognjen 5y ago
I think it would be wise to slightly change the design so as not to look so similar to Companies House.
tedk-42 5y ago
Please no. CI systems like GitHub actions and CircleCI use to have unlimited CI minutes until people started abusing them for things they weren't intended for (i.e. crypto mining). I would hate if I had to pay for DNS services (or be forced into using an ISP one which does things like blacklist certain domains) just because people on the internet wanted to be 'cool' and show off how smort they are. This is why we can't have nice things.
- bawolff 5y ago
  I don't see why this application would be abusive. Chrome does much worse things trying to detect if you are behind a captive portal. Edit: oh i see, they arent using generic dns but cloudflare's specific DoH gateway. Guess that could maybe be problematic if everyone did that, although if any company would tolerate that it would be cloudflare.
- adflux 5y ago
  Your domain names are free?
  - tedk-42 5y ago
    Someone needs to host the DNS servers which deliver DNS results. Serverless deployment models under the cover still uses servers.
    - adflux 5y ago
      What I am saying is that you are already paying for DNS services
      - tedk-42 5y ago
        What I said was I didn't want to have to pay to do DNS lookups. E.g. if a client side DNS call was rate limited because DNS services like CloudFlare (1.1.1.1) or Google (8.8.8.8) got tired of caching data that was never suppose to be there. There are people who have joked you can use route53 as a database (Corey Quinn originally?). But taking a joke too far is irresponsible. Imagine having to pay a subscription to resolve DNS queries... We have a good thing with it and this idea is like a knife in it (like my free CI minutes and cryptominers example)
bawolff 5y ago
I don't get it. You have to host the static page somewhere. Why not host the data in the same place (as a json blob or whatever)? It seems like the implication is all the data in dns is static, so what's the benefit of adding dns to the mix - you already have to use something else to host some of your static resources, why not just have a single static host instead of two separate static hosting systems? (Yes in theory you could write a dns server to dynamically answer queries, but that doesn't seem like its what is being proposed here) Dont get me wrong, its cool you can access dns from a browser, but i dont think this is a compelling usecase (something like web torrent in the browser with magnet links in dns for a decentralized web, seems like it would make a good demo for this technique)
- elliottinvent 5y ago
  What we're trying to do here is standardise how machine-readable data is stored and retrieved, primarily for companies. Any of the companies on this page can adopt the standard and override the data on the website. So the point is, it's not static. Sure, at the moment all this data is coming from one DNS zone (num.net) but that's only a failover query location, first we query e.g. barclays.co.uk for this data, so Barclays can override (or remove) this data if they want to.
  - bawolff 5y ago
    So the interesting part here is not supposed to be querying dns data from the client side of a website, but a new standard for machine readable data in TXT records? I mean between microdata, rdf, etc there's been a lot of standards for machine readable data storage on the internet. Most fail not for technical reasons, but because there is an incentive mismatch, where the people who have to maintain such data get no benefit out of maintaining it so they don't bother. In this scheme you can't even google it, so why would a company bother to use this?
    - elliottinvent 5y ago
      > So the interesting part here is not supposed to be querying dns data from the client side of a website, but a new standard for machine readable data in TXT records? Correct > I mean between microdata, rdf, etc there's been a lot of standards for machine readable data storage on the internet. Most fail not for technical reasons, but because there is an incentive mismatch, where the people who have to maintain such data get no benefit out of maintaining it so they don't bother. I'd argue the inconsistent way these standards have been adopted (or not in many cases) is the reason they are not an effective, reliable way for developers to find machine-readable data about _any_ business. Since they are not a reliable way to find data, developers can't build anything with that data and there's little incentive for companies to keep it up to date. Currently, the only reason to adopt these standards is for SEO or to make your website look pretty when you share your website on Facebook / whatever. > In this scheme you can't even google it, so why would a company bother to use this? Google hold a lot of the data we're proposing to publish using NUM (contact data for example). The difference between our approaches is that we're going to make it freely available to developers, Google don't (not really). Google could consume NUM data like anyone else. As far as the protocol is concerned Google is the same as any other developer so it's a level playing field.
ksec 5y ago
>CompanyDirectory.UK is a NUM Technology [1] >What is NUM? >NUM is a DNS-based alternative to the World Wide Web for storing and retrieving structured data. The web is amazing but websites are built for browsing and are an inefficient way to find precise pieces of data like telephone numbers, bank details and more. [1] https://www.num.uk https://www.num.uk
yakshaving_jgt 5y ago
This is interesting. I run https://newbusinessmonitor.co.uk/ https://newbusinessmonitor.co.uk/ and I’m often asked if I can provide contact data for UK companies, but this is difficult because no contact data is published when a company registers. I wonder how you’ll be getting this data for smaller companies.
- elliottinvent 5y ago
  The long game is for companies to store NUM data when they register a domain, potentially through their DNS provider or web designer. Maybe even storing machine readable data instead of a website, since many small business websites are cookie-cutter anyway – contact info, directions, menu etc. Public data is available for most domain registries, e.g Nominet [0] and others [1] so we crawl new domains to discover contact data. It could be good to chat (email elliott[dot]brown@num.uk), we're in the process of mashing up Companies House data with our contact data in an effort to provide company numbers in NUM records too. 0. https://registrars.nominet.uk/uk-namespace/the-uk-zone-files/ https://registrars.nominet.uk/uk-namespace/the-uk-zone-files... 1. https://czds.icann.org/ https://czds.icann.org/
bfung 5y ago
In a cursory read through, I’m still not sure what problem NUM actually solves. It seems like it moves data storage into DNS TXT records and defines some parsers. What benefits does NUM have over html over http? One can use some semantic html tags to organize the data for parsing, making it roughly equivalent to NUM modules. If a NUM client is running w/o a newly developed NUM module, how does it parse that data and how is it better than html without semantic meaning?
- elliottinvent 5y ago
  > What benefits does NUM have over html over http? Web standards (JSON-LD, microdata, RDFa) are inefficient, since you have to download the entire HTML to find a particular piece of data like a phone number. Web standards are also not widely adopted enough for developers to build something on top of that data. For example, web standards can't be used to find a phone number for _any_ company with a phone number published to their website, since many small businesses don't mark up phone numbers with semantic web data. We're pre-populating NUM with millions of pieces of data so if a phone number is on the public website, it'll be in the NUM record for that domain. > One can use some semantic html tags to organize the data for parsing, making it roughly equivalent to NUM modules. Of course, the problem is that not enough websites do this (especially those of small business). > If a NUM client is running w/o a newly developed NUM module, how does it parse that data and how is it better than html without semantic meaning? If NUM is used for general purpose data storage (instead of for a particular module), then parsing of that data would need to be handled in a bespoke client – not a general purpose NUM client. Any advantages of using NUM over HTTP in this example but would be related to scaling and cost savings only.
  - lolinder 5y ago
    > Web standards (JSON-LD, microdata, RDFa) are inefficient, since you have to download the entire HTML to find a particular piece of data like a phone number. You might have picked a bad demo project to show off the efficiency of NUM. It takes me 58 requests and (on my very fast connection) 5.3 seconds to download all the information for Barclays. Downloading a JSON blob with all the data for Barclays, parsing it, and displaying it, would take a fraction of the time. Sure, NUM might be great if you knew ahead of time exactly which DNS record the user would need, but most cases where something like this would be helpful I'd rather see a complete list of all contact information so I can make the correct choice given all the options. Are there use cases you've thought of where the application would know exactly which DNS record to load and display, rather than displaying an index the way you do with companydirectory.uk?
unknown_error 5y ago
I think you're leeching off someone else's infrastructure and using it to do things they never meant it to do. Sure, the technical capability is there, but your use case would drastically increase their costs. You are essentially cost-shifting your customers' costs onto theirs. Not cool. It's like building a cloud storage solution off Gmail's free storage. It can be done, has been done, but that doesn't mean it's cool to do so. Your system would increase costs for DNS providers all over the world, without their consent, just because you're using it as a loophole. It was a problem that wasn't there fixed in a way that leeches from rather than gives back to the community.
- elliottinvent 5y ago
  Thanks for your point of view, I find it really interesting. > I think you're leeching off someone else's infrastructure Ok, who’s the victim here? CloudFlare? Since we use their DoH end point? Google Cloud DNS? Since that’s where we’re storing the data in DNS? All of this is just standard DNS - CloudFlare DoH and GCDNS can be switched out for any other because it’s just vanilla DNS. Let’s say Barclays wanted to serve out data using NUM and stored data in their own DNS zone, would they be abusing their DNS provider’s infrastructure? I don’t think so. If we’re successful with our plans for NUM, and it becomes mainstream then surely this presents a huge opportunity for DNS providers who will have increased query costs for clients. DNS revolvers will make their own decisions about whether they cache NUM queries (or perhaps even answer them at all) but revolvers that answer them quickly will surely have an edge on those that don’t. > and using it to do things they never meant it to do. The DNS is a distributed database. It’s designed to convert human friendly data to machine friendly data and I think NUM fits this perfectly. I understand not everyone shares my point of view. > Sure, the technical capability is there, but your use case would drastically increase their costs. You are essentially cost-shifting your customers' costs onto theirs. Not cool. It increases the costs of CloudFlare / Google? Ok, if it’s significant, they have a commercial decision to make - support full DNS as per the protocol spec, or partial-DNS where they block certain use cases. > It's like building a cloud storage solution off Gmail's free storage. It can be done, has been done, but that doesn't mean it's cool to do so. No, it’s not. The DNS is owned by no one and everyone. > Your system would increase costs for DNS providers all over the world, without their consent Most will just pass this on to domain owners, DNS query costs are peanuts - 200 USD per billion at scale. > It was a problem that wasn't there fixed in a way that leeches from rather than gives back to the community. I respect your point or view but think the opposite is true. We’re freeing data, opening it up for developers so that they can build things far outside the jurisdiction of the giants of the web - I think this is a fantastic way to give back to the community.
forgotmypw17 5y ago
Thank you for sharing your project, Elliott. It is an intriguing technology, and a clever way of putting together existing pieces in a new way. I think it is very much in the spirit of how things are done on the Internet, although I'm not entirely convinced that NUM wouldn't gum up the DNS system if widely adopted. My question to you is, how would I take advantage of NUM as a small-time personal-website operator, if my website was largely about small text-file snippets?
- elliottinvent 5y ago
  Thanks for your kind comment. The standardised use cases of NUM (currently contact data, images [gravatar for domains] and some others) would allow you as a small website operator to publish any public data that you wanted to be machine-readable. I think your question might be how to use NUM to serve out your small text-file snippets? If so, NUM could be used for that. NUM lookups are based on URIs, eg: num://numexample.com:1 In this example module 1 is contact data and lookups for this uri involve a query to the dns for: dig 1._num.numexample.com TXT If instead of publishing contact data you wanted to publish general purpose data you can do this using “module zero”. For example data could be published form the NUM URI num://numexample.com:0/foo The DNS location is: foo.0._num.numexample.com
chadxz 5y ago
If you are offering this to people for free, what is your company’s business model? Why centralize their info on a single domain instead of putting it on their own using a common format?
- elliottinvent 5y ago
  The data is free to developers and users. We offer a freemium service to businesses that they can use to publish and manage NUM data. We prepopulate data in our domain (NUM.net) so the protocol is useful from launch. The protocol allows anyone to publish data to their own DNS, they can also delegate their “_num” zone to us so we can easily manage the data. Our plan isn’t to centralise everyone’s data in one domain (num.net), but that’s a fallback. Realistically, many small businesses will use this hosted solution long term and our business model counts on that.

83 points by elliottinvent 5y ago

All of the contact data on this website is fetched from DNS TXT records. It's an example of how NUM [0] data can be used. A small team and I have created NUM because we believe that paid, restricted, rate-limited APIs provided by the giants of the web are holding back developers and we want to change that. NUM provides: a simple way for businesses to provide machine-readable data direct to their users, bypassing data hoarding web giants; a forever free, unlimited and unrestricted source of data to developers. The general-purpose semantic web has failed because the standards (microdata, JSON-LD, Open Graph) are too complicated and difficult for small businesses to adopt and often forgotten about by marketing teams in large businesses because data is too difficult to edit. NUM is different because it can be adopted by any business using a simple online form. The protocol includes two DNS queries – the first to the authoritative DNS zone; the second failover query to our NUM Server, any business can claim their domain on NUM Server using a simple online process. The Company Directory website linked to is just one example of how NUM data can be used. The website fetches all data from DNS using client-side Cloudflare DNS-over-HTTPS calls (see Network tab in Dev Tools). Barclays, or any other company listed can update the data on this site by adopting the NUM protocol [1] independently in their own DNS – for example, to replace the data for the Barclays listing, they can create a TXT record at 1._num.barclays.co.uk or claim the record shown by visiting numserver.com/claim/barclays.co.uk DNS-based protocols like NUM (other examples are SPF and DMARC) typically suffer from the chicken-and-egg problem where no-one looks for this data in DNS because it's not there, and no one stores this data in DNS because no-one is looking for it. To help overcome this, we've pre-populated the DNS with contact data for large UK companies and are in the process of automatically gathering contact data for all *.uk domain names (we expect to complete this in the next 10 days) before moving on to all other UK companies, then companies worldwide (the US, Canada, Australia next). NUM can be adopted / edited with a simple online form. Anyone can create a record for their domain, either independently in their own DNS or using our service at https://app.numserver.com/tools/editor/add https://app.numserver.com/tools/editor/add To ensure that NUM is as efficient as possible, we store all data in DNS using MODL [2] – a compact, DNS-friendly data serialisation format; we make compact DNS objects developer friendly using Unpacker [3] – we have developed both in-house. I have written a very basic Notion document to explain how all of these technologies fit together [4]. To simplify all of this for front-end developers (who by and large don't care about DNS), we've packaged it up in a Typscript library called company-api which you can query for a domain and get a beautiful object back containing company contact data [5]. We are very keen for feedback, good or bad. 0. https://www.num.uk https://www.num.uk 1. https://www.numprotocol.com https://www.numprotocol.com 2. https://www.modl.uk https://www.modl.uk 3. https://www.unpacker.uk https://www.unpacker.uk 4. https://www.notion.so/num/NUM-MODL-and-Unpacker-67d7cd59548d4eafb4852d0d89ff9493 https://www.notion.so/num/NUM-MODL-and-Unpacker-67d7cd59548d... 5. https://www.npmjs.com/package/company-api https://www.npmjs.com/package/company-api Edit: to make it clearer what problem we're trying to solve.

tptacek 5y ago
I guess I don't understand how DNS records work around the problem of metered APIs, since DNS providers are not going to let you store e.g. every item of your inventory in separate TXT records. Also: DNS is much, much slower than an HTTP API, which can fetch arbitrary, specific collections of records for a single query. I'm not trying to shoot this down so much as figure out what I'm missing here.
- nightfly 5y ago
  > since DNS providers are not going to let you store e.g. every item of your inventory in separate TXT records They aren't?
  - bawolff 5y ago
    I can't imagine dns providers would be cool with you storing a couple gb of records. You could of course host your own dns server... but you could also just host your own website.
    - elliottinvent 5y ago
      There are pretty standard industry limits in Cloud DNS (AWS, Microsoft, Google): 10,000 DNS zones per customer 10,000 resource records per zone Both can be increased by request At scale, query costs work out at $200 USD to answer 1 billion uncached queries, depending on TTLs that might actually deliver tens or hundreds of billions of answers to users. It's possible DNS providers and DNS resolvers (like Cloudflare, Google etc) might put limits in place. We're hoping not. Since TXT records are best delivered when below 5kb, it's pretty tough to store a couple of gb of data in your domain using NUM but certainly possible. If NUM is a big success, DNS providers might decide to charge for DNS data transfer rather than individual queries. Edit: line spacing
- elliottinvent 5y ago
  Thanks for your comment. > DNS providers are not going to let you store e.g. every item of your inventory in separate TXT records There are some industry-standard limits in Cloud DNS (10k zones per customer, 10k resource records [e.g. TXT] per zone). These limits would make it difficult to store a large inventory in DNS. NUM could be used for inventory but as you say, an API would be more effective. NUM is designed for standardised use cases like contact data, where all companies store their data in the same way – making it easy for developers to consume. We're trying to standardise this in DNS, instead of an API, for two reasons: 1. Any domain registrant can adopt it, relatively simply (although some providers have atrocious TXT record management) – they can delegate their NUM zone to us for easy management. 2. By using DNS, we can pre-populate huge amounts of this data (we do this from num.net) which makes the protocol useful from launch and helps overcome the chicken-and-egg problem. DNS caching helps us answer these queries, we can answer a billion DNS queries for $200 using standard Cloud DNS pricing. Once caching is factored in it's incredibly efficient. It's possible DNS resolvers might refuse to cache (or even answer) NUM queries but we hope that doesn't happen. > I'm not trying to shoot this down so much as figure out what I'm missing here. I appreciate that, hopefully my answers above have helped

nopcode 5y ago

Isn’t DNS de-facto centralised too?

elliottinvent 5y ago
I think you could argue DNS over HTTPS is centralised (for now) with Cloudflare, Quad9, Google making up the main players. But I think the DNS is the original decentralised system due to the amount of players in the domain registrar / DNS service provider / ISP [DNS resolver] space.
nine_k 5y ago
Yes in the sense that DNS is a tree with a single root. No in the sense that you can put your data into leaves of your choice and under your control. You can easily run your own DNS server. Halfway-yes in the sense that this thing uses HTTPS access to DNS which has few providers now, and not the classic DNS because UDP from a browser is hard, and even arbitrary TCP from a browser is hard. I think that the use of DNS is a bit of a gimmick on this demo site; it could be more practically used in more interesting ways. A demo site should be and is very straightforward. But the idea of storing semantic-web-style information in the DNS looks interesting to me, and the suggested simplified format may be worth a look. The service NUM is offering is also interesting, and one of the key parts of it is that it's not storing your data, so you are not beholden to it. It helps make them accessible, but you store the data yourself.
- tptacek 5y ago
  Ok, but, you can easily run your own Postgres server too, right?
  - nine_k 5y ago
    Its data won't be cached by any number of caching DNS resolvers. If your DNS server has 90% uptime, but your domain if often resolved, clients may not notice anything; if your database has 90% uptime, it's pretty noticeable. It of course only applies if the TTL of your records is long enough, but usually you don't need short TTL for TXT records of such kind.

darkr 5y ago

Sounds similar to Telnic https://datatracker.ietf.org/doc/html/draft-reid-dnsext-nkey-00 https://datatracker.ietf.org/doc/html/draft-reid-dnsext-nkey... https://icannwiki.org/TelNic https://icannwiki.org/TelNic

elliottinvent 5y ago
Thanks for your comment and for highlighting this. As a summary for others: Telnic (and Telnames) allowed people to store contact data in the DNS of their .tel domain name. > Sounds similar to Telnic Telnic used NAPTR records and was only available on .tel domains, NUM uses TXT and can be used on any domain name, by publishing a "_num" zone. Telnic was also "just" for contact data. What we're trying to do with NUM is make it possible to publish any kind of data. Contact data as just one "module".
- fredrivett 5y ago
  The other key difference here as far as I understand, is that Telnic required you to register .tel domains, you couldn't bring your own domain and store contact information against that.

o8r3oFTZPE 5y ago

The idea I had for this over 10 years ago was to modify djb's dnstxt to output raw HTML with a MIME header. tinydns allows one to store arbitrary data so I could put anything in a 512-byte DNS packet, including "text/html" (newlines and carriage returns). I could request tiny web pages from tinydns. There have been other similar things by other folks like putting deCSS code in DNS, Wikipedia data, audio files, etc. Now this might all sound silly, but since then 1. DNS packets are now massive and can carry much more data and 2. big companies are pushing a "next-gen" UDP-based protocol (reminds me of djb's CurveCP) for HTTP, serving HTML and other web junk. They also want to use this UDP-based protocol for other things, like DNS, eventually. Whatever. HTML in DNS worked great. Tiny pages that fit in a CPU cache.

o8r3oFTZPE 5y ago
Putting URLs in DNS. Could be useful.

rognjen 5y ago

I think it would be wise to slightly change the design so as not to look so similar to Companies House.

tedk-42 5y ago

Please no. CI systems like GitHub actions and CircleCI use to have unlimited CI minutes until people started abusing them for things they weren't intended for (i.e. crypto mining). I would hate if I had to pay for DNS services (or be forced into using an ISP one which does things like blacklist certain domains) just because people on the internet wanted to be 'cool' and show off how smort they are. This is why we can't have nice things.

bawolff 5y ago
I don't see why this application would be abusive. Chrome does much worse things trying to detect if you are behind a captive portal. Edit: oh i see, they arent using generic dns but cloudflare's specific DoH gateway. Guess that could maybe be problematic if everyone did that, although if any company would tolerate that it would be cloudflare.
adflux 5y ago
Your domain names are free?
- tedk-42 5y ago
  Someone needs to host the DNS servers which deliver DNS results. Serverless deployment models under the cover still uses servers.
  - adflux 5y ago
    What I am saying is that you are already paying for DNS services
    - tedk-42 5y ago
      What I said was I didn't want to have to pay to do DNS lookups. E.g. if a client side DNS call was rate limited because DNS services like CloudFlare (1.1.1.1) or Google (8.8.8.8) got tired of caching data that was never suppose to be there. There are people who have joked you can use route53 as a database (Corey Quinn originally?). But taking a joke too far is irresponsible. Imagine having to pay a subscription to resolve DNS queries... We have a good thing with it and this idea is like a knife in it (like my free CI minutes and cryptominers example)

bawolff 5y ago

I don't get it. You have to host the static page somewhere. Why not host the data in the same place (as a json blob or whatever)? It seems like the implication is all the data in dns is static, so what's the benefit of adding dns to the mix - you already have to use something else to host some of your static resources, why not just have a single static host instead of two separate static hosting systems? (Yes in theory you could write a dns server to dynamically answer queries, but that doesn't seem like its what is being proposed here) Dont get me wrong, its cool you can access dns from a browser, but i dont think this is a compelling usecase (something like web torrent in the browser with magnet links in dns for a decentralized web, seems like it would make a good demo for this technique)

elliottinvent 5y ago
What we're trying to do here is standardise how machine-readable data is stored and retrieved, primarily for companies. Any of the companies on this page can adopt the standard and override the data on the website. So the point is, it's not static. Sure, at the moment all this data is coming from one DNS zone (num.net) but that's only a failover query location, first we query e.g. barclays.co.uk for this data, so Barclays can override (or remove) this data if they want to.
- bawolff 5y ago
  So the interesting part here is not supposed to be querying dns data from the client side of a website, but a new standard for machine readable data in TXT records? I mean between microdata, rdf, etc there's been a lot of standards for machine readable data storage on the internet. Most fail not for technical reasons, but because there is an incentive mismatch, where the people who have to maintain such data get no benefit out of maintaining it so they don't bother. In this scheme you can't even google it, so why would a company bother to use this?
  - elliottinvent 5y ago
    > So the interesting part here is not supposed to be querying dns data from the client side of a website, but a new standard for machine readable data in TXT records? Correct > I mean between microdata, rdf, etc there's been a lot of standards for machine readable data storage on the internet. Most fail not for technical reasons, but because there is an incentive mismatch, where the people who have to maintain such data get no benefit out of maintaining it so they don't bother. I'd argue the inconsistent way these standards have been adopted (or not in many cases) is the reason they are not an effective, reliable way for developers to find machine-readable data about _any_ business. Since they are not a reliable way to find data, developers can't build anything with that data and there's little incentive for companies to keep it up to date. Currently, the only reason to adopt these standards is for SEO or to make your website look pretty when you share your website on Facebook / whatever. > In this scheme you can't even google it, so why would a company bother to use this? Google hold a lot of the data we're proposing to publish using NUM (contact data for example). The difference between our approaches is that we're going to make it freely available to developers, Google don't (not really). Google could consume NUM data like anyone else. As far as the protocol is concerned Google is the same as any other developer so it's a level playing field.

ksec 5y ago

>CompanyDirectory.UK is a NUM Technology [1] >What is NUM? >NUM is a DNS-based alternative to the World Wide Web for storing and retrieving structured data. The web is amazing but websites are built for browsing and are an inefficient way to find precise pieces of data like telephone numbers, bank details and more. [1] https://www.num.uk https://www.num.uk

yakshaving_jgt 5y ago

This is interesting. I run https://newbusinessmonitor.co.uk/ https://newbusinessmonitor.co.uk/ and I’m often asked if I can provide contact data for UK companies, but this is difficult because no contact data is published when a company registers. I wonder how you’ll be getting this data for smaller companies.

elliottinvent 5y ago
The long game is for companies to store NUM data when they register a domain, potentially through their DNS provider or web designer. Maybe even storing machine readable data instead of a website, since many small business websites are cookie-cutter anyway – contact info, directions, menu etc. Public data is available for most domain registries, e.g Nominet [0] and others [1] so we crawl new domains to discover contact data. It could be good to chat (email elliott[dot]brown@num.uk), we're in the process of mashing up Companies House data with our contact data in an effort to provide company numbers in NUM records too. 0. https://registrars.nominet.uk/uk-namespace/the-uk-zone-files/ https://registrars.nominet.uk/uk-namespace/the-uk-zone-files... 1. https://czds.icann.org/ https://czds.icann.org/

bfung 5y ago

In a cursory read through, I’m still not sure what problem NUM actually solves. It seems like it moves data storage into DNS TXT records and defines some parsers. What benefits does NUM have over html over http? One can use some semantic html tags to organize the data for parsing, making it roughly equivalent to NUM modules. If a NUM client is running w/o a newly developed NUM module, how does it parse that data and how is it better than html without semantic meaning?

elliottinvent 5y ago
> What benefits does NUM have over html over http? Web standards (JSON-LD, microdata, RDFa) are inefficient, since you have to download the entire HTML to find a particular piece of data like a phone number. Web standards are also not widely adopted enough for developers to build something on top of that data. For example, web standards can't be used to find a phone number for _any_ company with a phone number published to their website, since many small businesses don't mark up phone numbers with semantic web data. We're pre-populating NUM with millions of pieces of data so if a phone number is on the public website, it'll be in the NUM record for that domain. > One can use some semantic html tags to organize the data for parsing, making it roughly equivalent to NUM modules. Of course, the problem is that not enough websites do this (especially those of small business). > If a NUM client is running w/o a newly developed NUM module, how does it parse that data and how is it better than html without semantic meaning? If NUM is used for general purpose data storage (instead of for a particular module), then parsing of that data would need to be handled in a bespoke client – not a general purpose NUM client. Any advantages of using NUM over HTTP in this example but would be related to scaling and cost savings only.
- lolinder 5y ago
  > Web standards (JSON-LD, microdata, RDFa) are inefficient, since you have to download the entire HTML to find a particular piece of data like a phone number. You might have picked a bad demo project to show off the efficiency of NUM. It takes me 58 requests and (on my very fast connection) 5.3 seconds to download all the information for Barclays. Downloading a JSON blob with all the data for Barclays, parsing it, and displaying it, would take a fraction of the time. Sure, NUM might be great if you knew ahead of time exactly which DNS record the user would need, but most cases where something like this would be helpful I'd rather see a complete list of all contact information so I can make the correct choice given all the options. Are there use cases you've thought of where the application would know exactly which DNS record to load and display, rather than displaying an index the way you do with companydirectory.uk?

unknown_error 5y ago

I think you're leeching off someone else's infrastructure and using it to do things they never meant it to do. Sure, the technical capability is there, but your use case would drastically increase their costs. You are essentially cost-shifting your customers' costs onto theirs. Not cool. It's like building a cloud storage solution off Gmail's free storage. It can be done, has been done, but that doesn't mean it's cool to do so. Your system would increase costs for DNS providers all over the world, without their consent, just because you're using it as a loophole. It was a problem that wasn't there fixed in a way that leeches from rather than gives back to the community.

elliottinvent 5y ago
Thanks for your point of view, I find it really interesting. > I think you're leeching off someone else's infrastructure Ok, who’s the victim here? CloudFlare? Since we use their DoH end point? Google Cloud DNS? Since that’s where we’re storing the data in DNS? All of this is just standard DNS - CloudFlare DoH and GCDNS can be switched out for any other because it’s just vanilla DNS. Let’s say Barclays wanted to serve out data using NUM and stored data in their own DNS zone, would they be abusing their DNS provider’s infrastructure? I don’t think so. If we’re successful with our plans for NUM, and it becomes mainstream then surely this presents a huge opportunity for DNS providers who will have increased query costs for clients. DNS revolvers will make their own decisions about whether they cache NUM queries (or perhaps even answer them at all) but revolvers that answer them quickly will surely have an edge on those that don’t. > and using it to do things they never meant it to do. The DNS is a distributed database. It’s designed to convert human friendly data to machine friendly data and I think NUM fits this perfectly. I understand not everyone shares my point of view. > Sure, the technical capability is there, but your use case would drastically increase their costs. You are essentially cost-shifting your customers' costs onto theirs. Not cool. It increases the costs of CloudFlare / Google? Ok, if it’s significant, they have a commercial decision to make - support full DNS as per the protocol spec, or partial-DNS where they block certain use cases. > It's like building a cloud storage solution off Gmail's free storage. It can be done, has been done, but that doesn't mean it's cool to do so. No, it’s not. The DNS is owned by no one and everyone. > Your system would increase costs for DNS providers all over the world, without their consent Most will just pass this on to domain owners, DNS query costs are peanuts - 200 USD per billion at scale. > It was a problem that wasn't there fixed in a way that leeches from rather than gives back to the community. I respect your point or view but think the opposite is true. We’re freeing data, opening it up for developers so that they can build things far outside the jurisdiction of the giants of the web - I think this is a fantastic way to give back to the community.

forgotmypw17 5y ago

Thank you for sharing your project, Elliott. It is an intriguing technology, and a clever way of putting together existing pieces in a new way. I think it is very much in the spirit of how things are done on the Internet, although I'm not entirely convinced that NUM wouldn't gum up the DNS system if widely adopted. My question to you is, how would I take advantage of NUM as a small-time personal-website operator, if my website was largely about small text-file snippets?

elliottinvent 5y ago
Thanks for your kind comment. The standardised use cases of NUM (currently contact data, images [gravatar for domains] and some others) would allow you as a small website operator to publish any public data that you wanted to be machine-readable. I think your question might be how to use NUM to serve out your small text-file snippets? If so, NUM could be used for that. NUM lookups are based on URIs, eg: num://numexample.com:1 In this example module 1 is contact data and lookups for this uri involve a query to the dns for: dig 1._num.numexample.com TXT If instead of publishing contact data you wanted to publish general purpose data you can do this using “module zero”. For example data could be published form the NUM URI num://numexample.com:0/foo The DNS location is: foo.0._num.numexample.com

chadxz 5y ago

If you are offering this to people for free, what is your company’s business model? Why centralize their info on a single domain instead of putting it on their own using a common format?

elliottinvent 5y ago
The data is free to developers and users. We offer a freemium service to businesses that they can use to publish and manage NUM data. We prepopulate data in our domain (NUM.net) so the protocol is useful from launch. The protocol allows anyone to publish data to their own DNS, they can also delegate their “_num” zone to us so we can easily manage the data. Our plan isn’t to centralise everyone’s data in one domain (num.net), but that’s a fallback. Realistically, many small businesses will use this hosted solution long term and our business model counts on that.