4 ms·

You'd think DNS KV Store's a "nice figment of imagination", but one of the teams I worked for did use DNS TXT records genuinely as a (faster) way to reconcile s

You'd think DNS KV Store's a "nice figment of imagination", but one of the teams I worked for did use DNS TXT records genuinely as a (faster) way to reconcile state between the control plane and the data plane. It sounded ugly on-paper, but the design was approved by everyone in the chain of command for what I presume was for lack of other viable options given the architecture. And in ironic turn of events, instead of using DNS for service discovery, another set of engineers in the team opted for EC2 metadata (for example, security groups) instead.

OJFord 6y ago
I did something similar (for personal use) for a sort of 'peer discovery' where load balancing didn't actually matter, I just wanted to pick one of N without knowing where they are ahead of time, using IPVS: https://github.com/OJFord/ipvs-txt-lb https://github.com/OJFord/ipvs-txt-lb To be honest I'm not sure if it's a total hack or a reasonable use as intended, or somewhere in between. But it got it done, and is at least more elegant than hard coding something arbitrary which may change (just not too frequently), I think.
- tyingq 6y ago
  DNS SRV records seem appropriate for that.
alexfrydl 6y ago
Using TXT records for this kind of thing is not that unusual. For example, Kubernetes's external-dns uses TXT records to track which entries it manages.
strictfp 6y ago
I've used the AWS metadata trick as well. I kind of liked the solution; better use something that the machine is already relying on than adding something extra. DNS is also usually a much better option than the framework-specific service discovery mechanism that every framework seems to be destined to reinvent. I think the problem is that most computer scientists aren't very well-versed in networking, and DNS doesn't look like the best match at first glance.
- aECBNhWl4sqXr2 6y ago
  I've done the same - tag EBS volumes with metadata rather than having to maintain a separate datastore for a tiny amount of information. Worked well.
inopinatus 6y ago
It should not be surprising to anyone that long-established protocol standards are capable and versatile. DNS and LDAP are robust tree-structured replicated attribute stores; mix in Kerberos for a complete foundation of general purpose directory services. NNTP supplies a straightforward Gossip protocol for eventually-consistent distributed pub/sub. MQTT is still good for lightweight telemetry. I have services I authenticate with via Lamport's mostly forgotten scheme (S/KEY). And so on. There is a current fad of HTTP+JSON as a generic protocol substrate, but it's almost always a mediocre fit for the problem at hand. Go read the RFC archives, there's diamonds to be found. On the other hand, and perhaps even by the same token: using EC2 metadata isn't a completely terrible idea if it's the infrastructure you already have and the semantics line up with specific needs.

OJFord 6y ago

I did something similar (for personal use) for a sort of 'peer discovery' where load balancing didn't actually matter, I just wanted to pick one of N without knowing where they are ahead of time, using IPVS: https://github.com/OJFord/ipvs-txt-lb https://github.com/OJFord/ipvs-txt-lb To be honest I'm not sure if it's a total hack or a reasonable use as intended, or somewhere in between. But it got it done, and is at least more elegant than hard coding something arbitrary which may change (just not too frequently), I think.

tyingq 6y ago
DNS SRV records seem appropriate for that.

alexfrydl 6y ago

Using TXT records for this kind of thing is not that unusual. For example, Kubernetes's external-dns uses TXT records to track which entries it manages.

strictfp 6y ago

I've used the AWS metadata trick as well. I kind of liked the solution; better use something that the machine is already relying on than adding something extra. DNS is also usually a much better option than the framework-specific service discovery mechanism that every framework seems to be destined to reinvent. I think the problem is that most computer scientists aren't very well-versed in networking, and DNS doesn't look like the best match at first glance.

aECBNhWl4sqXr2 6y ago
I've done the same - tag EBS volumes with metadata rather than having to maintain a separate datastore for a tiny amount of information. Worked well.

inopinatus 6y ago

It should not be surprising to anyone that long-established protocol standards are capable and versatile. DNS and LDAP are robust tree-structured replicated attribute stores; mix in Kerberos for a complete foundation of general purpose directory services. NNTP supplies a straightforward Gossip protocol for eventually-consistent distributed pub/sub. MQTT is still good for lightweight telemetry. I have services I authenticate with via Lamport's mostly forgotten scheme (S/KEY). And so on. There is a current fad of HTTP+JSON as a generic protocol substrate, but it's almost always a mediocre fit for the problem at hand. Go read the RFC archives, there's diamonds to be found. On the other hand, and perhaps even by the same token: using EC2 metadata isn't a completely terrible idea if it's the infrastructure you already have and the semantics line up with specific needs.