5 ms·

Hey everybody, im the creator of dnsperf. I just wanted to clear some things up. 1. The point of the service is to have an objective way to compare different D

by jimaek 12y ago

Hey everybody, im the creator of dnsperf. I just wanted to clear some things up. 1. The point of the service is to have an objective way to compare different DNS services. dnsperf is supposed only to help you in your search for the best DNS service and not crown the absolute winner. 2. I plan to add as many locations as possible to get more reliable data. More locations will also solve the problem of "DNS at the same datacenter as my test nodes" 3. Some people are confused about the node locations. Here is the map http://www.dnsperf.com/network http://www.dnsperf.com/network 4. Its not open source but some time in the future it will be. 5. The tests I am running are supposed to exclude variables such as local/ISP caching, DNS proxies, resolvers, SRTT and so on. I wanted to test the nameservers directly and get the raw performance data of the provider himself. I believe thats the best way to do fair comparison. Thank you for your comments and love. I'm glad people liked the service :) Feel free to email me with your feedback or questions.

deleted 12y ago
[deleted]
Florin_Andrei 12y ago
The word "speed" usually suggests that more is better. What the graphs actually display is latency - in which case less is better. That was confusing for a few moments.
- jimaek 12y ago
  I guess I will need to fix that. thanks for the feedback
- windexh8er 12y ago
  Latency is a function of the speed of light - and in most engineering realms "speed" is well understood when using latency as a description. I had no qualms understanding what the creator meant, but I agree that the technical reality is the graph should be defined by "query latency" vs "query speed". Either way, fantastic data! Hopefully this data is trended over daily, weekly, monthly, quarterly, and yearly rollups. I see monthly when drilling into a provider, but that's all. A rolling annual graph would really tell a true story of how consistent a provider is and that, in itself, is very important when considering outsourcing DNS.
sandstrom 12y ago
Another way of gathering data would be a js-snippet that participating websites can include, which would run a random subset of the tests on some share of of visitors, and report back to your service. It'll be more realistic then DC -> DC queries. - http://www.w3.org/TR/resource-timing/ http://www.w3.org/TR/resource-timing/ - http://googledevelopers.blogspot.se/2013/12/measuring-network-performance-with.html http://googledevelopers.blogspot.se/2013/12/measuring-networ...
- jimaek 12y ago
  Its realistic but not consistent and reliable. Like I said I want to test their raw performance without any variables in between. All providers get tested directly from the exact same locations and times. Thats the data I wanted to get. It makes it easy and fair to compare.
- acdha 12y ago
  > It'll be more realistic then DC -> DC queries. That depends on what you're trying to measure. If you want to know what the total end-user experience is like, Resource Timing is awesome. If you're trying to compare DNS providers without noise from the differences in client implementations and ISP competency, however, it'll be very noisy and you need a massive number of samples to avoid seeing trends which are really just sampling artifacts. The best approach would be to do both so you could compare the server response times with the measured client values so you can get an idea whether a slow-down this week is caused by something you have any control over.
corford 12y ago
Hi jimaek! Thanks for putting this service together. For an African testing location maybe a VPS provider from ZA would work? Following are a few providers with Joburg or Cape Town presences (haven't personally used any though so YMMV): http://www.vpsnine.com/hosting/locations.php http://www.vpsnine.com/hosting/locations.php http://www.webafrica.co.za/hosting/vps/ http://www.webafrica.co.za/hosting/vps/ http://www.webnow.co.za/vps/ http://www.webnow.co.za/vps/ http://www.web-telecoms.co.za/vps/ http://www.web-telecoms.co.za/vps/ (posted these to the comments on your site but re-posting here in case you miss it).
- jimaek 12y ago
  Thank you, I will look into them.
rajacombinator 12y ago
This is cool and I've seen it before but I have a hard time interpreting the results. For instance, I have a hard time believing Google's DNS is substantially slower than anyone else's which leads me to question all the results. Also, and I'm hoping someone here can clear this up for me, but it's not clear to me how these results translate to actual DNS performance for my site. I only vaguely understand how the magic of the internet works but don't the DNS results get distributed throughout the network making the performance of the root servers less relevant? If I could chop 50-100ms off my TTFB by switching my DNS from Namecheap to someone else, I would definitely pay for that, but it's not clear to me.
- flipp3r 12y ago
  Why would you pay more for faster DNS? Most browsers come with DNS prefretching built in, I doubt it really matters for a website.
  - pixl97 12y ago
    Because the first page load is what matters most.
    - moe 12y ago
      Your DNS information should already be cached in all major ISPs DNS caches (unless you mean the first page load ever).
      - corford 12y ago
        How about the first subsequent load after the TTL for the previous load expires? Admittedly, not much of an issue if you're a big site with hundreds/thousands of requests a minute but it is an issue if you're a less trafficed site and have a low TTL.
  - rajacombinator 12y ago
    Did you read what I wrote? I said I would pay if I were convinced it would lead to faster loads, but I'm not.
Loic 12y ago
Could you please add the IP or the provider of the client performing the tests? It seems that you are using servers hosted with OVH, this makes the OVH results better because your tests are within the provider network.
- jimaek 12y ago
  I use only 2 servers hosted with OVH. I see the problem here but there is no way to avoid it. Monitoring 37 providers will result in multiple "same DC" problems. I plan to solve this problem by keep adding new locations with different datacenters. The more locations I have the less impact this kind of issues will have.
  - Loic 12y ago
    If you provide the "same DC" information, this helps making the right decision. Maybe you can exclude the "same DC" results or offer an option to filter them out. This of course reduces the number of probes for the given provider, but maybe no probes is better than a too good to be true probe.
    - sandstrom 12y ago
      This! If you have dc-info, you can ignore intra-dc lookups.
    - jimaek 12y ago
      I simply cant know what DCs are used by the providers that I monitor. So I have no idea where we share datacenters. What happens if they change DCs? I would need to constantly monitor their infrastructure. Filtering out would be too hard and pretty useless. By simply adding more locations I fix the problem, add more data and simplify the whole process.
deleted 12y ago
[deleted]
rilita 12y ago
Would you mind updating your bar charts to say "Query time" instead of "Query speed". Some mention of the unit of the numbers would be nice as well, such as "Query speed ( ms/req )"

by jimaek 12y ago

deleted 12y ago

[deleted]

Florin_Andrei 12y ago

The word "speed" usually suggests that more is better. What the graphs actually display is latency - in which case less is better. That was confusing for a few moments.

jimaek 12y ago
I guess I will need to fix that. thanks for the feedback
windexh8er 12y ago
Latency is a function of the speed of light - and in most engineering realms "speed" is well understood when using latency as a description. I had no qualms understanding what the creator meant, but I agree that the technical reality is the graph should be defined by "query latency" vs "query speed". Either way, fantastic data! Hopefully this data is trended over daily, weekly, monthly, quarterly, and yearly rollups. I see monthly when drilling into a provider, but that's all. A rolling annual graph would really tell a true story of how consistent a provider is and that, in itself, is very important when considering outsourcing DNS.

sandstrom 12y ago

Another way of gathering data would be a js-snippet that participating websites can include, which would run a random subset of the tests on some share of of visitors, and report back to your service. It'll be more realistic then DC -> DC queries. - http://www.w3.org/TR/resource-timing/ http://www.w3.org/TR/resource-timing/ - http://googledevelopers.blogspot.se/2013/12/measuring-network-performance-with.html http://googledevelopers.blogspot.se/2013/12/measuring-networ...

jimaek 12y ago
Its realistic but not consistent and reliable. Like I said I want to test their raw performance without any variables in between. All providers get tested directly from the exact same locations and times. Thats the data I wanted to get. It makes it easy and fair to compare.
acdha 12y ago
> It'll be more realistic then DC -> DC queries. That depends on what you're trying to measure. If you want to know what the total end-user experience is like, Resource Timing is awesome. If you're trying to compare DNS providers without noise from the differences in client implementations and ISP competency, however, it'll be very noisy and you need a massive number of samples to avoid seeing trends which are really just sampling artifacts. The best approach would be to do both so you could compare the server response times with the measured client values so you can get an idea whether a slow-down this week is caused by something you have any control over.

corford 12y ago

Hi jimaek! Thanks for putting this service together. For an African testing location maybe a VPS provider from ZA would work? Following are a few providers with Joburg or Cape Town presences (haven't personally used any though so YMMV): http://www.vpsnine.com/hosting/locations.php http://www.vpsnine.com/hosting/locations.php http://www.webafrica.co.za/hosting/vps/ http://www.webafrica.co.za/hosting/vps/ http://www.webnow.co.za/vps/ http://www.webnow.co.za/vps/ http://www.web-telecoms.co.za/vps/ http://www.web-telecoms.co.za/vps/ (posted these to the comments on your site but re-posting here in case you miss it).

jimaek 12y ago
Thank you, I will look into them.

rajacombinator 12y ago

This is cool and I've seen it before but I have a hard time interpreting the results. For instance, I have a hard time believing Google's DNS is substantially slower than anyone else's which leads me to question all the results. Also, and I'm hoping someone here can clear this up for me, but it's not clear to me how these results translate to actual DNS performance for my site. I only vaguely understand how the magic of the internet works but don't the DNS results get distributed throughout the network making the performance of the root servers less relevant? If I could chop 50-100ms off my TTFB by switching my DNS from Namecheap to someone else, I would definitely pay for that, but it's not clear to me.

flipp3r 12y ago
Why would you pay more for faster DNS? Most browsers come with DNS prefretching built in, I doubt it really matters for a website.
- pixl97 12y ago
  Because the first page load is what matters most.
  - moe 12y ago
    Your DNS information should already be cached in all major ISPs DNS caches (unless you mean the first page load ever).
    - corford 12y ago
      How about the first subsequent load after the TTL for the previous load expires? Admittedly, not much of an issue if you're a big site with hundreds/thousands of requests a minute but it is an issue if you're a less trafficed site and have a low TTL.
- rajacombinator 12y ago
  Did you read what I wrote? I said I would pay if I were convinced it would lead to faster loads, but I'm not.

Loic 12y ago

Could you please add the IP or the provider of the client performing the tests? It seems that you are using servers hosted with OVH, this makes the OVH results better because your tests are within the provider network.

jimaek 12y ago
I use only 2 servers hosted with OVH. I see the problem here but there is no way to avoid it. Monitoring 37 providers will result in multiple "same DC" problems. I plan to solve this problem by keep adding new locations with different datacenters. The more locations I have the less impact this kind of issues will have.
- Loic 12y ago
  If you provide the "same DC" information, this helps making the right decision. Maybe you can exclude the "same DC" results or offer an option to filter them out. This of course reduces the number of probes for the given provider, but maybe no probes is better than a too good to be true probe.
  - sandstrom 12y ago
    This! If you have dc-info, you can ignore intra-dc lookups.
  - jimaek 12y ago
    I simply cant know what DCs are used by the providers that I monitor. So I have no idea where we share datacenters. What happens if they change DCs? I would need to constantly monitor their infrastructure. Filtering out would be too hard and pretty useless. By simply adding more locations I fix the problem, add more data and simplify the whole process.

deleted 12y ago

[deleted]

rilita 12y ago

Would you mind updating your bar charts to say "Query time" instead of "Query speed". Some mention of the unit of the numbers would be nice as well, such as "Query speed ( ms/req )"