4 ms·

WarpURL – URL Shorteners Latency Monitoring

14 points by rehanvdm 5y ago

rehanvdm 5y ago
Author here, trying to get awareness to the latency URL Shorteners force onto every user that clicks on them. You can read more about it here: https://link.warpurl.com/DrMbLW https://link.warpurl.com/DrMbLW but below is a quick summary. Double Linking - Shortening a URL that has already been shortened. The biggest culprit is social media sites shorteners: - Twitter ~189ms (+-81ms), max ~420ms Mumbai - LinkedIn ~447ms (+-245ms), max ~850ms Mumbai - Facebook ~148ms (+-66ms), max ~240ms Singapore Average URL Shortener: ~300ms (+-150ms) When a user in Mumbai clicks on a Bitly link (~300ms) shared on LinkedIn (~850ms) then it adds ~1150ms of latency, which is more than 1 second before even reaching your website! WarpURL(https://warpurl.com/ https://warpurl.com/) is ~6 times faster (<50ms) and ~5 times (~30) more consistent.
PaulHoule 5y ago
It's what I'd expect. Most people lack the attention to detail to care about UX and latency.
- rehanvdm 5y ago
  Yeah, as developers we want to optimize the first page the user hits but we don’t often think about how the user gets there. Just because it is fast for us, practically sitting next to a server does not mean others will experience the same. Suddenly optimizing that last 100ms until first paint becomes insignificant. The real shocker after creating that page, was to learn that no other URL Shortener makes use of a CDN :0
tln 5y ago
Thanks for posting! Have you worked on optimizing DNS and TLS negotiation at all? Just now on my laptop, the first request took 1.9 seconds! $ time curl -v https://link.warpurl.com/DrMbLW ... real 0m1.936s $ time curl -v https://link.warpurl.com/DrMbLW ... real 0m0.079s What did you use to check the latency worldwide? I've got a quickly hacked shortener running on CF workers, curious how it stacks up :) PS. there is a small typo on homepage... "and the number of changes you"
- rehanvdm 5y ago
  DNS no, and also not TLS. You can actually navigate it with HTTPS => http://link.warpurl.com/DrMbLW http://link.warpurl.com/DrMbLW The first hit any user from your country/city navigating the URL will be slow aka normal URL Shortener speed. There after it is cached in that region/city/CDN POP. That is why the second curl was so fast. If you ask a friend living in close proximity to you (same region or city) to also navigate it, they should see the fast time. I wrote more about the logic behind it in this blog post: https://link.warpurl.com/x4BSyC https://link.warpurl.com/x4BSyC I also delve into the architecture a bit. Spoiler alert, it is 100% serverless!
  - anonred 5y ago
    Seems quite disingenuous to claim low latency and variance based on a synthetic benchmark when the real world p95 latency and variance will likely be quite high with a normal distribution of user geos. And FWIW I requested the same link on my laptop and then phone and my phone took >1s to resolve the redirect.
    - rehanvdm 5y ago
      I appreciate the response. I clarified the method that I am using on the comparison page, but did not want to go into too much detail because it was taking more real estate than I wanted it to. I will extend that text by saying 1 ping is taking the average time of hitting the URL 5 times. The reason for taking 5 readings directly after each other is to filter out any network related issues and noise on the graphs. When you see a spike in the x axis, it means that was the average latency of hitting that URL 5 times for that interval(every 10 minutes). I am doing it every 10 minutes because I want to show the variance not only for a single region but especially across regions. If we were to take the p95 or even p99 tail latencies over say 100k clicks, WarpURL will be significantly faster. I don't think I am being dishonest in my claim?
      - tln 5y ago
        I don't see how you can get a sub 100ms latency average over 5 hits if your cold starts are > 1s. Maybe hitting the URL every 10 minutes keeps the URLs you test in your cache. If users are frequently getting >1s due to cold starts and your stats are systematically excluding them, then the stats aren't very realistic. I'm guessing you can see how many cold starts there were in AWS Lambda somehow... I can say anecdotally that I got slow responses several times from link.warpurl.com
rehanvdm 5y ago
FYI, if you want to follow me along the journey, I am building WarpURL in public, https://link.rehanvdm.com/BaSR1y https://link.rehanvdm.com/BaSR1y was last months(also the first) retro of many.

14 points by rehanvdm 5y ago

Author here, trying to get awareness to the latency URL Shorteners force onto every user that clicks on them. You can read more about it here: https://link.warpurl.com/DrMbLW https://link.warpurl.com/DrMbLW but below is a quick summary. Double Linking - Shortening a URL that has already been shortened. The biggest culprit is social media sites shorteners: - Twitter ~189ms (+-81ms), max ~420ms Mumbai - LinkedIn ~447ms (+-245ms), max ~850ms Mumbai - Facebook ~148ms (+-66ms), max ~240ms Singapore Average URL Shortener: ~300ms (+-150ms) When a user in Mumbai clicks on a Bitly link (~300ms) shared on LinkedIn (~850ms) then it adds ~1150ms of latency, which is more than 1 second before even reaching your website! WarpURL(https://warpurl.com/ https://warpurl.com/) is ~6 times faster (<50ms) and ~5 times (~30) more consistent.

PaulHoule 5y ago

It's what I'd expect. Most people lack the attention to detail to care about UX and latency.

rehanvdm 5y ago
Yeah, as developers we want to optimize the first page the user hits but we don’t often think about how the user gets there. Just because it is fast for us, practically sitting next to a server does not mean others will experience the same. Suddenly optimizing that last 100ms until first paint becomes insignificant. The real shocker after creating that page, was to learn that no other URL Shortener makes use of a CDN :0

tln 5y ago

Thanks for posting! Have you worked on optimizing DNS and TLS negotiation at all? Just now on my laptop, the first request took 1.9 seconds! $ time curl -v https://link.warpurl.com/DrMbLW ... real 0m1.936s $ time curl -v https://link.warpurl.com/DrMbLW ... real 0m0.079s What did you use to check the latency worldwide? I've got a quickly hacked shortener running on CF workers, curious how it stacks up :) PS. there is a small typo on homepage... "and the number of changes you"

rehanvdm 5y ago
DNS no, and also not TLS. You can actually navigate it with HTTPS => http://link.warpurl.com/DrMbLW http://link.warpurl.com/DrMbLW The first hit any user from your country/city navigating the URL will be slow aka normal URL Shortener speed. There after it is cached in that region/city/CDN POP. That is why the second curl was so fast. If you ask a friend living in close proximity to you (same region or city) to also navigate it, they should see the fast time. I wrote more about the logic behind it in this blog post: https://link.warpurl.com/x4BSyC https://link.warpurl.com/x4BSyC I also delve into the architecture a bit. Spoiler alert, it is 100% serverless!
- anonred 5y ago
  Seems quite disingenuous to claim low latency and variance based on a synthetic benchmark when the real world p95 latency and variance will likely be quite high with a normal distribution of user geos. And FWIW I requested the same link on my laptop and then phone and my phone took >1s to resolve the redirect.
  - rehanvdm 5y ago
    I appreciate the response. I clarified the method that I am using on the comparison page, but did not want to go into too much detail because it was taking more real estate than I wanted it to. I will extend that text by saying 1 ping is taking the average time of hitting the URL 5 times. The reason for taking 5 readings directly after each other is to filter out any network related issues and noise on the graphs. When you see a spike in the x axis, it means that was the average latency of hitting that URL 5 times for that interval(every 10 minutes). I am doing it every 10 minutes because I want to show the variance not only for a single region but especially across regions. If we were to take the p95 or even p99 tail latencies over say 100k clicks, WarpURL will be significantly faster. I don't think I am being dishonest in my claim?
    - tln 5y ago
      I don't see how you can get a sub 100ms latency average over 5 hits if your cold starts are > 1s. Maybe hitting the URL every 10 minutes keeps the URLs you test in your cache. If users are frequently getting >1s due to cold starts and your stats are systematically excluding them, then the stats aren't very realistic. I'm guessing you can see how many cold starts there were in AWS Lambda somehow... I can say anecdotally that I got slow responses several times from link.warpurl.com

rehanvdm 5y ago

FYI, if you want to follow me along the journey, I am building WarpURL in public, https://link.rehanvdm.com/BaSR1y https://link.rehanvdm.com/BaSR1y was last months(also the first) retro of many.