7 ms·

Rolling your own CDN for $25 in 1 hour

324 points by mxpxrocks10 12y ago

kbar13 12y ago
rolling your CDN on VPS instances isn't exactly cost-effective, and is probably more of an exercise for fun than anything else. Unless you need features out of your CDN that you can't get from existing CDN providers, I don't really see the point. also kind of unusual that this blog post is by the president of MaxCDN.
- mxpxrocks10 12y ago
  hey. thanks for the feedback. Yeah, it's more of an exercise for fun but it can also be done w/ high bandwidth dedicated servers as well. It's good as well if it's heavily dynamic stuff that needs really low latency and you need to execute something custom on the edge. Otherwise MaxCDN.com or a similar service scales better. I'm looking forward to building more custom stuff into our edges at MaxCDN.com so feel free to throw out any ideas. This was used to create a presentation for a docker meetup.
- pushrax 12y ago
  Actually, with DigitalOcean, you get quite a lot of bandwidth for the price. Additionally, their transfer limits are high (1TB for the $5 instances), and overage is only $0.02/GB. If you combine this with something like Route 53's anycast latency-routed DNS, it can work very well.
  - michaelmior 12y ago
    Maybe it can work. I'm still a little skeptical. But I don't see why you would ever want to deal with managing all this infrastructure yourself when there are many established providers that can likely do a much better job at a comparable price.
    - pushrax 12y ago
      Managing the infrastructure is definitely where it falls short, but if you can get it done quickly it can be very cost-efficient for smaller projects. Three $5 DO boxes can easily handle > 1TB per month, which will cost over $100 on most CDNs.
      - gaadd33 12y ago
        Couldn't you just use cloudflare's free tier?
        pushrax 12y ago
        CloudFlare's hit rate is predictably terrible for large binary files. We actually use DigitalOcean + Varnish + S3 for serving downloads from mods.io, since the files are relatively big and the site has such a low margin. Traditional CDNs (we were using Fastly) are not scalable cost-wise for that particular site.
    - xin3td 12y ago
      Managing the infrastructures is quite easy with Ansible, Puppet, or whatever your flavor of config management is. You can also take advantage of anycast DNS and GEO targeting by signing up for a free NSOne.net account. That would probably shave $5 off the total.
  - eli 12y ago
    If you have realtively low usage, Amazon CloudFront is already pretty cheap.
    - _delirium 12y ago
      If you have quite low usage yes, and if you have quite high usage you probably won't be running a CDN on DigitalOcean anyway, but there's an in-between point where CloudFront is pretty expensive (by about 10x over the roll-your-own solution, depending on some assumptions about traffic). If you build a smallish network of, say, 10 of the $10/mo, 2-TB-traffic DO servers, and can manage to spread the load such that you use an average of half of the free allocation, you pay $100/mo for 10 TB. Cloudfront would charge you $1200/mo for that data transfer. Obviously it's not the same level of reliability or ease of scaling, but it's cheaper enough to be vaguely intriguing—$1200/yr versus $14k/yr is non-peanuts!
  - chrisan 12y ago
    with DigitalOcean they ask you do not run a CDN: https://www.digitalocean.com/community/questions/do-you-have-a-fair-usage-policy-for-bandwidth https://www.digitalocean.com/community/questions/do-you-have...
    - caw 12y ago
      I'm assuming that's a paid CDN with clients like CloudFlare or Akamai, not you distributing your website to multiple datacenters in order to serve more clients faster. If you happen to use CDN like DNS features to route the client to the closest Digital Ocean datacenter, you're probably in the clear.
      - chrisan 12y ago
        Good point. I wonder if they meant "we ask you don't run a CDN _service_" rather than just a CDN for your website. Afterall, what is the difference between serving assets out of your webserver
    - nly 12y ago
      That question is 2 years old, and refers to an "unlimited bandwidth" fair-use policy. DO does not have unmetered bandwidth, so is it still relevant? Their current policy FAQ says they do not even explicitly forbid Tor exit nodes, although there is a stern warning, so why would they disallow a proxy configuration dedicated solely to your site?
- Oculus 12y ago
  kind of unusual that this blog post is by the president of MaxCDN It makes a lot of sense for the president of MaxCDN to make a blog post like this. Bob the developer decides to learn about how CDN's work and happens upon this blog post. Cool, Bob just built his own mini CDN network for fun. Now Bob understands that his CDN, though a fun weekend project, is definitely not production ready. What does Bob do when he needs a CDN he can put into production? He goes to the guys that established themselves as authorities on the topic: MaxCDN. Content Marketing 101: http://www.kalzumeus.com/2012/02/09/why-i-dont-host-my-own-blog-anymore/ http://www.kalzumeus.com/2012/02/09/why-i-dont-host-my-own-b...
  - corobo 12y ago
    Exactly this. I'm likely to need a CDN in the near-to-distant future. MaxCDN is now on my list because of this post
deleted 12y ago
[deleted]
fmela 12y ago
Thanks for sharing. Note that there's a typo in the link to Ewan's github profile (should start with https, not ttps).
- mxpxrocks10 12y ago
  thanks. will get it fixed shortly. this wasn't really scheduled for prime time but started taking off. appreciate the feedback!
  - TheBiv 12y ago
    "This wasn't really scheduled for prime time, but started taking off" :) You submitted the link to HN and that comment makes it seem like the post accidentally took off!
    - mxpxrocks10 12y ago
      itchy trigger finger :-) didn't expect people to respond so well on a Sunday.
      - TheBiv 12y ago
        Just messing! :) Cheers
  - petercooper 12y ago
    Also wanted to note the post doesn't have enough Ron Smooth
- mxpxrocks10 12y ago
  got this fixed. thanks for the heads up.
akerl_ 12y ago
I'm more than a little bit concerned that the article doesn't point out the serious risk of running a single DNS server for the records. I get that they're trying to keep things simple for their demo, but unsuspecting folks following this guide are in for quite a shock when their GeoDNS is taken out by a single physical server fault.
- mxpxrocks10 12y ago
  Good point - it seemed obvious. We'll cover different options for that in a future blog post.
- mxpxrocks10 12y ago
  added this to the article in the "Todo" section. thanks for the feedback.
  - Lennie 12y ago
    In similar efforts, I also really enjoyed the slides from talk at DNS OARC 2014 about Anycast on a shoe string: http://www.slideshare.net/natmorris/anycast-on-a-shoe-string http://www.slideshare.net/natmorris/anycast-on-a-shoe-string
Oculus 12y ago
Cool article for understanding the basics of a CDN. What sort of page load improvement can one get by adding geographically distributed servers?
- mxpxrocks10 12y ago
  it can be quite a lot. you can test using a CDN or a setup like this and the tools over at www.webpagetest.org (from Patrick Meehan - who is awesome) from different locations.
xxdesmus 12y ago
I'd recommend CloudFlare if you want AnyCast geo-DNS, and a free CDN. We don't charge for bandwidth or our DNS service. Disclosure: yes, I work at CloudFlare but the previous details are simply facts.
- kawsper 12y ago
  Do you have a link to CloudFlare geo-DNS? I have multiple servers in multiple parts of the world, and didn't know that Cloudflare supported that.
  - xxdesmus 12y ago
    CloudFlare uses AnyCast. We'll route any incoming requests to the nearest data center automatically. http://blog.cloudflare.com/a-brief-anycast-primer http://blog.cloudflare.com/a-brief-anycast-primer No additional configuration required. If you're referring to making different origin calls based on the geographic region -- such as when we go back to your origin for a request hitting our LAX data center if you happen to have an origin near LAX we'd use that origin specifically versus your other origin which might be located in let's say LHR. We currently only support that functionality for our higher end enterprise contracts. The functionality will likely to be available for other plan levels in the future though.
    - pyvpx 12y ago
      anycast does not route to the geographically nearest datacenter but the topologically nearest. anycast is great, but it isn't magic. I get it: you have marketing folks that probably specifically told you not to delve into the details but let's face it..."anycast routes it to the nearest datacenter automatically!" isn't completely true.
      - mxpxrocks10 12y ago
        Hey, this brings up a good point for some more clarification. There are two common ways CDNs do routing: method 1) by using a customized dns server (like the one I used in the example) that responds differently to the resolvers end users use. You can use something like Maxmind (like I did in the example) to determine where you think that resolver is. method 2) you can use anycast all the way to the TCP level for terminating traffic to ports 80 and 443. This is the most common way we do it at MaxCDN.com. We find it to be the fastest. To do method 1 you can use the Go GeoDNS server from github (and there's one in PERL) or you can use a service like Dyn.com, DNSMadeEasy (only has broad geo), NSone.net, Verisign's DNS product or Cedexis (which can also incorporate latency data). One thing to keep in mind with doing DNS based routing is eDns for the big public DNS providers. The downside is that you can only respond by the resolver or edns subnet someone uses which can lead to a lot of inaccuracies. The big pro about method 1 is it's easy to deploy and it's easy to balance traffic if a PoP gets over loaded since it's just DNS records. Anycast (Method 2) is great and fast, but it is high maintenance. We've spent a lot of time tweaking this over the past 4 years at MaxCDN. I've heard you can announce anycast blocks with ServerCenteral and Internap, but I haven't tried it. I've done it with Softlayer for testing. You need to have your own /24 to do it with them. We have our own infrastructure and several upstream providers. The biggest challenges with anycast are dealing with Asia (to make sure routes don't trombone) and traffic management since you can't control where traffic goes. One of our guys did a blog post on this a while ago: http://blog.maxcdn.com/anycast-ip-routing-used-maxcdn/ http://blog.maxcdn.com/anycast-ip-routing-used-maxcdn/ Hope this is useful for some!
fakeyfake 12y ago
Reminds me of this article about hosting a CDN on digitalocean for the dash documentation software. http://blog.kapeli.com/a-poor-mans-cdn http://blog.kapeli.com/a-poor-mans-cdn
- mxpxrocks10 12y ago
  cool - thanks
lingben 12y ago
or you know, if your concern is cost, go with cdn.net and pay as you go for a kickass global cdn network full disclosure, I have no business or personal connection to them
- Kudos 12y ago
  There's 6TB of bandwidth built into this home rolled solution. That's going to cost at least $350 at cdn.net.
  - lingben 12y ago
    you're comparing apples to oranges
natch 12y ago
Nice. For tutorials like this it would be nice to see more details on the "how" of each step. For example: "We setup DNS info to point here (both NS1 and NS2..." That's great, but it would be nice to see: "Using the geodns web-based configuration tool, we setup DNS info to point here (both NS1 and NS2..." or "Running <insert full command line here including tool and all options used here> on <insert environment here>, we setup DNS info to point here (both NS1 and NS2..." or "By editing /etc/<file> on the <some environment>, we setup DNS info to point here (both NS1 and NS2..." In other words specifics are always really great to include. But still this was a really great writeup, not just for the overview it gives of one lightweight approach but for the comments it has kicked off.
- mxpxrocks10 12y ago
  great points. happy to take this approach on future posts and would love to collaborate in the future.
  - akg_67 12y ago
    I really liked your write-up. As natch mentioned, it will be great to include more details in the article. I like the detail level of Digital Ocean articles. It seems based on my recent trial experience with CloudFlare that both CDN startups might have this issue of incomplete/insufficient instructions and information for their users/customers to properly implement the solution and lot of assumptions about the skill level of users implementing the solution. Actually, I landed up providing detailed instructions to CloudFlare on configuration and testing of their service. And a list of questions, answers to which should be readily available if CF is targeting enterprise customers.
    - mxpxrocks10 12y ago
      thanks for the feedback. appreciate it.
- joesavage 12y ago
  I was planning to publish a post with literally the same title as this one later this week (annoying coincidence as I guess mine will now be largely ignored - maybe I should wait a few weeks to let this space calm down, I'm not sure) with some in-depth step-by-step explanation - so watch out for that :)
deleted 12y ago
[deleted]
jpmonette 12y ago
Definitely nice to see a real-world example of how to use Docker.
- mxpxrocks10 12y ago
  woot woot.
ColinCera 12y ago
I'm wondering if it would be useful to plug groupcache into this. It's what Google uses for serving downloads (e.g., Chrome downloads). Inherently distributed and self-balancing. I might give that a try.
- mxpxrocks10 12y ago
  great idea.
- xin3td 12y ago
  Looks promising, but might need to add Doozer to the stack as groupcache has an issue with maintaining a list of peers where the cached data is distributed. Definitely worth looking into though. https://github.com/ha/doozerd https://github.com/ha/doozerd
danielrhodes 12y ago
I would highly recommend Varnish using S3 as a backend origin. Add to that a few 100TB.com dedicated server and you have an extremely cheap CDN with pretty decent bandwidth and minimal maintenance. CDNs have the advantage that they control more of the stack so can do more precise routing and have more edge nodes in more places. However, under the right circumstances, you can take the above quite far before a real CDN becomes necessary.
- mxpxrocks10 12y ago
  hey Daniel- thanks for the comment. Do you run any special varnish config settings w/ s3?
  - danielrhodes 12y ago
    Nothing fancy right now. It's all static files. I think if I were doing caching of a dynamic application, some special settings would be far more valuable.
- RKearney 12y ago
  100TB.com specifically forbids using their service to create a CDN in their ToS[0]. 9. Acceptable Use/Illegal Activity d. We strive to maintain a high level of service, and a lot of customers depend on our high standards of quality. As such, we will not provide Services to those that are using our Services for: vii Using the Services for a content delivery network or content distribution network (CDN). An authorized CDN network offered through 100TB is accepted. Special requests to use the Services to run an unauthorized CDN network may be approved on a case- by-case basis. Failure to comply with this policy will result in termination of this TOS, and you will not receive a refund of the Fees. [0]: http://www.100tb.com/tos.php http://www.100tb.com/tos.php
  - ddorian43 12y ago
    people forget the story of simplecdn
AdamN 12y ago
Now with libcloud (http://libcloud.readthedocs.org/en/latest/compute/drivers/ http://libcloud.readthedocs.org/en/latest/compute/drivers/) you can use multiple platforms in order to get all continents represented. Including Africa (http://kili.io http://kili.io) where I'm at.
- mxpxrocks10 12y ago
  awesome. Thanks for posting. I had that as one of my todo's in the blog post. Feel free to reach out to me if you want to help with a follow up post!
- muriithi 12y ago
  Why Africa and not Kenya?
- mxpxrocks10 12y ago
  has anyone ever heard about these guys in South Africa: http://www.teraco.co.za/ http://www.teraco.co.za/? Any other recommendations for Africa and South Africa?
hiddenpearls 12y ago
that's awesome +Chris
edoceo 12y ago
This is similar to the process I used to build http://gcdn.org/ http://gcdn.org/ which I and a few clients + friends use on multiple projects. We used to use RR but GeoDNS has proven to be awesome and reliable. Full NS control makes it easier to do rolling upgrades or maintenance too. Excellent write up. More notes on GeoDNS http://edoceo.com/howto/geodns http://edoceo.com/howto/geodns
- umsm 12y ago
  This returns a 404-not found for me: https://gcdn.org/jquery/1.8.0/jquery.js https://gcdn.org/jquery/1.8.0/jquery.js
adarshr 12y ago
Why is this page blocked by my UK work network?
- Kudos 12y ago
  The people doing the blocking can probably better answer that.
leapius 12y ago
Excellent tut!
ksec 12y ago
Wouldn't it be better on Linode? Since they have wider selection of PoP. But i wonder if the port speed would be a problem.

324 points by mxpxrocks10 12y ago

rolling your CDN on VPS instances isn't exactly cost-effective, and is probably more of an exercise for fun than anything else. Unless you need features out of your CDN that you can't get from existing CDN providers, I don't really see the point. also kind of unusual that this blog post is by the president of MaxCDN.

mxpxrocks10 12y ago
hey. thanks for the feedback. Yeah, it's more of an exercise for fun but it can also be done w/ high bandwidth dedicated servers as well. It's good as well if it's heavily dynamic stuff that needs really low latency and you need to execute something custom on the edge. Otherwise MaxCDN.com or a similar service scales better. I'm looking forward to building more custom stuff into our edges at MaxCDN.com so feel free to throw out any ideas. This was used to create a presentation for a docker meetup.
pushrax 12y ago
Actually, with DigitalOcean, you get quite a lot of bandwidth for the price. Additionally, their transfer limits are high (1TB for the $5 instances), and overage is only $0.02/GB. If you combine this with something like Route 53's anycast latency-routed DNS, it can work very well.
- michaelmior 12y ago
  Maybe it can work. I'm still a little skeptical. But I don't see why you would ever want to deal with managing all this infrastructure yourself when there are many established providers that can likely do a much better job at a comparable price.
  - pushrax 12y ago
    Managing the infrastructure is definitely where it falls short, but if you can get it done quickly it can be very cost-efficient for smaller projects. Three $5 DO boxes can easily handle > 1TB per month, which will cost over $100 on most CDNs.
    - gaadd33 12y ago
      Couldn't you just use cloudflare's free tier?
      - pushrax 12y ago
        CloudFlare's hit rate is predictably terrible for large binary files. We actually use DigitalOcean + Varnish + S3 for serving downloads from mods.io, since the files are relatively big and the site has such a low margin. Traditional CDNs (we were using Fastly) are not scalable cost-wise for that particular site.
  - xin3td 12y ago
    Managing the infrastructures is quite easy with Ansible, Puppet, or whatever your flavor of config management is. You can also take advantage of anycast DNS and GEO targeting by signing up for a free NSOne.net account. That would probably shave $5 off the total.
- eli 12y ago
  If you have realtively low usage, Amazon CloudFront is already pretty cheap.
  - _delirium 12y ago
    If you have quite low usage yes, and if you have quite high usage you probably won't be running a CDN on DigitalOcean anyway, but there's an in-between point where CloudFront is pretty expensive (by about 10x over the roll-your-own solution, depending on some assumptions about traffic). If you build a smallish network of, say, 10 of the $10/mo, 2-TB-traffic DO servers, and can manage to spread the load such that you use an average of half of the free allocation, you pay $100/mo for 10 TB. Cloudfront would charge you $1200/mo for that data transfer. Obviously it's not the same level of reliability or ease of scaling, but it's cheaper enough to be vaguely intriguing—$1200/yr versus $14k/yr is non-peanuts!
- chrisan 12y ago
  with DigitalOcean they ask you do not run a CDN: https://www.digitalocean.com/community/questions/do-you-have-a-fair-usage-policy-for-bandwidth https://www.digitalocean.com/community/questions/do-you-have...
  - caw 12y ago
    I'm assuming that's a paid CDN with clients like CloudFlare or Akamai, not you distributing your website to multiple datacenters in order to serve more clients faster. If you happen to use CDN like DNS features to route the client to the closest Digital Ocean datacenter, you're probably in the clear.
    - chrisan 12y ago
      Good point. I wonder if they meant "we ask you don't run a CDN _service_" rather than just a CDN for your website. Afterall, what is the difference between serving assets out of your webserver
  - nly 12y ago
    That question is 2 years old, and refers to an "unlimited bandwidth" fair-use policy. DO does not have unmetered bandwidth, so is it still relevant? Their current policy FAQ says they do not even explicitly forbid Tor exit nodes, although there is a stern warning, so why would they disallow a proxy configuration dedicated solely to your site?
Oculus 12y ago
kind of unusual that this blog post is by the president of MaxCDN It makes a lot of sense for the president of MaxCDN to make a blog post like this. Bob the developer decides to learn about how CDN's work and happens upon this blog post. Cool, Bob just built his own mini CDN network for fun. Now Bob understands that his CDN, though a fun weekend project, is definitely not production ready. What does Bob do when he needs a CDN he can put into production? He goes to the guys that established themselves as authorities on the topic: MaxCDN. Content Marketing 101: http://www.kalzumeus.com/2012/02/09/why-i-dont-host-my-own-blog-anymore/ http://www.kalzumeus.com/2012/02/09/why-i-dont-host-my-own-b...
- corobo 12y ago
  Exactly this. I'm likely to need a CDN in the near-to-distant future. MaxCDN is now on my list because of this post

deleted 12y ago

[deleted]

fmela 12y ago

Thanks for sharing. Note that there's a typo in the link to Ewan's github profile (should start with https, not ttps).

mxpxrocks10 12y ago
thanks. will get it fixed shortly. this wasn't really scheduled for prime time but started taking off. appreciate the feedback!
- TheBiv 12y ago
  "This wasn't really scheduled for prime time, but started taking off" :) You submitted the link to HN and that comment makes it seem like the post accidentally took off!
  - mxpxrocks10 12y ago
    itchy trigger finger :-) didn't expect people to respond so well on a Sunday.
    - TheBiv 12y ago
      Just messing! :) Cheers
- petercooper 12y ago
  Also wanted to note the post doesn't have enough Ron Smooth
mxpxrocks10 12y ago
got this fixed. thanks for the heads up.

akerl_ 12y ago

I'm more than a little bit concerned that the article doesn't point out the serious risk of running a single DNS server for the records. I get that they're trying to keep things simple for their demo, but unsuspecting folks following this guide are in for quite a shock when their GeoDNS is taken out by a single physical server fault.

mxpxrocks10 12y ago
Good point - it seemed obvious. We'll cover different options for that in a future blog post.
mxpxrocks10 12y ago
added this to the article in the "Todo" section. thanks for the feedback.
- Lennie 12y ago
  In similar efforts, I also really enjoyed the slides from talk at DNS OARC 2014 about Anycast on a shoe string: http://www.slideshare.net/natmorris/anycast-on-a-shoe-string http://www.slideshare.net/natmorris/anycast-on-a-shoe-string

Oculus 12y ago

Cool article for understanding the basics of a CDN. What sort of page load improvement can one get by adding geographically distributed servers?

mxpxrocks10 12y ago
it can be quite a lot. you can test using a CDN or a setup like this and the tools over at www.webpagetest.org (from Patrick Meehan - who is awesome) from different locations.

xxdesmus 12y ago

I'd recommend CloudFlare if you want AnyCast geo-DNS, and a free CDN. We don't charge for bandwidth or our DNS service. Disclosure: yes, I work at CloudFlare but the previous details are simply facts.

kawsper 12y ago
Do you have a link to CloudFlare geo-DNS? I have multiple servers in multiple parts of the world, and didn't know that Cloudflare supported that.
- xxdesmus 12y ago
  CloudFlare uses AnyCast. We'll route any incoming requests to the nearest data center automatically. http://blog.cloudflare.com/a-brief-anycast-primer http://blog.cloudflare.com/a-brief-anycast-primer No additional configuration required. If you're referring to making different origin calls based on the geographic region -- such as when we go back to your origin for a request hitting our LAX data center if you happen to have an origin near LAX we'd use that origin specifically versus your other origin which might be located in let's say LHR. We currently only support that functionality for our higher end enterprise contracts. The functionality will likely to be available for other plan levels in the future though.
  - pyvpx 12y ago
    anycast does not route to the geographically nearest datacenter but the topologically nearest. anycast is great, but it isn't magic. I get it: you have marketing folks that probably specifically told you not to delve into the details but let's face it..."anycast routes it to the nearest datacenter automatically!" isn't completely true.
    - mxpxrocks10 12y ago
      Hey, this brings up a good point for some more clarification. There are two common ways CDNs do routing: method 1) by using a customized dns server (like the one I used in the example) that responds differently to the resolvers end users use. You can use something like Maxmind (like I did in the example) to determine where you think that resolver is. method 2) you can use anycast all the way to the TCP level for terminating traffic to ports 80 and 443. This is the most common way we do it at MaxCDN.com. We find it to be the fastest. To do method 1 you can use the Go GeoDNS server from github (and there's one in PERL) or you can use a service like Dyn.com, DNSMadeEasy (only has broad geo), NSone.net, Verisign's DNS product or Cedexis (which can also incorporate latency data). One thing to keep in mind with doing DNS based routing is eDns for the big public DNS providers. The downside is that you can only respond by the resolver or edns subnet someone uses which can lead to a lot of inaccuracies. The big pro about method 1 is it's easy to deploy and it's easy to balance traffic if a PoP gets over loaded since it's just DNS records. Anycast (Method 2) is great and fast, but it is high maintenance. We've spent a lot of time tweaking this over the past 4 years at MaxCDN. I've heard you can announce anycast blocks with ServerCenteral and Internap, but I haven't tried it. I've done it with Softlayer for testing. You need to have your own /24 to do it with them. We have our own infrastructure and several upstream providers. The biggest challenges with anycast are dealing with Asia (to make sure routes don't trombone) and traffic management since you can't control where traffic goes. One of our guys did a blog post on this a while ago: http://blog.maxcdn.com/anycast-ip-routing-used-maxcdn/ http://blog.maxcdn.com/anycast-ip-routing-used-maxcdn/ Hope this is useful for some!

fakeyfake 12y ago

Reminds me of this article about hosting a CDN on digitalocean for the dash documentation software. http://blog.kapeli.com/a-poor-mans-cdn http://blog.kapeli.com/a-poor-mans-cdn

mxpxrocks10 12y ago
cool - thanks

lingben 12y ago

or you know, if your concern is cost, go with cdn.net and pay as you go for a kickass global cdn network full disclosure, I have no business or personal connection to them

Kudos 12y ago
There's 6TB of bandwidth built into this home rolled solution. That's going to cost at least $350 at cdn.net.
- lingben 12y ago
  you're comparing apples to oranges

natch 12y ago

Nice. For tutorials like this it would be nice to see more details on the "how" of each step. For example: "We setup DNS info to point here (both NS1 and NS2..." That's great, but it would be nice to see: "Using the geodns web-based configuration tool, we setup DNS info to point here (both NS1 and NS2..." or "Running <insert full command line here including tool and all options used here> on <insert environment here>, we setup DNS info to point here (both NS1 and NS2..." or "By editing /etc/<file> on the <some environment>, we setup DNS info to point here (both NS1 and NS2..." In other words specifics are always really great to include. But still this was a really great writeup, not just for the overview it gives of one lightweight approach but for the comments it has kicked off.

mxpxrocks10 12y ago
great points. happy to take this approach on future posts and would love to collaborate in the future.
- akg_67 12y ago
  I really liked your write-up. As natch mentioned, it will be great to include more details in the article. I like the detail level of Digital Ocean articles. It seems based on my recent trial experience with CloudFlare that both CDN startups might have this issue of incomplete/insufficient instructions and information for their users/customers to properly implement the solution and lot of assumptions about the skill level of users implementing the solution. Actually, I landed up providing detailed instructions to CloudFlare on configuration and testing of their service. And a list of questions, answers to which should be readily available if CF is targeting enterprise customers.
  - mxpxrocks10 12y ago
    thanks for the feedback. appreciate it.
joesavage 12y ago
I was planning to publish a post with literally the same title as this one later this week (annoying coincidence as I guess mine will now be largely ignored - maybe I should wait a few weeks to let this space calm down, I'm not sure) with some in-depth step-by-step explanation - so watch out for that :)

deleted 12y ago

[deleted]

jpmonette 12y ago

Definitely nice to see a real-world example of how to use Docker.

mxpxrocks10 12y ago
woot woot.

ColinCera 12y ago

I'm wondering if it would be useful to plug groupcache into this. It's what Google uses for serving downloads (e.g., Chrome downloads). Inherently distributed and self-balancing. I might give that a try.

mxpxrocks10 12y ago
great idea.
xin3td 12y ago
Looks promising, but might need to add Doozer to the stack as groupcache has an issue with maintaining a list of peers where the cached data is distributed. Definitely worth looking into though. https://github.com/ha/doozerd https://github.com/ha/doozerd

danielrhodes 12y ago

I would highly recommend Varnish using S3 as a backend origin. Add to that a few 100TB.com dedicated server and you have an extremely cheap CDN with pretty decent bandwidth and minimal maintenance. CDNs have the advantage that they control more of the stack so can do more precise routing and have more edge nodes in more places. However, under the right circumstances, you can take the above quite far before a real CDN becomes necessary.

mxpxrocks10 12y ago
hey Daniel- thanks for the comment. Do you run any special varnish config settings w/ s3?
- danielrhodes 12y ago
  Nothing fancy right now. It's all static files. I think if I were doing caching of a dynamic application, some special settings would be far more valuable.
RKearney 12y ago
100TB.com specifically forbids using their service to create a CDN in their ToS[0]. 9. Acceptable Use/Illegal Activity d. We strive to maintain a high level of service, and a lot of customers depend on our high standards of quality. As such, we will not provide Services to those that are using our Services for: vii Using the Services for a content delivery network or content distribution network (CDN). An authorized CDN network offered through 100TB is accepted. Special requests to use the Services to run an unauthorized CDN network may be approved on a case- by-case basis. Failure to comply with this policy will result in termination of this TOS, and you will not receive a refund of the Fees. [0]: http://www.100tb.com/tos.php http://www.100tb.com/tos.php
- ddorian43 12y ago
  people forget the story of simplecdn

AdamN 12y ago

Now with libcloud (http://libcloud.readthedocs.org/en/latest/compute/drivers/ http://libcloud.readthedocs.org/en/latest/compute/drivers/) you can use multiple platforms in order to get all continents represented. Including Africa (http://kili.io http://kili.io) where I'm at.

mxpxrocks10 12y ago
awesome. Thanks for posting. I had that as one of my todo's in the blog post. Feel free to reach out to me if you want to help with a follow up post!
muriithi 12y ago
Why Africa and not Kenya?
mxpxrocks10 12y ago
has anyone ever heard about these guys in South Africa: http://www.teraco.co.za/ http://www.teraco.co.za/? Any other recommendations for Africa and South Africa?

hiddenpearls 12y ago

that's awesome +Chris

edoceo 12y ago

This is similar to the process I used to build http://gcdn.org/ http://gcdn.org/ which I and a few clients + friends use on multiple projects. We used to use RR but GeoDNS has proven to be awesome and reliable. Full NS control makes it easier to do rolling upgrades or maintenance too. Excellent write up. More notes on GeoDNS http://edoceo.com/howto/geodns http://edoceo.com/howto/geodns

umsm 12y ago
This returns a 404-not found for me: https://gcdn.org/jquery/1.8.0/jquery.js https://gcdn.org/jquery/1.8.0/jquery.js

adarshr 12y ago

Why is this page blocked by my UK work network?

Kudos 12y ago
The people doing the blocking can probably better answer that.

leapius 12y ago

Excellent tut!

ksec 12y ago

Wouldn't it be better on Linode? Since they have wider selection of PoP. But i wonder if the port speed would be a problem.