26 ms·
Rearchitecting GitHub Pages
- VeejayRampay 11y agoWell done GitHub. Also a special mention to the invisible workers making nginx such a cornerstone of the modern infrastructure, it's a project that I don't hear about often, probably due to the fact that it's not the sexiest piece of technology, but it really seems solid and battle-tested. Kudos.
- adriancooney 11y agoI'm amazed that Github Pages ran on just two servers (well, aside from the MySQL clusters). That is absolutely incredible given the sheer amount of projects and people who rely on it for their sites (me included!). I love the philosophy behind Pages of abstaining from over-engineering and sticking to the simple, proven solutions. It's a great service and I'm a massive fan.
- vog 11y agoIn the same vein, I'm surprised how many static websites go down just because they appear on a news site.
- munificent 11y agoMost of those I see either: 1. Aren't static sites but are things like WordPress blogs running server-side PHP essentially unnecessarily. 2. Don't take down the server, but just hit the tiny bandwidth quota of some cheap shared hosting provider, so the host blocks it.
- josephmx 11y agoA significant amount also seem to saturate their line, especially on photo-heavy static sites
- mortehu 11y ago3. Use the distro's default Apache connection limit of 20, despite the server being able to handle thousands of concurrent connections. The limited connections are then quickly used by users with slow connections.
- joshmoz 11y agoIt's cool that they were able to do this with two machines, and I don't want to detract from that, but it's probably worth pointing out that a "machine" is not a very useful unit of capacity. These two machines could be dual core i5s or they could be 20 core xeon boxes with hugely varying amounts of memory and storage. Too bad they don't clarify, I'm curious.
- ceequof 11y agoThe fileserver tier consists of pairs of Dell R720s running in active/standby configuration. Each pair is largely similar to the single pair of machines that the old Pages infrastructure ran on. http://www.dell.com/us/business/p/poweredge-r720/pd http://www.dell.com/us/business/p/poweredge-r720/pd Two Xeon E5-2600s per machine.
- deleted 11y ago[deleted]
- ChiperSoft 11y agoThe ES-2600 series comes in 4 to 12 cores, so that's still a pretty wide range of compute power.
- vinhboy 11y agoI am going to remember this article and bring it up every time some ops crazy people try to trick me into over-engineering shit.
- clinta 11y agoAs an ops guy, I'm going to remember this article and use it as a talking point for the benefits of static pages when devs want to put everything into their dynamic framework of choice.
- tracker1 11y agoIt's a bit of a trade off... static generation of dynamic content isn't so much different from front-side caching, if the bulk of your requests can cache for 30+ minutes, you can setup some pretty decent caching rules on a couple of servers fronting a dynamic backend that doesn't need to be too big to support many requests. For the most part, the load tends to come down to less than optimized database storage and queries on the backend, often duplicated for every page load for content that could easily be offloaded to the client browser, constructed client-side, and requested on demand. It's a matter of striking a balance... In this case the content changes very infrequently, and having a publish step to static storage makes sense. It really depends on one's needs. If you have a million pages you'd need to regenerate daily for a single site, and thousands every hour, then you might think differently.
- rajeevk 11y agoThey also use CDN. As the pages are cacheable, the CDN will decrease the load on their server significantly.
- serverholic 11y agoCan you explain a bit more about the load balancer tier? The blog post doesn't say much about it and I'm curious about your haproxy config.
- jssjr 11y agoThis sounds like a great idea for a future blog post. Thanks for the suggestion!
- nvk 11y agoIt's time for github to start offering some basic hosting infrastructure of small projects, a light heroku, at least for JavaScript (which kind of already works). I'd pay extra for that, I (we all) have a bunch of personal sites, landing pages, marketing sites and tiny side projects that'd love to not have to deal with hosting – I think they'd make a killing, but also think must be in the works.
- lstoll 11y agoHeroku already does this pretty well, not sure what the benefit would be?
- spdionis 11y agoI think the leap they'd have to make in infrastructure and architecture to support that is not worth it in their mind. But who knows.
- chralieboy 11y agoAlso the mental jump. Just because something is easy for them to do doesn't mean it is worth the distraction cost. Github builds tools for developers. Atom, chat (abandoned), Pages, Gists, and github.com all fit within this. They tie into how teams operate. Serving JS is tangentially related — certainly something a web developer does — but not really core to their mission.
- keehun 11y agoAgreed. I think static sites have their niche and Github probably isn't interested in more than that.
- kyledrake 11y agoI'm keen to the demand level via our research. IMHO it's not worth it for their business size and growth plans. Their other initiatives are far more lucrative. There's also a lot of competition at that level. I think of this more as a convenience feature for their existing business that adds value. They use this instead of the "project page" design that the other code sites used, and it gives their users more control over the presentation. Which is awesome, and feeds into my conspiracy to get everybody to know and use HTML for presentation. :)
- nemothekid 11y agoI've never known about this way to extend nginx - I've been looking at a way to get nginx "smarter" on our mesos config as dns-based configuration had some holes.
- tangent128 11y ago"Hooks" like that is where the Lua language really shines.
- lamby 11y agoCuriously, I remember sticking with lighttpd on a project for quite a while simply for their Lua integration (nginx didn't have it at the time).
- maxmcd 11y agoThis is all sitting behind a CDN correct?[1] Might explain why it was able to survive on two servers for so long. Or is that automatically assumed when reading about a static hosting setup? 1. https://www.fastly.com/customers/github/ https://www.fastly.com/customers/github/
- manigandham 11y agoIt's mentioned in the article. > We also have Fastly sitting in front of GitHub Pages caching all 200 responses. This helps minimise the availability impact of a total Pages router outage. Even in this worst case scenario, cached Pages sites are still online and unaffected.
- nicolewhite 11y agoI've been using GitHub pages for a while now and I always wondered why they had the "your site may not be available for another 30 minutes" message on creating a new GitHub pages site while pushes to an already-existing gh-pages branch were displayed instantly. Neat to see that explained here.
- jrochkind1 11y agoAll of github pages was run off of _one_ server? (with one failover standby). That's pretty amazing. If all you're serving is static assets, apparently you have to grow to pretty huge scale before one server will not be sufficient. I'm curious if there was at least a caching layer, so every request didn't hit the SSD. They didn't mention it.
- vog 11y agoI believe this is automatically done by the operating system. Nginx usually performs some kind of mmap of the requested file, and the virtual memory management will automatically take care of caching. BTW, I'm often surprised how people are afraid from opening files from their scripts, as if they think this will always lead to disk access. Then, they start implementing a hand-written caching layer on top of that, which usually performs worse than what the OS already offers.
- deleted 11y ago[deleted]
- falcolas 11y agoIf it's a static file with no dynamic compression (nginx allows the use of pre-compressed assets) or SSL, nginx doesn't even have to open the file, it just uses the sendfile system call.
- shanemhansen 11y agoMinor nit. The sendfile system call requires an open file. nginx can just skip reading the file contents into userspace. Also, BSD has some recent extensions to make sendfile work with ssl [https://people.freebsd.org/~rrs/asiabsd_2015_tls.pdf https://people.freebsd.org/~rrs/asiabsd_2015_tls.pdf] (pdf)
- nomel 11y agoSSD cache and os file system cache is always there. I'm sure they were filled with ram, and that's all it would be used for.
- gabrtv 11y agoI was hoping to see something about optional SSL/TLS. I'm even willing to pay for it.
- jsingleton 11y agoYou can use CloudFlare with flexible SSL in front of it and set up a page rule to redirect to HTTPS.
- gabrtv 11y agoI had no idea. That's great news!
- drewbug 11y agoBe warned, though: this will only create a secure connection between the user and CloudFlare, not between CloudFlare and GitHub Pages.
- tracker1 11y agoWhile a potential for concern... I would guess that CloudFlare's own routing would take them to an exit node close to Github's... that does make quite a few assumptions though. It would still be more complicated to mitm between CloudFlare and Github than ISP-X (or China) and Github...
- cben 11y agoThere already kinda is TLS on github.io but it's only user<->CDN, CDN<->Github is unencrypted. [https://konklone.com/post/github-pages-now-sorta-supports-https-so-use-it https://konklone.com/post/github-pages-now-sorta-supports-ht...] And it doesn't work on custom domains (Github can't present a cert for your domain); you can make it "work" via Cloudflare but that only works if Cloudflare doesn't validate Github's cert, which introduces yet another unsecure link. [https://github.com/isaacs/github/issues/156 https://github.com/isaacs/github/issues/156]
- jsingleton 11y agoNice! Does HN still run off of a single server and CDN too? The CDN is key here, which you get if you use a CNAME (or ALIAS) instead of an A record for your custom domain on GH pages. I've found pairing pages with CloudFlare works great if you want to use a naked domain and you get HTTPS too. You can set up a page rule on CF to redirect all HTTP to HTTPS as well.
- methyl 11y agoI'm wondering why GitHub prefers MySQL over PostgreSQL.
- samlambert 11y agoStable, proven, popular, great roadmap, great replication story, great tooling and an awesome community.
- oblio 11y agoAnd possibly in-house experience, which is always important.
- chralieboy 11y agoThis. The rest of GitHub runs on MySQL. As the rest of the post alludes, adding unnecessary complexity isn't what they are about (in this case.)
- samlambert 11y agoWe avoid complexity as much as possible.
- brador 11y agoWould postgreSQL be meaningfully faster?
- samlambert 11y agoMySQL is extremely fast.
- twic 11y agoMy heuristic is that MySQL is faster for inserts and single-table reads by primary key, but PostgreSQL is faster for more complex queries, particularly if they involve joins or subqueries. Basically, MySQL is a key-value store in an RDBMS's clothing.
- wiradikusuma 11y agooff-topic, but since we're talking about github pages.. i own github.id, and i'm thinking of making "linkedin for github users", do you guys think there's a market for that?
- tsm 11y agoWouldn't there be huge copyright issues with that?
- krallja 11y agoNo. "GitHub" is a trademark, not a copyrightable work.
- brandonwamboldt 11y agoThat's a clear violation of GitHub's trademark so you'd almost certainly receive a C&D (GitHub legally has to protect their trademark or they could lose it).
- mellett68 11y agoLegal issues aside, github already offers social features, what features would you offer that would enhance that existing functionality?
- ozten 11y agoI agree with brandonwamboldt, mellett68, and tsm; the domain name is useless. If you had a different business name... I think a site that summarizes someone's github contributions from a recruiter / interviewer's perspective would be very helpful. Poking around github to research a candidate is time consuming. Perhaps a more useful one page snapshot could be created. X profile views is free and you sell recruiters a per-company subscription fee.
- oblio 11y agoSomewhat related, what's the SLA for Github pages? Let's say someone wants to move a highly successful blog, are there some quotas? Maybe I missed them but I don't remember seeing any...
- benbalter 11y agoBen from GitHub here. We don't offer an SLA for GitHub Pages (or GitHub.com generally). We've talked a bit about our typical traffic in the past (https://github.com/blog/1992-eight-lessons-learned-hacking-on-github-pages-for-six-months https://github.com/blog/1992-eight-lessons-learned-hacking-o...), but suffice to say, we host some highly trafficked sites like the Bootstrap documentation. If your blog isn't on that scale, and conforms to our terms of service (https://help.github.com/articles/github-terms-of-service/#g-general-conditions https://help.github.com/articles/github-terms-of-service/#g-...), we generally don't enforce hard usage quotas, but if you're concerned, feel free to reach out to support@github.com any time.
- oblio 11y agoWell, I think this questions would benefit more people so I'll ask in public: are there any plans for a commercial offering? Thanks!
- benbalter 11y agoWhat do you mean by commercial offering? A paid version of GitHub Pages? What would you want to see in a paid offering that couldn't be baked into the current version?
- oblio 11y agoPaid version, something that would rival web page hosting. Regarding features, I'm mostly thinking about the possibility of basing business on top of Github, for example uptime guarantees and notifications before termination in case of abuse. Maybe even stuff like throttling instead of pulling the plug in case of DDoS. If these can be added to the free version, cool! :)
- nosideeffects 11y agoWhy did they choose the 98th percentile, specifically? Why not 99th and friends?
- BillinghamJ 11y agoSeems odd to me that the router hits a MySQL database on every single request rather than just hashing the hostname as the key for the filesystem node.
- tdicola 11y agoHash-based partitioning has a big problem that when you change the hash size all of the data moves around. Eventually you'll need to do a lookup-based partitioning scheme. You also probably want control over where some users live since you don't want two super hot users on the same server.
- TheLoneWolfling 11y agoThere are hash schemes that prevent that, however. Look up consistent hashing. And you could probably have a (small) hashmap that allows overrides for the most used pages.
- twic 11y agoMy thought too. If they have half a million sites, and only 1% are hot enough to need deliberate placement, then they only need to store five thousand special cases, which is a few megabytes of memory, easily stored on each load balancer and loaded at boot. Use hashing for the rest. Ideally consistent hashing, or rendezvous hashing, which i just read about on Wikipedia so must be good: http://en.wikipedia.org/wiki/Rendezvous_hashing http://en.wikipedia.org/wiki/Rendezvous_hashing
- kid0m4n 11y agoWon't CDN come into force for the two super hot users? I would gone with consistent hashing as well....
- BillinghamJ 11y agoYou could make the hash, say, 2 bytes long. That would allow for up to 65536 servers. Say you have 8 servers at present, you'd allocate 8192 sequential keys to each server. If you need more capacity, use 16 servers instead, and move half the files from each server to the corresponding new server. Thus exactly half the data is moved each time the number of servers is doubled or halved.
- siliconc0w 11y agoIf you use the magic of hashing you can lookup where user's pages are without the dependency on the database.
- deleted 11y ago[deleted]
- brador 11y agoHow would you use hashing for that?
- slig 11y agoUsing "Consistent Hash" [1], for instance. > Consistent hashing is a very simple solution to a common problem: how can you find a server in a distributed system to store or retrieve a value identified by a key, while at the same time being able to cope with server failures and network partitions? [2] [1]: http://en.wikipedia.org/wiki/Consistent_hashing http://en.wikipedia.org/wiki/Consistent_hashing [2]: http://www.martinbroadhurst.com/Consistent-Hash-Ring.html http://www.martinbroadhurst.com/Consistent-Hash-Ring.html
- TheLoneWolfling 11y agoHuh. I wonder if a consistent hash + remalloc would be faster than a standard hashmap for frequent insertions/deletions.
- charliesome 11y agoUnfortunately this isn't an option for us. We need to be able to move sites between fileservers periodically to keep disk usage and load balanced. Adding new fileservers when using a hash modulus based routing scheme is also quite complex as it would require copying quite a lot of site data between fileservers. Pages also supports custom domains for both user sites and per-project sites, so we'd still need a way to resolve domains to users.
- 11y ago
- el33th4xx0r 11y agoSurprisingly, they uses mysql (instead of current hype k/v store) to map hostname and fileserver.
- charliesome 11y agoMySQL is actually a really good key value store! Here's the schema we use for the routing information: CREATE TABLE `pages_routes` ( `id` int(11) NOT NULL AUTO_INCREMENT, `user_id` int(11) NOT NULL, `host` varchar(255) NOT NULL, PRIMARY KEY (`id`), UNIQUE KEY `index_pages_routes_on_user_id` (`user_id`) ); Since we use MySQL for everything else, we decided it made the most sense to keep this routing data here rather than introducing a new database.
- toomuchtodo 11y ago> Since we use MySQL for everything else, we decided it made the most sense to keep this routing data here rather than introducing a new database. Very smart. "Perfection is Achieved Not When There Is Nothing More to Add, But When There Is Nothing Left to Take Away"
- jamesblonde 11y agoDo you run database management software, like severalnines clustercontrol, to manage your MySQL instances?
- MrBuddyCasino 11y agoI was surprised the lookup takes just 3ms. Is Lua pooling the db connections?
- mwcampbell 11y agoOnly tangentially related, but I sometimes wonder if GitHub's management regret making GitHub Pages available for free, now that it's being used so heavily for personal and even business blogs, rather than just companion sites for open-source projects. They could be charging for static websites, as Amazon S3 does.
- ceejayoz 11y agoIf it was able to run off a single server for that long, I suspect the goodwill it engendered (as well as the familiarity with Git/Github.com it built in a lot of people) was well worth the minimal resources and cost it entailed.
- dyladan 11y agoI doubt it. After seeing how simple the architecture is to run it, I'm sure it's a drop in the proverbial bucket. Pages drives traffic to the site and in order to serve a site from a private repo you have to be a paying customer anyways.
- holman 11y agoI never heard anyone gripe about it... not even once. The cost is pretty negligible, and there's a lot of halo benefit (i.e., you just get more people involved on GitHub the platform itself). The fact that a lot of non-technical employees in marketing and other fields are using it for corporate blogs is actually a nice bit of pressure on the organization to make Pages and web editing even simpler for those users. It becomes harder to lean on "oh it's a developer site so they'll figure it out". Mostly, though, I think it's just a matter that we wanted it for ourselves. It's pretty awesome from an industry bystander's perspective to have something free, simple, and static, so we can all benefit from more stable docs, blogs, and so on. Maybe that'll change in the future and something Totally Different will change the industry, but for right now I think it's pretty rad, and totally worth the investment.
- sneak 11y agoYou've just optimized for my happiness. Thank you.
- spdionis 11y agoGreat example of YAGNI, related to the post by martin fowler posted earlier today on HN.
- joshrotenberg 11y agoI remember seeing a talk about a Github Pages rearchitecture at Erlang Factory in SF in 2012: http://www.erlang-factory.com/conference/SFBay2012/speakers/JesseNewland http://www.erlang-factory.com/conference/SFBay2012/speakers/... I don't see anything about those components in this post. Did that architecture never make it to production?
- jnewland 11y agoI threw out that prototype soon after the talk. At the time, there weren't a lot of other engineers at the company doing Erlang, so maintenance was considered to be a long-term problem. I'm glad we made that call.
- krallja 11y agoanswered elsewhere - https://news.ycombinator.com/item?id=9612914 https://news.ycombinator.com/item?id=9612914
- linc01n 11y agoI thought github pages is running on riak and webmachine from 2012[0]. [0] https://speakerdeck.com/jnewland/github-pages-on-riak-and-webmachine https://speakerdeck.com/jnewland/github-pages-on-riak-and-we...
- jnewland 11y agoI threw out that prototype soon after the talk. At the time, there weren't a lot of other engineers at the company doing Erlang, so maintenance was considered to be a long-term problem. I'm glad we made that call.
- misterbee 11y agoThere are so many presentations of the form "We're using Erlang/Scala/whatever, it's so awesome!", but so few followups when they give up on the idea for production..
- troutwine 11y agoIt's hard to sustain some alternate technology in the face of common knowledge. Rarely does technical advantage outweigh hard-won operational experience.
- nosequel 11y agoAs much of a fan of Erlang and Riak as I am, you did make the right call. If you only have one or two people on the team who want to know that technology, then it isn't a smart move to base a core piece of technology on it (Erlang) just because it might be the best answer. Sometimes an okay answer that everyone is familiar with is much better.
- rattray 11y agoI hadn't heard of Fastly before. How do they compare to Cloudflare?
- nullrouted 11y agoFastly is actually really great but they cost $50/month to start with. Their throughput and routing is amazing through.
- pbowyer 11y agoI hear many great things about Fastly from users, but whenever I look at CDN performance comparisons (e.g. https://cloudharmony.com/reports/editions/cdn-performance-report/basic/cdn-performance-report-0914.pdf https://cloudharmony.com/reports/editions/cdn-performance-re...) they never do brilliantly for US/EU regions. What gives?
- cagenut 11y agofastly is vi, cloudflare is emacs
- jsingleton 11y agoSomething I would really like is to be able to set the custom MIME type for an app cache manifest file. That way you could easily host offline web apps from GH pages. Anyone know a way to do this without using S3 or similar? https://en.wikipedia.org/wiki/Cache_manifest_in_HTML5#Basics https://en.wikipedia.org/wiki/Cache_manifest_in_HTML5#Basics
- ryanseys 11y agoYou shouldn't need to specify a custom one. GitHub Pages will automatically serve the file with the appropriate mime type given its file extension. Here [1] is the list. [1]: https://github.com/jekyll/jekyll/blob/master/lib/jekyll/mime.types https://github.com/jekyll/jekyll/blob/master/lib/jekyll/mime... Edit: As you can see in that link, both .manifest and .appcache file extensions map to text/cache-manifest mime type.
- jsingleton 11y agoNice. Very helpful. I must have been using the wrong (or no) extension before.
- datums 11y agoHave you thought about using bind as the db for the routing ? an internal dns lookup for the storage node storage.url -> 10.0.12.1
- kyledrake 11y agoPretty similar to how Neocities serves static sites (https://neocities.org https://neocities.org). There's a few differences. We don't use SQL in the routing chain, we use regex to pick out the site name and then serve from a directory of the same name (this is NOT as bad as it sounds, most filesystems can do this quite well now and take MUCH more than half a million sites to bottleneck). DRBD is also a little hardcore for my tastes. Nothing wrong with it, I just don't know it well, and I don't like being dependant on things I don't know how to debug. An alternative I wanted to show uses inotify, rsync and ssh combined into a simple replication daemon. It's obviously not as fast, but if you enable persistent SSH connections, it's not too bad. If it screws up, you can just run rsync. Rumor has it the Internet Archive uses an approach not too far away from this for Petabox. Check it out if you're looking for something a little more lightweight for real-time replication: https://code.google.com/p/lsyncd/ https://code.google.com/p/lsyncd/ We're still working on open sourcing (!) our serving infrastructure, so eventually you will be able to see all of the code we use for this (sans the secrets and keys, of course). I've just been having trouble coming up with a good solution for doing this. For now, enjoy the source of our web app: https://github.com/neocities/neocities https://github.com/neocities/neocities
- datums 11y agoHave you looked at http://www.cis.upenn.edu/~bcpierce/unison/ http://www.cis.upenn.edu/~bcpierce/unison/
- charliesome 11y ago> Nothing wrong with it, I just don't know it well, and I don't like being dependant on things I don't know how to debug. Yep, this is basically our approach as well. We've been using DRBD for quite a long time now on our Git fileservers (which also run in active/standby pairs - in fact, they look a lot like our Pages fileservers) so we have quite a lot of in-house experience with it and it's a technology we're pretty comfortable with. Given this, using it for the new Pages infrastructure was a pretty straight-forward decision.
- kyledrake 11y ago
- wcdolphin 11y agoI love this write up. Awesome example of how to build the minimal, simple thing first and expand as traction and needs develop.
- baghali 11y agoQuestion from Githubbers: Have considered using cluster file systems such as GlusterFS or Ceph?
- kyledrake 11y agoI looked into GlusterFS at one point. GlusterFS is a no-go for static file serving in hostile environments. It asks every node to look for a file, even if it's not there. You can imagine the DDoS attacks you could build here using a bunch of 404 requests for files that don't exist. One story I heard from a PHP dev is that it would take 30 seconds to load a page while it looked for all the files needed to run it.
- jldugger 11y agoYea, GlusterFS is terrible at PHP, or anything involving lots of small files. Like, static sites or say git repos.
- samlambert 11y agoI don't know if it has been considered, however we do have a strong pattern for using DRDB.
- ngrilly 11y agoGreat summary of your architecture. Thanks for sharing! A few questions: - Is everything in the same datacenter or in different datacenters? What happens if the datacenter is unavailable for some reason? Are data replicated somewhere? - You moved from 2 machines to at least 10 (at least 2 load balancers, 2 front ends, 1 MySQL master, 1 MySQL slave and 2 pairs of fileservers). That's a lot more. Do you need more machines because you need more capacity (to serve the growing traffic) or just because the new architecture is more distributed and requires more machines by "definition"? - I understand the standby fileservers are idle most of the time: reads go the active fileserver, only writes are replicated to the standby. Am I understanding correctly? If yes, it looks like a "wasted" capacity?
- outworlder 11y agoEven if Lua is all but a footnote, it's nice to see that, yet again, things that rely on it work as advertised.
- Sir_Cmpwn 11y agoI remember the time I mistakenly drove huge amounts of traffic to Github Pages, believing they had the infrastructure to handle it. I apologise for last year's downtime :) Glad to hear it's being improved. I'm impressed that it was able to run on such simple infrastructure for so long.
- cddotdotslash 11y agoI doubt you caused any downtime. All the page content is behind a CDN.
- Sir_Cmpwn 11y agoI did, confirmed by the angered emails from GitHub and the suspension of the relevant repository.
- sneak 11y agoGuess I'm learning Lua now. :D
- tracker1 11y agoIt seems to me, they could have gone a farther step removed via something like Cassandra. With a Cassandra cluster, they could have used a partition key that is the domain name + route in question, they could then do lookup against that entry, with the resource path (excluding querystring params) could be used to find a single resource in cassandra, and return it directly. A preliminary hit against a domain forwarder would be a good idea as well, but for those CNAME domains, dual-publishing might be a better idea... where the github name would be a pointer for said redirect. While Cassandra itself might not be quite as comfortable as say mySQL, in my mind this would have been a much better fit... Replacing the file servers and the database servers with a Cassandra cluster... Any server would be able to talk to the cluster, and be able to resolve a response, with a reduced number of round trips and requests... though the gossip in Cassandra would probably balance/reduce some of that benefit.
- samlambert 11y agoAdding a database that is new to GitHub would not be a pragmatic move.
- coolrhymes 11y agoawesome write up. I thought i was the only crazy guy to use lua to route subdomains to internal servers. Little surprised on the MySql part instead of redis like fast look up key.
- k_bx 11y agoI find that for such tasks it makes sense instead of trying to hack around nginx config and its lua scripting, to throw it away and write a small app in, say, Haskell+Warp that would do the job. It's as fast as nginx (and probably faster than nginx+lua), would have much more static guarantees, express logic more clear.