6 ms·
I think their blog being hugged to death is a good example of how this technology (read: web backend tech in general, not containers) is still too complicated f
by peterwwillis 9y ago
I think their blog being hugged to death is a good example of how this technology (read: web backend tech in general, not containers) is still too complicated for people in general to use it for simple cases, like scaling the load of otherwise static content. (It's wordpress, but still) We need solutions that require less resources and smaller learning curves.
- markbnj 9y agoI don't know how their blog is hosted but wordpress in general is very chatty with the database backend, and I really don't think the site getting crushed by HN is indicative of fundamental issues with container technology. If it is then there is a long list of technologies that HN has exposed fundamental issues with by crushing their content sites :).
- peterwwillis 9y agoOh I'm not pointing at container technology. I'm talking web backend technology in general. I mean this is 2018. No well known company's blog on AWS should ever go down at all, unless the AWS service in the region is down. The fact that it's down is not a Docker failure or a Wordpress failure or a Database failure, as there are widely known methods to scale all these things. The problem is that putting the puzzle together is just complicated enough that it's easy to get wrong. If even Docker's blog can't scale, we have bigger issues as an industry.
- markbnj 9y agoFair enough. I took "this technology" to be a reference to Docker given the context.
- andrewstuart2 9y agoIf Docker's blog was their core product, I would probably be more likely to agree. The truth is, they don't get ROI from engineering their blog to anticipate or quickly respond to all the traffic of a major announcement, because those are few and far between. The level of complexity is moot because the benefits will only outweigh a very small cost. They build container tech. That doesn't mean they will benefit from engineering the best solution for everything they do.
- peterwwillis 9y agoI'm not arguing with that, I'm saying it's dumb that we still have to work hard to get a blog to scale.
- jabv 9y agoI'm sorry, is your argument based on the current year? I presume you mean otherwise, but that's how it's written.
- TheOtherHobbes 9y agoYes, and it's a perfectly fair argument. Content hosting sites are supposed to host content. That's what they do. If a site doesn't work under unusual but not outrageous conditions, it's fundamentally broken and not fit for purpose. The industry should be much less tolerant of excuses for tools and systems that perform poorly under load. It's had a couple of decades to get this right now. How is it acceptable that this is even a problem?
- seanp2k2 9y agoStatic site generators are an infinitely better choice for making actual blogs. Wordpress has evolved into a CMS similar to Drupal or Joomla with all the available plugins. You can make it into a store or a wiki or basically anything, but you probably shouldn’t. Wordpress was really cool back in the PHP5 days, but Moveable Type and other “old school” static CMS generators always scaled better. Today with node and the dozens of good SSGs, there’s really no excuse to use WP for a blog. The content doesn’t change between page loads, so why would you server-side generate the page dynamically on each page load? The way that it works is fundamentally wrong, and while it’s not very hard to fix with Varnish or WPSuperCache...why not start with the right tool for the job?
- zenlikethat 9y agoTry getting 5+ marketing people to learn git and GitHub sometime and let me know how it goes.
- skellera 9y agoFor just the blogging portion, you can just add the blog posts directly through github (the markdown files for posts). https://help.github.com/articles/creating-new-files/ https://help.github.com/articles/creating-new-files/ But yeah, it’s less user friendly. I’m sure someone will make a good solution for this soon.
- OtterCoder 9y agoGit is not for deployment. Also, hardcoding blog posts in the repo is a dumb solution, and not necessary for a static site. The user built markup should be separate from the code that generates static pages from it.
- thinkloop 9y agoThere is a slight irony in them being an infrastructure company.
- BjoernKW 9y agoWordPress works very well with high loads if you use proper caching (via WP Super Cache, for instance) so the database doesn’t get hit for every page hit. You don’t need containers or in fact anything fancy for that. With WordPress in particular containers make some maintenance and scalability issues more difficult to deal with.
- staticassertion 9y agoOr it's completely unrelated.
- rb808 9y agoMaybe he originally set up the blog using his work login and HR auto-deleted his credentials or account. Sounds like something that would happen around here...
- nunez 9y agoI don't think Docker is too complicated; instead, I think a lot of companies misunderstand the benefit of containers aside from "my application boots faster"
- forgot-my-pw 9y agoWe need more static page generation like Jekyll. It doesn't make sense that an entire blog needs to be dynamically generated all the time. Dynamic contents can be fetched using ajax, but the rest should be static HTML.
- makapuf 9y agoStatic content can and should be cached, not necessarily pre generated. Thus you can use your site as is and handle caching with whatever you need. It also takes care of semi static (slowly moving).
- dwc 9y agoI've been mumbling under my breath about this since the 90s. The basic issue as I see it is that nobody wants to do the up front work of separating out static, client side dynamic, and server side dynamic. If that work were done then nominal load would be much less to begin with, and scaling up would be much more straightforward.
- mrec 9y agoMaybe I'm misunderstanding your categories, but why would a server care about the distinction between "static" and "client side dynamic"?
- dwc 9y agoA server would not care. It's the devs who need to separate these things.
- segmondy 9y agoMumbling since 90s? Well, if you stopped mumbling and showed the world how it ought to be done, perhaps we would be doing it your way. Show us how.
- nu-world-odor 9y agoshow us wordpressers and drupalers how to separate shit? good luck with that.
- deleted 9y ago[deleted]
- callumjones 9y agoI don't understand why companies take on hosting their blog and marketing page (usually the homepage) when it isn't their core competency and a distraction for the engineering team to maintain. It looks like this is true for Docker which seems to be hosting it themselves instead of using Wordpress.com » nslookup blog.docker.com Server: 192.168.86.1 Address: 192.168.86.1#53 Non-authoritative answer: blog.docker.com canonical name = blog-1556129762.us-east-1.elb.amazonaws.com.
- nodesocket 9y agoOr use WpEngine or FlyWheel (https://getflywheel.com https://getflywheel.com).
- merb 9y agowell if a company which tries to sell solutions for high scale , high performance solutions and fast rollout and scaling. hosts their own blog (instead of using a hosted solution) AND can't scale it to their needs, than I would say their product failed. Either because they couldn't use it in this case (because too hard, etc, whatever) OR because it basically didn't worked.
- callumjones 9y agoThey are a container company, not a scale your blog + scale your database + scale your cache + scale your security company.
- merb 9y agoThan they should‘ve hosted it somwhere else instead of on their own tech
- callumjones 9y agoSee my first comment.
- jotm 9y ago
- manigandham 9y agoWhat does this have to do with containers? They just need a CDN in front of the site.
- smallbigfish 9y ago> being hugged to death Whenever I read about this I can't help to think that people don't make a good use of a load balancer like HAProxy in front of their web-servers. Even if it's a single server behind the load balancer. There's this sweet spot where your server is handling requests as fast as it can but if you cross above it everything halts. To make it simple, this usually happens when one or more hardware resources (CPU, RAM, disk) reaches 90% utilization. So just do some basic benchmarks and find that sweet spot and put a limit in HAProxy. Say 100 concurrent connections for static assets and 50 concurrent connections for everything else. All requests above the limit will go in a HAProxy queue but they should not stay there for long because the web-server is able to work at full speed without being overloaded. PS: I have quite a dream. To write some fabulous blog post that reaches the HN front page and serve it from a raspberry pi device. Too bad I don't write at all.
- notheguyouthink 9y agoThis is informative, thank you. Any HAProxy-like software you recommend? Or is HAProxy the king?
- xigency 9y agoYou might be able to climb the ladder with a meta-post. Just do a write-up on how to host a front page post on a Raspberry Pi and tune the headline for the right audience. "This Site Can Handle 1,000,000 Connections on a Raspberry Pi" If you could collect data from the experiment and do a postmortem, that'd be great, too.
- smallbigfish 9y ago> postmortem Hehe, why so negative?
- jaggederest 9y agoI've actually dealt with this precise problem, HAProxy doesn't really solve the problem of an excess of traffic. At some point you'll start dropping connections or people will give up waiting. What you really want is something more like Varnish, a pass through caching proxy that handles known requests from memory, and any novel requests go through.