45 ms·
Use one big server
- herbst 4y agoI am using a semi big cloud VPS to host all my live services. It's 'just' a few thousand users per day over 10+ websites. The combination of Postgres, Nginx and Passenger & Cloudflare make this a easy experience. The cloud (In this case Vultr) allows on demand scaling, backups and so far I've had zero downtime because of them. In the past I've run a mixture of cloud and some dedicated servers and since migrating I have less downtime and way less work and no worse load times.
- up2isomorphism 4y agoThe author missed the most important factor why cloud is dominating the world today. It is never about the actual hardware cost. It is the cost of educating people be able to use that big server. I can guarantee you you will need to pay at least $40k a month to hire someone to be able to write and deploy softwares that can actually realize the performance he claims on that big server. And your chance to be able to find one in 2 month is closed to 0, at least in today’s job market. Also even if you find one , he can leave you in one year to some others places, and your business will be dead.
- kbenson 4y agoOne server is for a hobby, not a business. Maybe that's fine, but keep that in mind. Backups at that level are something that keeps you from losing all data, not something that keeps you running and gets you up in any acceptable timeframe for most businesses. That doesn't mean you need to use the cloud, it just means one big piece of hardware with all its single points of failure is often not enough. Two servers gets you so much more than one. You can make one a hot spare, or actually split services between them and have each be ready to take over for specific services for the other, greatly including your burst handling capability and giving you time to put more resources in place to keep n+1 redundancy going if you're using more than half of a server's resources.
- vitro 4y agoLet's Encrypt's database server [1] would beg to differ. For businesses at certain scale two servers are really an overkill. [1] https://letsencrypt.org/2021/01/21/next-gen-database-servers.html https://letsencrypt.org/2021/01/21/next-gen-database-servers...
- mh- 4y agoThat says they use a single database, as in a logical MySQL database. I don't see any claim that they use a single server. In fact, the title of the article you've linked suggests they use multiple.
- simonw 4y agohttps://letsencrypt.status.io/ https://letsencrypt.status.io/ shows a list of their servers, which look to be spread across three data centers (one "public", two "high availability").
- kbenson 4y agoDo we know if it shows cold spares? That's all I think is needed at a minimum to avoid the problems I'm talking about, and I doubt they would note those if they don't necessarily have a hostname.
- kbenson 4y agoDo they actually say they don't have a slave to that database ready to take over? I seriously doubt Let's Encrypt has no spare. Note I didn't say you shouldn't run one service (as in daemon) or set of services from one box, just that one box is not enough and you need that spare. It Let's Encrypt actually has no spare for their database server and they're one hardware failure away from being down for what may be a large chunk of time (I highly doubt it), then I wouldn't want to use them even if free. Thankfully, I doubt your interpretation of what that article is saying.
- 4y ago
- kubb 4y agoAs per usual, don't copy Google if you don't have the same requirements. Google Search never goes down. HN goes down from time and nobody minds. Google serves tens (hundreds?) of thousands of queries per second. HN serves ten. HN is fine with one server because it's small. How big is your service going to be? Do that boring math :)
- FartyMcFarter 4y agoEven Google search has gone down apparently, for five minutes in 2013: https://www.cnet.com/tech/services-and-software/google-goes-down-for-5-minutes-internet-traffic-drops-40/ https://www.cnet.com/tech/services-and-software/google-goes-...
- terafo 4y agoThere were huge availability issues as recent as December 14th 2020, for 45 minutes.
- roflyear 4y agoCorrect. I like to ask "how much money do we lose if the site goes down for 1hr? a day?" etc.. and plan around that. If you are losing 1m an hour, or 50m if it goes down for a day, hell yeah you should spend a few million on making sure your site stays online! But, it is amazing how often c-levels cannot answer this question!
- dugmartin 4y agoI think Elixir/Erlang is uniquely positioned to get more traction in the inevitable microservice/kubernetes backlash and the return to single server deploys (with a hot backup). Not only does it usually sip server resources but it also scales naturally as more cores/threads are available on a server.
- dboreham 4y agoThis is the norm. It's only weird things like Node.js and Ruby that don't have this property.
- hunterloftis 4y agoWhile individual Node.js processes are single-threaded, Node.js includes a standard API that distributes its load across multiple processes, and therefor cores. - https://nodejs.org/api/cluster.html#cluster https://nodejs.org/api/cluster.html#cluster
- lliamander 4y agoGoing from an Erlang "monolith" to a java/k8s cluster, I was amazed at how much more work it is takes to build a "modern" microservice. Erlang still feels like the future to me.
- dougmoscrop 4y agoCan you imagine if even a fraction of the effort poured in to k8s tooling had gone in to the Erlang/OTP ecosystem instead?
- nostrebored 4y agoAs someone who's worked in cloud sales and no longer has any skin in the game, I've seen firsthand how cloud native architectures improve developer velocity, offer enhanced reliability and availability, and actually decrease lock-in over time. Every customer I worked with who had one of these huge servers introduced coupling and state in some unpleasant way. They were locked in to persisted state, and couldn't scale out to handle variable load even if they wanted to. Beyond that, hardware utilization became contentious at any mid-enterprise scale. Everyone views the resource pool as theirs, and organizational initiatives often push people towards consuming the same types of resources. When it came time to scale out or do international expansion, every single one of my customers who had adopted this strategy had assumptions baked into their access patterns that made sense given their single server. When it came time to store some part of the state in a way that made sense for geographically distributed consumers, it was months not sprints of time spent figuring out how to hammer this in to a model that's fundamentally at odds. From a reliability and availability standpoint, I'd often see customers tell me that 'we're highly available within a single data center' or 'we're split across X data centers' without considering the shared failure modes that each of these data centers had. Would a fiber outage knock out both of your DCs? Would a natural disaster likely knock something over? How about _power grids_? People often don't realize the failure modes they've already accepted. This is obviously not true for every workload. It's tech, there are tradeoffs you're making. But I would strongly caution any company that expects large growth against sitting on a single-server model for very long.
- secabeen 4y agoThe common element in the above is scaling and reliability. While lots of startups and companies are focused on the 1% chance that they are the next Google or Shopify, the reality is that nearly all aren't, and the overengineering and redundancy-first model that cloud pushes does cost them a lot of runway. It's even less useful for large companies; there is no world in which Kellogg is going to increase sales by 100x, or even 10x.
- nostrebored 4y agoBut most companies aren't startups. Many companies are established, growing businesses with a need to be able to easily implement new initiatives and products. The benefits of cloud for LE are completely different. I'm happy to break down why, but I addressed the smb and mid-enterprise space here because most large enterprises already know they shouldn't run on a single rack.
- stevenjgarner 4y agoIf you are not maxing out or even getting above 50% utilization of 128 physical cores (256 threads), 512 GB of memory, and 50 Gbps of bandwidth for $1,318/month, I really like the approach of multiple low-end consumable computers as servers. I have been using arrays of Intel NUCs at some customer sites for years with considerable cost savings over cloud offerings. Keep an extra redundant one in the array ready to swap out a failure. Another often overlooked option is that in several fly-over states it is quite easy and cheap to register as a public telecommunication utility. This allows you to place a powered pedestal in the public right-of-way, where you can get situated adjacent to an optical meet point and get considerable savings on installation costs of optical Internet, even from a tier 1 provider. If your server bandwidth is peak utilized during business hours and there is an apartment complex nearby you can use that utility designation and competitively provide residential Internet service to offset costs.
- erichocean 4y ago> it is quite easy and cheap to register as a public telecommunication utility Is North Carolina one of those states? I'm intrigued…
- stevenjgarner 4y agoI have only done a few midwestern states. Call them and ask [0] - (919) 733-7328. You may want to first call your proposed county commissioner's office or city hall (if you are not rural), and ask them who to talk with about a new local business providing Internet service. If you can show the Utilities Commission that you are working with someone at the local level I have found they will treat you more seriously. In certain rural counties, you can even qualify for funding from the Rural Utilities Service of the USDA. [0] https://www.ncuc.net/ https://www.ncuc.net/ EDIT: typos + also most states distinguish between facilities-based ISP's (ie with physical plant in the regulated public right-of-way) and other ISPs. Tell them you are looking to become a facilities-based ISP.
- erichocean 4y ago
- notacoward 4y agoAt various points in my career, I worked on Very Big Machines and on Swarms Of Tiny Machines (relative to the technology of their respective times). Both kind of sucked. Different reasons, but sucked nonetheless. I've come to believe that the best approach is generally somewhere in the middle - enough servers to ensure a sufficient level of protection against failure, but no more to minimize coordination costs and data movement. Even then there are exceptions. The key is don't run blindly toward the extremes. Your utility function is probably bell shaped, so you need to build at least a rudimentary model to explore the problem space and find the right balance.
- mamcx 4y agoYes, totally. Among the setups the one that I think is the golden is BIG Db Server, 1-4 front-end(web/api/cache) servers. Off-hand the backups and CDN. That is.
- gregmac 4y ago> However, cloud providers have often had global outages in the past, and there is no reason to assume that cloud datacenters will be down any less often than your individual servers. A nice thing about being in a big provider is when they go down a massive portion of the internet goes down, and it makes news headlines. Users are much less likely to complain about your service being down when it's clear you're just caught up in the global outage that's affecting 10 other things they use.
- ramesh31 4y agoNobody ever got fired for buying IBM!
- notjustanymike 4y agoWe may need to update this one, I would definitely fire someone today for buying IBM.
- dtparr 4y agoThese days we just call it licensing Red Hat.
- kkielhofner 4y agoNobody ever got fired for buying AWS!
- lanstin 4y agoThe AWS people now are just like the IBM people in the 80s - mastering a complex and not standards based array of products and optional product add-ons. The internet solutions were open and free for a few decades and now it’s AWS SNADS I mean AWS load balancers and edge networks.
- namose 4y agoAWS services are usually based on standards anyway. If you use an architecturally sound approach to AWS you could learn to develop for GCP or Azure pretty easily.
- lordleft 4y agoInteresting write-up that acknowledges the benefits of cloud computing while starkly demonstrating the value proposition of just one powerful, on-prem server. If it's accurate, I think a lot of people are underestimating the mark-up cloud providers charge for their services. I think one of the major issues I have with moving to the cloud is a loss of sysadmin knowledge. The more locked in you become to the cloud, the more that knowledge atrophies within your organization. Which might be worth it to be nimble, but it's a vulnerability.
- phpisthebest 4y agoGiven that AWS holds up the entire Amazon Company, and is a large part of Bezo's personal wealth, I think the market up is pretty good.
- reillyse 4y agoNope. Multiple small servers. 1) you need to get over the hump and build in multiple servers into your architecture from the get go (the author says you need two servers minimum), so really we are talking about two big servers. 2) having multiple small servers allows us to spread our service into different availability zones 3) multiple small servers allows us to do rolling deploys without bringing down our entire service 4) once we use the multiple small servers approach it’s easy to scale up and down our compute by adding or removing machines. Having one server it’s difficult to scale up or down without buying more machines. Small servers we can add incrementally but with the large server approach scaling up requires downtime and buying a new server.
- Nextgrid 4y ago> you need to get over the hump and build in multiple servers into your architecture from the get go (the author says you need two servers minimum), so really we are talking about two big servers. Managing a handful of big servers can be done manually if needed - it's not pretty but it works and people have been doing it just fine before the cloud came along. If you intentionally plan on having dozens/hundreds of small servers, manual management becomes unsustainable and now you need a control plane such as Kubernetes, and all the complexity and failure modes it brings. > having multiple small servers allows us to spread our service into different availability zones So will 2 big servers in different AZs (whether cloud AZs or old-school hosting providers such as OVH). > multiple small servers allows us to do rolling deploys without bringing down our entire service Nothing prevents you from starting multiple instances of your app on one big server nor doing rolling deploys with big bare-metal assuming one server can handle the peak load (so you take out your first server out of the LB, upgrade it, put it back in the LB, then do the same for the second and so on). > once we use the multiple small servers approach it’s easy to scale up and down our compute by adding or removing machines. Having one server it’s difficult to scale up or down without buying more machines. Small servers we can add incrementally but with the large server approach scaling up requires downtime and buying a new server. True but the cost premium of the cloud often offsets the savings of autoscaling. A bare-metal capable of handling peak load is often cheaper than your autoscaling stack at low load, therefore you can just overprovision to always meet peak load and still come out ahead.
- tiffanyh 4y agoDedicated servers are hugely under valued / under appreciated.
- throwaway787544 4y agoI have been doing this for two decades. Let me tell you about bare metal. Back in the day we had 1,000 physical servers to run a large scale web app. 90% of that capacity was used only for two months. So we had to buy 900 servers just to make most of our money over two events in two seasons. We also had to have 900 servers because even one beefy machine has bandwidth and latency limits. Your network switch simply can't pump more than a set amount of traffic through its backplane or your NICs, and the OS may have piss-poor packet performance too. Lots of smaller machines allow easier scaling of network load. But you can't just buy 900 servers. You always need more capacity, so you have to predict what your peak load will be, and buy for that. And you have to do it well in advance because it takes a long time to build and ship 900 servers and then assemble them, run burn-in, replace the duds, and prep the OS, firmware, software. And you have to do this every 3 years (minimum) because old hardware gets obsolete and slow, hardware dies, disks die, support contracts expire. But not all at once, because who knows what logistics problems you'd run into and possibly not get all the machines in time to make your projected peak load. If back then you told me I could turn on 900 servers for 1 month and then turn them off, no planning, no 3 year capital outlay, no assembly, burn in, software configuration, hardware repair, etc etc, I'd call you crazy. Hosting providers existed but nobody could just give you 900 servers in an hour, nobody had that capacity. And by the way: cloud prices are retail prices. Get on a savings plan or reserve some instances and the cost can be half. Spot instances are a quarter or less the price. Serverless is pennies on the dollar with no management overhead. If you don't want to learn new things, buy one big server. I just pray it doesn't go down for you, as it can take up to several days for some cloud vendors to get some hardware classes in some regions. And I pray you were doing daily disk snapshots, and can get your dead disks replaced quickly.
- taylodl 4y agoThat's a good point about cloud services being retail. My company gets a very large discount from one of the most well-known cloud providers. This is available to everybody - typically if you commit to 12 months of a minimum usage then you can get substantial discounts. What I know is so far everything we've migrated to the cloud has resulted in significantly reduced total costs, increased reliability, improved scalability, and is easier to enhance and remediate. Faster, cheaper, better - that's been a huge win for us!
- faizshah 4y agoIn the paper on Twitter’s “Who to Follow” service they mention that they designed the service around storing the entire twitter graph in the memory of a single node: > An interesting design decision we made early in the Wtf project was to assume in-memory processing on a single server. At first, this may seem like an odd choice, run- ning counter to the prevailing wisdom of “scaling out” on cheap, commodity clusters instead of “scaling up” with more cores and more memory. This decision was driven by two rationales: first, because the alternative (a partitioned, dis- tributed graph processing engine) is significantly more com- plex and dicult to build, and, second, because we could! We elaborate on these two arguments below. > Requiring the Twitter graph to reside completely in mem- ory is in line with the design of other high-performance web services that have high-throughput, low-latency require- ments. For example, it is well-known that Google’s web indexes are served from memory; database-backed services such as Twitter and Facebook require prodigious amounts of cache servers to operate smoothly, routinely achieving cache hit rates well above 99% and thus only occasionally require disk access to perform common operations. However, the additional limitation that the graph fits in memory on a single machine might seem excessively restrictive. I always wondered if they still do this and if this influenced any other architectures at other companies. Paper: https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.699.1726&rep=rep1&type=pdf https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.69...
- 3pt14159 4y agoYeah I think single machine has its place, and I once sped up a program by 10000x by just converting it to Cython and having it all fit in the CPU cache, but the cloud still does have a place! Even for non-bursty loads. Even for loads that theoretically could fit in a single big server. Uptime. Or are you going to go down as all your workers finish? Long connections? Etc. It is way easier to gradually handover across multiple API servers as you do an upgrade than it is to figure out what to do with a single beefy machine. I'm not saying it is always worth it, but I don't even think about the API servers when a deploy happens anymore. Furthermore if you build your whole stack this way it will be non-distributed by default code. Easy to transition for some things, hell for others. Some access patterns or algorithms are fine when everything is in a CPU cache or memory but would fall over completely across multiple machines. Part of the nice part about starting with cloud first is that it is generally easier to scale to billions of people afterwards. That said, I think the original article makes a nuanced case with several great points and I think your highlighting of the Twitter example is a good showcase for where single machine makes sense.
- porker 4y agoI like One Big (virtual) Server until you come to software updates. At a current project we have one server running the website in production. It runs an old version of Centos, the web server, MySQL and Elasticsearch all on the one machine. No network RTTs when doing too many MySQL queries on each page - great! But when you want to upgrade one part of that stack... we end up cloning the server, upgrading it, testing everything, and then repeating the upgrade in-place on the production server. I don't like that. I'd far rather have separate web, DB and Elasticsearch servers where each can be upgraded without fear of impacting the other services.
- rlpb 4y agoYou could just run system containers (eg. lxd) for each component, but still on one server. That gets you multiple "servers" for the purposes of upgrades, but without the rest of the paradigm shift that Docker requires.
- porker 4y agoDoesn't the container help with versioning the software inside it, but you're still tied to the host computer's operating system, and so when you upgrade that you have to test every single container to see if anything broke? Whereas if running a VM you have a lot more OS upgrades to do, but you can do them individually and they have no other impact? This is the bit I've never understood with containers...
- ansible 4y agoI use LXC a lot for our relatively small production setup. And yes, I'm treating the servers like pets, not cattle. What's nice is that I can snapshot a container and move it to another physical machine. Handy for (manual) load balancing and upgrades to the physical infrastructure. It is also easy to run a snapshot of the entire server and then run an upgrade, then if the upgrade fails, you roll back to the old snapshot.
- 0xbadcafebee 4y agoWhich is great until there's a security vuln in an end-of-life piece of core software (the distro, the kernel, lxc, etc) and you need to upgrade the whole thing, and then it's a 4+ week slog of building a new server, testing the new software, fixing bugs, moving the apps, finding out you missed some stuff and moving that stuff, shutting down the old one. Better to occasionally upgrade/reinstall the whole thing with a script and get used to not making one-off changes on servers. If I were to buy one big server, it would be as a hypervisor. Run Xen or something and that way I can spin up and down VMs as I choose, LVM+XFS for snapshots, logical disk management, RAID, etc. But at that point you're just becoming a personal cloud provider; might as well buy smaller VMs from the cloud with a savings plan, never have to deal with hardware, make complex changes with a single API call. Resizing an instance is one (maybe two?) API call. Or snapshot, create new instance, delete old instance: 3 API calls. Frickin' magic. "the EC2 Instance Savings Plans offer up to 72% savings compared to On-Demand pricing on your Amazon EC2 Instances" - https://aws.amazon.com/savingsplans/ https://aws.amazon.com/savingsplans/
- taylodl 4y agoIf you're hosting on-prem then you have a cluster to configure and manage, you have multiple data centers you need to provision, you need data backups you have to manage plus the storage required for all those backups. Data centers also require power, cooling, real estate taxes, administration - and you need at least two of them to handle systemic outages. Now you have to manage and coordinate your data between those data centers. None of this is impossible of course, companies have been doing this everyday for decades now. But let's not pretend it doesn't all have a cost - and unless your business is running a data center, none of these costs are aligned with your business' core mission. If you're running a start-up it's pretty much a no-brainer you're going to start off in the cloud. What's the real criteria to evaluate on-prem versus the cloud? Load consistency. As the article notes, serverless cloud architectures are perfect for bursty loads. If your traffic is highly variable then the ability to quickly scale-up and then scale-down will be of benefit to you - and there's a lot of complexity you don't have to manage to boot! Generally speaking such a solution is going to be cheaper and easier to configure and manage. That's a win-win! If your load isn't as variable and you therefore have cloud resources always running, then it's almost always cheaper to host those applications on-prem - assuming you have on-prem hosting available to you. As I noted above, building data centers isn't cheap and it's almost always cheaper to stay in the cloud than it is to build a new data center, but if you already have data center(s) then your calculus is different. Another thing to keep in mind at the moment is even if you decide to deploy on-prem you may not be able to get the hardware you need. A colleague of mine is working on a large project that's to be hosted on-prem. It's going to take 6-12 months to get all the required hardware. Even prior to the pandemic the backlog was 3-6 months because the major cloud providers are consuming all the hardware. Vendors would rather deal with buyers buying hardware by the tens of thousands than a shop buying a few dozen servers. You might even find your hardware delivery date getting pushed out as the "big guys" get their orders filled. It happens.
- ozim 4y agoYou know you can run a server in the cellar under your stairs. You know that if you are a startup you can just keep servers in a closet and hope that no one turns on coffee machine while airco runs because it will pop circuit breakers, which will take down your server or maybe you might have UPS at least so maybe not :) I have read horror stories about companies having such setups. While they don't need multiple data centers, power, cooling and redundancy sounds for them like some kind of STD - getting cheap VPS should be default for such people. That is a win as well.
- cal85 4y ago> In comparison, buying servers takes about 8 months to break even compared to using cloud servers, and 30 months to break even compared to renting. Can anyone help me understand why the cloud/renting is still this expensive? I'm not familiar with this area, but it seems to me that big data centers must have some pretty big cost-saving advantages (maintenance? heat management?). And there are several major providers all competing in a thriving marketplace, so I would expect that to drive the cost down. How can it still be so much cheaper to run your own on-prem server?
- marcosdumay 4y agoRenting is not very expensive. 30 months is a large share of a computer's lifetime, and you are paying for space, electricity, and internet access too.
- klysm 4y agoThe huge capital required to get a data center with those cost savings serves as a nice moat to let people price things high.
- WJW 4y agoSeveral points: - The price for on-prem conveniently omits costs for power, cooling, networking, insurance and building space, it's only the purchase price. - The price for the cloud server includes (your share of) the costs of replacing a broken power supply or hard drive, which is not included in the list price for on-prem. You will have to make sure enough of your devs know how to do that or else hire a few sysadmin types. - As the article already mentions, the cloud has to provision for peak usage instead of average usage. If you buy an on-prem server you always have the same amount of computing power available and can't scale up quickly if you need 5x the capacity because of a big event. That kind of flexibility costs money.
- cal85 4y agoThank you, that explains it.
- zucker42 4y ago
- softfalcon 4y agoSo... I guess these folks haven't heard of latency before? Fairly sure you have to have "one big server" in every country if you do this. I feel like that would get rather costly compared to geographically distributed cloud services long term.
- Nextgrid 4y agoAs opposed, to "many small servers" in every country? The vast majority of startups out there run out of a single AWS region with a CDN caching read-only content. You can apply the same CDN approach to a bare-metal server.
- softfalcon 4y agoYeah, but if I'm a startup and running only a small server, the cloud hosting costs are minimal. I'm not sure how you think it's cheaper to host tiny servers in lots of countries and pay someone to manage that for you. You'll need IT in every one of those locations to handle the service of your "small servers". I run services globally for my company, there is no way we could do it. The fact that we just deploy containers to k8s all over the world works very well for us. Before you give me the "oh k8s, well you don't know bare metal" please note that I'm an old hat that has done the legacy C# ASP.NET IIS workflows on bare metal for a long time. I have learned and migrated to k8s on AWS/GCloud and it is a huge improvement compared to what I used to deal with. Lastly, as for your CDN discussion, we don't just host CDN's globally. We also host geo-located DB + k8s pods. Our service uses web sockets and latency is a real issue. We can't have 500 ms ping if we want to live update our client. We choose to host locally (in what is usually NOT a small server) so we get optimal ping for the live-interaction portion of our services that are used by millions of people every day.
- kkielhofner 4y agoYou don't need IT in every location or even different hosting facility contracts. Most colo hosting companies have multiple regions. From the 800lb gorilla (Equinix): https://www.equinix.com/data-centers https://www.equinix.com/data-centers Or a smaller US focused colo provider: https://www.coresite.com/data-centers/locations https://www.coresite.com/data-centers/locations Between vendor (Dell, HP, IBM, etc) and the remote hands offered by the hosting facility you don't ever have to have a member of your team even enter a facility. Anywhere. Depending on the warranty/support package the vendor will dispatch someone to show up to the facility to replace failed components with little action from you. The vendor will be happy to ship the server directly to the facility (anywhere) and for a nominal fee the colo provider will rack it and get IPMI, iLo, IP KVM, whatever up for you to do your thing. When/if something ever "hits the fan" they have on site 24 hour "remote hands" that can either take basic pre-prescribed steps/instructions -or- work with your team directly and remotely. Interestingly, at my first startup we had a facility in the nearest big metro area that not only hosted our hardware but also provided an easy, cheap, and readily available meeting space: https://www.coresite.com/data-centers/data-center-design/amenities https://www.coresite.com/data-centers/data-center-design/ame...
- dekhn 4y agoScience advances as RAM on a single machine increases. For many years, genomics software was non-parallel and depending on having a lot of RAM- often a terabyte or more- to store data in big hash tables. Converting that to distributed computing was a major effort and to this day many people still just get a Big Server With Lots of Cores, RAM, and SSD. Personally after many years of working wiht distributed, I absolutely enjoy working on a big fat server that I have all to myself.
- notacoward 4y ago> Science advances as RAM on a single machine increases. Also as people learn that correlation does not equal causation. ;)
- bee_rider 4y agoOn the other hand in science, it sure is annoying that the size of problems that fit in a single node is always increasing. PARDISO running on a single node will always be nipping at your heels if you are designing a distributed linear system solver...
- meeks 4y agoThe whole argument comes down to bursty vs. non-bursty workloads. What type of workloads make up the fat part of the distribution? If most use cases are bursty (which I would argue they are) then the author's argument only applies for specific applications. Therefore, most people do indeed see cost benefits from the cloud.
- cfors 4y agoYep, there's a premium on making your architecture more cloudy. However, the best point for Use One Big Server is not necessarily running your big monolithic API server, but your database. Use One Big Database. Seriously. If you are a backend engineer, nothing is worse than breaking up your data into self contained service databases, where everything is passed over Rest/RPC. Your product asks will consistently want to combine these data sources (they don't know how your distributed databases look, and oftentimes they really do not care). It is so much easier to do these joins efficiently in a single database than fanning out RPC calls to multiple different databases, not to mention dealing with inconsistencies, lack of atomicity, etc. etc. Spin up a specific reader of that database if there needs to be OLAP queries, or use a message bus. But keep your OLTP data within one database for as long as possible. You can break apart a stateless microservice, but there are few things as stagnant in the world of software than data. It will keep you nimble for new product features. The boxes that they offer on cloud vendors today for managed databases are giant!
- hoosieree 4y agoThis is the macro version of the von Neumann bottleneck.
- Closi 4y agoBreaking apart a stateless microservice and then basing it around a giant single monolithic database is pretty pointless - at that stage you might as well just build a monolith and get on with it as every microservice is tightly coupled to the db.
- kkielhofner 4y agoGreat article overall with many good points worth considering. Nothing is one size fits all so I won't get into the crux of the article: "just get one big server". I recently posted a comment breaking down the math for my situation: https://news.ycombinator.com/item?id=32250470#32253635 https://news.ycombinator.com/item?id=32250470#32253635 For the most "extreme" option of buying your own $40k server from Dell I'm always surprised at how many people don't consider leasing. No matter what it breaks the cost into an operating expense vs a capital one which is par with the other options in terms of accounting and doesn't require laying out $40k. Adding on that, in the US we have some absolutely wild tax advantages for large "capital expenditures" that also apply to leasing: https://www.section179.org/section_179_leases/ https://www.section179.org/section_179_leases/
- bob1029 4y ago> 1 million IOPS on a NoSQL database I have gone well beyond this figure by doing clever tricks in software and batching multiple transactions into IO blocks where feasible. If your average transaction is substantially smaller than the IO block size, then you are probably leaving a lot of throughput on the table. The point I am trying to make is that even if you think "One Big Server" might have issues down the road, there are always some optimizations that can be made. Have some faith in the vertical. This path has worked out really well for us over the last ~decade. New employees can pick things up much more quickly when you don't have to show them the equivalent of a nuclear reactor CAD drawing to get started.
- mathisonturing 4y ago> batching multiple transactions into IO blocks where feasible. If your average transaction is substantially smaller than the IO block size, then you are probably leaving a lot of throughput on the table. Could you expand on this? A quick Google search didn't help. Link to an article or a brief explanation would be nice!
- bob1029 4y agoSure. If you are using some micro-batched event processing abstraction, such as the LMAX Disruptor, you have an opportunity to take small batches of transactions and process them as a single unit to disk. For event sourcing applications, multiple transactions can be coalesced into a single IO block & operation without much drama using this technique. Surprisingly, this technique also lowers the amount of latency that any given user should experience, despite the fact that you are "blocking" multiple users to take advantage of small batching effects.
- lanstin 4y agoI didn’t see a point of cloudy services being easier to manage. If some team gets a capital budget to buy that one big server, they will put every thing on it, no matter your architectural standards. Cron jobs editing state on disk, tmux sessions shared between teams, random web servers doing who knows what, non-DBA team Postgres installs, etc. at least in cloud you can limit certain features and do charge back calculations. Not sure if that is a net win for cloud or physical, of course, but I think it is a factor
- kgeist 4y agoOne of our projects uses 1 big server and indeed, everyone started putting everything on it (because it's powerful): the project itself, a bunch of corporate sites, a code review tool, and god knows what else. Last week we started having issues with the projects going down because something is overloading the system and they still can't find out what exactly without stopping services/moving them to a different machine (fortunately, it's internal corporate stuff, not user-facing systems). The main problem I've found with this setup is that random stuff can accumulate with time and then one tool/process/project/service going out of control can bring down the whole machine. If it's N small machines, there's greater isolation.
- pclmulqdq 4y agoIt sounds like you need some containers.
- locutous 4y agoI believe that the "one big server" is intended for an application rather than trying to run 500 applications. Does your application run on a single server? If yes. Don't use a distributed system for it's architecture or design. Simply buy bigger hardware when necessary. Because the top end of servers are insanely big and fast. It does not mean, IMHO, throw everything on a single system without suitable organization, oversight, isolation, and recovery plans.
- MrStonedOne 4y ago/tg/station, the largest open source multiplayer video game on github, gets cloudheads trying to help us "modernize" the game server for the cloud all the time. Here's how that breaks down: The servers (sorry, i mean compute) cost the same (before bandwidth, more on that at the bottom) to host one game server as we pay (amortized) per game server to host 5 game servers on a rented dedicated server. ($175/month for the rented server with 64gb of ram and a 10gbit uplink) They run twice as slow because high core count slow clock speed servers aren't all they are cracked up to be, and our game engine is single threaded, but even if it wasn't, there is an overhead to multithreading things which combined with most high core count servers also having slow clock speed, rarely squares out to an actual increase in real world performance. You can get the high clock speed units, they are twice to three times as expensive. And still run 20% slower over windows vms on rented bare metal because the sad fact is enterprise cpus by either intel or amd have slower clock speeds and single threaded performance then their gaming cpu counterparts, and getting gaming cpus for rented servers is piss easy, but next to impossible for cloud servers. Each game server uses 2tb of bandwidth to host 70 player high pops. This works with 5 servers on 1 machine because our hosting provider gives us 15tb of bandwidth included in the price of the server. Well now the cloud bill just got a new 0. 10 to 30x more expensive once you remember to price in bandwidth isn't looking too great. "but it would make it cheaper for small downstreams to start out" until another youtuber mentions our tiny game, and every game server is hitting the 120 hard pop cap, and a bunch of downstreams get a surprise 4 digit bill for what would normally run 2 digits. The take away from this being that even adding in docker or k8s deployment support to the game server is seen as creating the risk some kid bankrupts themselves trying to host a game server of their favorite game off their mcdonalds paycheck, and we tell such tech "pros" to sod off with their trendy money wasters.
- mwcampbell 4y ago> $175/month for the rented server with 64gb of ram and a 10gbit uplink) Wow, what provider is that?
- corford 4y agoHetzner's PX line offers 64GB ECC RAM, Xeon CPU, dual 1TB NVME for < $100/month. A dedicated 10Gbit b/w link (plus 10Gbit NIC) is then an extra ~$40/month on top (incls. 20TB/month traffic, with overage billed at $1/TB).
- bilekas 4y agoI don't agree with EVERYTHING in the article such as getting 2 big rather than multiple smaller, this is really just a cost/requirement issue though. The biggest cost I've noticed with enterprises who go full cloud is that they are locked in for the long term. I don't mean contractually though, basically the way they design and implement any system or service MUST follow the providers "way" this can be very detrimental for leaving the provider or god forbid the provider decides to sunset certain service versions etc. That said, for enterprise it can make a lot of sense and the article covers it well by admitting some "clouds" are beneficial. For anything I've ever done outside of large businesses the go to has always been "if it doesn't require a SRE to maintain, just host your own".
- malkia 4y agoIt was all good, until NUMA came, and now you have to careful rethought your process, or you get lots of performance issues in your (otherwise) well threaded code. Speaking from first-hand experience, when our level editor ended up being used by artists on a server class machine, and supposedly 4x faster machine was actually going 2x slower (why, lots of std::shared_ptr<> use on our side, or any atomic reference counting) caused slowdowns, as the cache (my understanding) had to be synchronized between the two physical CPUs each having 12 threads. But really not the only issue, just pointing out - that you can't expect everything to scale smoothly there, unless well thought, like ask your OS to allocate your threads/memory only on one of the physical CPUS (and their threads), and somehow big disconnected part of your process(es) on the other one(s), and make sure the communication between them is minimal.. which actually wants micro-services design again at that level. so why not go with micro-services instead...
- jedberg 4y agoI'm a huge advocate of cloud services, and have been since 2007 (not sure where this guy got 2010 as the start of the "cloud revolution"). That out of the way, there is something to be said for starting off with a monolith on a single beefy server. You'll definitely iterate faster. Where you'll get into trouble is if you get popular quickly. You may run into scaling issues early on, and then have to scramble to scale. It's just a tradeoff you have to consider when starting your project -- iterate quickly early and then scramble to scale, or start off more slowly but have a better ramping up story. One other nitpick I had is that OP complains that even in the cloud you still have to pay for peak load, but while that's strictly true, it's amortized over so many customers that you really aren't paying for it unless you're very large. The more you take advantage of auto-scaling, the less of the peak load you're paying. The customers who aren't auto-scaling are the ones who are covering most of that cost. You can run a pretty sizable business in the free tier on AWS and let everyone else subsidize your peak (and base!) costs.
- rmbyrro 4y agoIsn't this simplistic? It really depends on the service, how it is used, the shape of the data generated/consumed, what type of queries are needed, etc. I've worked for a startup that hit scaling issues with ~50 customers. And have seen services with +million users on a single machine. And what does "quickly" and "popular" even mean? It also depends a lot on the context. We need to start discussing about mental models for developers to think of scaling in a contextual way.
- Phil_Latio 4y ago> Where you'll get into trouble is if you get popular quickly. You may run into scaling issues early on Did it ever occur to you that you can still use the cloud for on demand scaling? =)
- jedberg 4y agoSure but only if you architect it that way, which most people don't if they're using one big beefy server, because the whole reason they're doing that is to iterate quickly. It's hard to build something that can bust to the cloud while moving quickly. Also, the biggest issue is where your data is. If you want to bust to the cloud, you'll probably need a copy of your data in the cloud. Now you aren't saving all that much money anymore and adding in architectural overhead. If you're going to bust to the cloud, you might as well just build in the cloud. :)
- _nhh 4y agoI agree
- jvanderbot 4y agoNo thanks. I have a few hobby sites, a personal vanity page, and some basic CPU expensive services that I use. Moving to Aws server-less has saved me so much headache with system updates, certificate management, archival and backup, networking, and so much more. Not to mention with my low-but-spikey load, my breakeven is a long way off.
- soruly 4y agothat's why letsencrypt use a single database on a powerful server https://letsencrypt.org/2021/01/21/next-gen-database-servers.html https://letsencrypt.org/2021/01/21/next-gen-database-servers...
- wahnfrieden 4y agoI've started augmenting one big server with iCloud (CloudKit) storage, specifically syncing local Realm DBs to the user's own iCloud storage. Which means I can avoid taking custody of PII/problematic data, can include non-custodial privacy in product value/marketing, and means I can charge enough of a premium for the one big server to keep it affordable. I know how to scale servers in and out, so I feel the value of avoiding all that complexity. This is a business approach that leans into that, with a way to keep the business growing with domain complexity/scope/adoption (iCloud storage, probably other good APIs like this to work with along similar lines).
- PeterCorless 4y agoWe have a different take on running "one big database." At ScyllaDB we prefer vertical scaling because you get better utilization of all your vCPUs, but we still will keep a replication factor of 3 to ensure that you can maintain [at least] quorum reads and writes. So we would likely recommend running 3x big servers. For those who want to plan for failure, though, they might prefer to have 6x medium servers, because then the loss of any one means you don't take as much of a "torpedo hit" when any one server goes offline. So it's a balance. You want to be big, but you don't want to be monolithic. You want an HA architecture so that no one node kills your entire business. I also suggest that people planning systems create their own "torpedo test." We often benchmark to tell maximal optimum performance, presuming that everything is going to go right. But people who are concerned about real-world outage planning may want to "torpedo" a node to see how a 2-out-of-3-nodes-up cluster operates, versus a 5-out-of-6-nodes-up cluster. This is like planning for major jets, to see if you can work with 2 of 3 engines, or 1 of 2. Obviously, if you have 1 engine, there is nothing you can do if you lose that single point of failure. At that point, you are updating your resume, and checking on the quality of your parachute.
- pclmulqdq 4y agoI think this is the right approach, and I really admire the work you do at ScyllaDB. For something truly critical, you really do want to have multiple nodes available (at least 2, and probably 3 is better). However, you really should want to have backup copies in multiple datacenters, not just the one. Today, if I were running something that absolutely needed to be up 24/7, I would run a 2x2 or 2x3 configuration with async replication between primary and backup sites.
- PeterCorless 4y agoExactly. Regional distribution can be vital. Our customer Kiwi.com had a datacenter fire. 10 of their 30 nodes were turned to a slag heap of ash and metal. But 20 of 30 nodes in their cluster were in completely different datacenters so they lost zero data and kept running non-stop. This is a rare story, but you do NOT want to be one of the thousands of others that only had one datacenter, and their backups were also stored there and burned up with their main servers. Oof! https://www.scylladb.com/2021/03/23/kiwi-com-nonstop-operations-with-scylla-even-through-the-ovhcloud-fire/ https://www.scylladb.com/2021/03/23/kiwi-com-nonstop-operati...
- tonymet 4y agopeople don't account for the cpu & wall-time cost of encode-decode. I've seen it take up 70% of cpu on a fleet. That means 700/1000 servers are just doing encode decode. You can see high efficiency setups like stackexchange & hackernews are orders of magnitude more efficient.
- pclmulqdq 4y agoThis is exactly correct. If you have a microservice running a Rest API, you are probably spending most of your CPU time on HTTP and JSON handling.
- abrax3141 4y agoI may be misunderstanding, but it looks like the micro-services comparison here is based on very high usage. Another use for micro-services, like lambda, is exactly the opposite. If you have very low usage, you aren't paying for cycles you don't use the way you would be if you either owned the machine, or rented it from AWS or DO and left it on all the time (which you'd have to do in order to serve that randomly-arriving one hit per day!)
- pclmulqdq 4y agoIf you have microservices that truly need to be separate services and have very little usage, you probably should use things like serverless computing. It scales down to 0 really well. However, if you have a microservice with very little usage, turning that service into a library is probably a good idea.
- abrax3141 4y agoYes. I think that the former case is the situation we’re in. Lambdas are annoying (the whole AWS is annoying!) but, as you say, scales to 0 very well.
- marcosdumay 4y agoWhy open yourself to random $300k bills from Amazon when the alternative is wasting a $5/month server?
- abrax3141 4y agoI don’t understand what these numbers are referring to.
- marcosdumay 4y agoOne is a normal size of those rare, but not too rare bills people get from Amazon when their unused unoptmized application gets some surprise usage. The other is how much it costs to have an always-on server paid VPS capable of answering the once a day request you specified.
- amelius 4y agoNice until your server gets hugged by HN.
- sllabres 4y agoI would think that it can hold 1TB of RAM _per_socket_ (with 64GB DIMM), so _2TB_ total.
- rbanffy 4y agoI wouldn't recommend one, but at least two, for redundancy.
- bluehatbrit 4y agoThis is exactly what the article suggests.
- merb 4y ago> If you compare to the OVHCloud rental price for the same server, the price premium of buying your compute through AWS lambda is a factor of 25 and there is a factor of 25 that ovh is not a company where you should rent servers: https://www.google.com/search?q=ovh+fire https://www.google.com/search?q=ovh+fire
- efortis 4y agoSome comments wrongly equate bare-metal with on-premise. Bare-metal servers can be rented out, collocated, or installed on-premise. Also, when renting, the company takes care of hardware failures. Furthermore, as hard disk failures are the most common issue, you can have hot spares and opt to let damaged disks rot, instead of replacing them. For example, in ZFS, you can mirror disks 1 and 2, while having 3 and 4 as hot spares, with the following command: zpool create pool mirror $d1 $d2 spare $d3 $d4 --- The 400Gbps are now 700Gbps https://twitter.com/DanRayburn/status/1519077127575855104 https://twitter.com/DanRayburn/status/1519077127575855104 --- About the break even point: Disregarding the security risks of multi-tenant cloud instances, bare-metal is more cost-effective once your cloud bill exceeds $3,000 per year, which is the cost of renting two bare-metal servers. --- Here's how you can create a two-server infrastructure: https://blog.uidrafter.com/freebsd-jails-network-setup https://blog.uidrafter.com/freebsd-jails-network-setup
- cpursley 4y agoYou've got features to ship. Stick your stuff on Render.com and don't think about it again. Even a dummy like me can manage that.
- system2 4y agoIt blows my mind people are spending $2000+ per month for a server they can get used for $4000-5000 one time only cost. VMWare + Synology Business Backup + Synology C2 backup is our way of doing business and never failed us for over 7 years. Why do people spend so much money for cloud while they can host it themselves less than 5% of the cost? (2 year usage assumed).
- adlpz 4y agoI've tried it all except this, including renting bare metal. Nowadays I'm in the cloud but not cloudy camp. Still, I'm intrigued. Apart from the $4-5k server, what are your running costs? Licenses? Colocation? Network?
- vgeek 4y agohttps://www.he.net/colocation.html https://www.he.net/colocation.html They have been around forever and their $400 deal is good, but that is for 42U, 1G and only 15 amps. With beefier servers, you will need more current (both BW and amperage) if you intend on filling the rack.
- bebrws 4y agoSomeone call Brahm
- runeks 4y ago> The big drawback of using a single big server is availability. Your server is going to need downtime, and it is going to break. Running a primary and a backup server is usually enough, keeping them in different datacenters. What about replication? I assume the 70k postgres IOPS fall to the floor when needing to replicate the primary database to a backup server in a different region.
- zhoujianfu 4y ago10 years ago I had a site running on an 8GB of ram VM ($80/mo?) that ran a site serving over 200K daily active users on a completely dynamic site written in PHP running MySQL locally. Super fast and never went down!
- sirobg 4y agoCould you share how long you maintained this website? No problem with the db (schema updates, backups, replication, etc...)? No problem with your app updates (downtime, dependencies updates, code updates)? Did you work alone, or with a team? Did you setup a CI/CD? ... I wrote down some questions, but in fact I just think it would be interesting to understand what was your setup in a bit more detailed fashion. You probably made some concessions and it seems they worked well for you. Would be interesting to know which ones! Thanks
- titzer 4y agoLast year I did some consulting for a client using Google cloud services such as Spanner and cloud storage. Storing and indexing mostly timeseries data with a custom index for specific types of queries. It was difficult for them to define a schema to handle the write bandwidth needed for their ingestion. In particular it required a careful hashing scheme to balance load across shards of the various tables. (It seems to be a pattern with many databases to suck at append-often, read-very-often patterns, like logs). We designed some custom in-memory data structures in Java but also also some of the standard high-performance concurrent data structures. Some reader/write locks. gRPC and some pub/sub to get updates on the order of a few hundred or thousand qps. In the end, we ended up with JVM instances that had memory requirements in the 10GB range. Replicate that 3-4x for failover, and we could serve queries at higher rates and lower latency than hitting Spanner. The main thing cloud was good for was the storage of the underlying timeseries data (600GB maybe?) for fast server startup, so that they could load the index off disk in less than a minute. We designed a custom binary disk format to make that blazingly fast, and then just threw binary files into a cloud filesystem. If you need to serve < 100GB of data and most of it is static...IMHO, screw the cloud, use a big server and replicate it for fail-over. Unless you got really high write rates or have seriously stringent transactional requirements, then man, a couple servers will do it. YMMV, but holy crap, servers are huge these days.
- eastbound 4y agoWhen you say “screw the cloud”, you mean “administer an EC2 machine yourself” or really “buy your own hardware”?
- titzer 4y agoThe former, mostly. You don't necessarily have to use EC2, but that's easy to do. There are many other, smaller providers if you really want to get out from under the big 3. I have no experience managing hardware, so I personally wouldn't take that on myself.
- locutous 4y agoI find disk io to be a primary reason to go with bare metal. The vm abstractions just kill io performance. In a single server you can fill up the PCI lanes with flash and hit some ridiculous throughput numbers.
- phendrenad2 4y agoThe problem with "one big server" is, you really need good IT/ops/sysadmin people who can think in non-cloud terms. (If you catch them installing docker on it, throw them into a lava pit immediately).
- henry700 4y agoWhat's the problem with installing Docker so you can run containers of diferent distros, languages & flavors using the same one big server though?
- blue_cookeh 4y agoYeah I don't get that - if anything Docker would probably make the use case for "one big server" even easier to justify?
- SassyGrapefruit 4y ago>Use the Cloud, but don’t be too Cloudy The number of applications I have inherited that were messes falling apart at the seams because of misguided attempts to avoid "vendor lockin" with the cloud can not be understated. There is something I find ironic about people paying to use a platform but not using it because they feel like using it too much will make them feel compelled to stay there. Its basically starving yourself so you don't get too familiar with eating regularly. Kids this PSA is for you. Auto Scaling Groups are just fine as are all the other "Cloud Native" services. Most business partners will tell you a dollar of growth is worth 5x-10x the value of a dollar of savings. Building a huge tall computer will be cheaper but if it isn't 10x cheaper(And that is Total Cost of Ownership not the cost of the metal) and you are moving more slowly than you otherwise would its almost a certainty you are leaving money on the table.
- bluehatbrit 4y agoAggressively avoiding lock-in is something I've never quite understood. Unless your provider of choice is also your competitor (like Spotify with Amazon) it shouldn't really be a problem. I'm not saying I'm a die hard cloud fan in all aspects but if you're going with it you may as well use it. Typically trying to avoid vendor lockin really ends up more expensive in the long run, you start avoiding the cheaper services (lambda for background job processing) for what may never really be a problem. The one place I can see avoiding vendor lock-in as really useful is it often makes running things locally much easier. You're kind of screwed if you want to properly run something locally that uses SQS, DynamoDB, and Lambda. But that said, I think this is often better thought of as "keep my system simple" rather than "avoid vendor lock-in" as it focuses on the valuable side rather than the theoretical side.
- evilotto 4y agoMany people will respond that "one big server" is a massive single point of failure, but in doing so they miss that it is also a single point of success. If you have a distributed system, you have to test and monitor lots of different failure scenarios. With a SPOS, you only have one thing to monitor. For a lot of cases the reliability of that SPOS is plenty. Bonus: Just move it to the cloud, because AWS is definitely not its own SPOF and it never goes down taking half the internet with it.
- EddySchauHai 4y ago> But if I use Cloud Architecture, I Don’t Have to Hire Sysadmins > Yes you do. They are just now called “Cloud Ops” and are under a different manager. Also, their ability to read the arcane documentation that comes from cloud companies and keep up with the corresponding torrents of updates and deprecations makes them 5x more expensive than system administrators. I don't believe "Cloud Ops" is more complex than system administration, having studied for the CCNA so being on the Valley of Despair slope of the Dunning Kruger effect. If keeping up with cloud companies updates is that much of a challenge to warrant a 5x price over a SysAdmin then that's telling you something about their DX...
- londons_explore 4y agoHybrid! If you are at all cost sensitive, you should have some of your own infrastructure, some rented, and some cloud. You should design your stuff to be relatively easily moved and scaled between these. Build with docker and kubernetes and that's pretty easy to do. As your company grows, the infrastructure team can schedule which jobs run where, and get more computation done for less money than just running everything in AWS, and without the scaling headaches of on-site stuff.
- GordonS 4y agoClouds potentially make this an expensive option because of their silly egress bandwidth fees.
- rstephenson2 4y agoIt seems like lots of companies start in the cloud due to low commitments, and then later when they have more stability and demand and want to save costs, making bigger cloud commitments (RIs, enterprise agreements etc) are a turnkey way to save money but always leave you on the lower-efficiency cloud track. Has anyone had good experiences selectively offloading workloads from the cloud to bare metal servers nearby?
- londons_explore 4y agoDon't be scared of 'one big server' for reliability. I'd bet that if you hired a big server today in a datacenter, the hardware will have more uptime than something cloud-native with az-failover hosted on AWS. Just make sure you have a tested 30 minute restoration plan in case of permanent hardware failure. You'll probably only use it once every 50 years on average, but it will be an expensive event when it happens.
- londons_explore 4y agoOne-big-VM is another approach... A big benefit is some providers will let you resize the VM bigger as you grow. The behind-the-scenes implementation is they migrate your VM to another machine with near-zero downtime. Pretty cool tech, and takes away a big disadvantage of bare metal which is growth pains.
- mochomocha 4y ago> Why Should I Pay for Peak Load? [...] someone in that supply chain is charging you based on their peak load Oh it's even worse than that: this someone oversubscribe your hardware a little during your peak and a lot during your trough, padding their great margins at the expense of extra cache misses/perf degradation of your software that most of the time you won't notice if they do their job well. This is one of the reasons why large companies such as my employer (Netflix) are able to invest into their own compute platforms to reclaim some of these gains back, so that any oversubscription & collocation gains materialize into a lower cloud bill - instead of having your spare CPU cycles be funneled to a random co-tenant customer of your cloud provider, the latter capturing the extra value.
- twblalock 4y agoAll your eggs in one basket? A single host, really? Curmudgeonly opinions about microservices, cloud, and containers? Nostalgia for the time before 2010? All here. All you are missing is a rant about how the web was better before JavaScript. It’s sad to see this kind of engineering malpractice voted to the top of HN. It’s even sadder to see how many people agree with it.
- zokier 4y agoIf you have just two servers how are you going to load-balance and fail-over them? Generally you need at least 3 nodes for any sort of quorum?
- adam_arthur 4y agoI'm building an app with Cloudflare serverless and you can emulate everything locally with a single command and debug directly... It's pretty amazing. But the way their offerings are structured means it will be quite expensive to run at scale without a multi cloud setup. You can't globally cache the results of a worker function in CDN, so any call to a semi dynamic endpoint incurs one paid invocation, and there's no mechanism to bypass this via CDN caching because the workers live in front of the CDN, not behind it. Despite their media towards lowering cloud costs, they have explicitly designed their products to contain people in a cost structure similar to but different than via egress fees. And in fact it's quite easily bypassed by using a non Cloudflare CDN in front of Cloudflare serverless. Anyway, I reached a similar conclusion that for my app a single large server instance works best. And actually I can fit my whole dataset in RAM, so disk/JSON storage and load on startup is even simpler than trying to use multiple systems and databases. Further, can run this on a laptop for effectively free, and cache everything via CDN, rather than pay ~$100/month for a cloud instance. When you're small, development time is going to be your biggest constraint, and I highly advocate all new projects start with a monolithic approach, though with a structure that's conducive to decoupling later.
- anonymousDan 4y agoAs someone who has only dabbled with serverless (Azure functions), the difficulty in setting up a local dev environment was something I found really off-putting. There is no way I am hooking up my credit card to test something that is still in development. It just seems crazy to me. Glad to hear Cloudflare workers provides a better experience. Does it provide any support for mocking commonly used services?
- adam_arthur 4y agoYes, you can run your entire serverless infrastructure locally with a single command and close to 0 config. It's far superior to other cloud offerings in that respect. You can even run it live in dev mode and remote debug the code. Check out miniflare/Wrangler v2 Just wish they would have ability for persistent objects. Everything is still request driven, yet I want to schedule things on subminute schedules. You can do it today, but it requires hacks
- alexpotato 4y agoMy favorite summary of why not to use microservices is from Grug: "grug wonder why big brain take hardest problem, factoring system correctly, and introduce network call too seem very confusing to grug" https://grugbrain.dev/#grug-on-microservices https://grugbrain.dev/#grug-on-microservices
- sagarm 4y agoIMO microservices primarily solve organizational problems, not technical problems. They allow a team to release independently of other teams that have or want to make different risk/velocity tradeoffs. Also smaller units being released means fewer changes and likely fewer failed releases.
- klysm 4y ago> Also smaller units being released means fewer changes and likely fewer failed releases. The interfaces are the hard part, so you may have fewer internal failures but problems between services seem more likely.
- sagarm 4y agoYeah, not to mention all the extra operational issues and failure modes that come with RPCs vs function calls. Integration testing and release coordination both become more difficult as well. But hundreds of people contributing to a single binary is probably not realistic; at some point you'll need to factor it into pieces that can be can have somewhat independent operations.
- klysm 4y agoI’m not sure one binary is the scaling problem you’d hit first though - makes me wonder just how many people can work on something that gets compiled together into a monolith
- robertlagrant 4y agoThis is why I like Cloudflare's worker model. It feels like the usefulness of cloud deployments, but with a pretty restrained pricing model.
- siliconc0w 4y agoOne thing to keep in mind is separation. The prod environment should be completely separated from the dev ones (plural, it should be cheap/fast to spin up dev environments). Access to production data should be limited to those that need it (ideally for just the time they need it). Teams should be able to deploy their app separately and not have to share dependencies (i.e operating system libraries) and it should be possible to test OS upgrades (containers do not make you immune from this). It's kinda possible to sort of do this with 'one big server' but then you're running your own virtualized infrastructure which has it's own costs/pains. Definitely also don't recommend one big database, as that becomes a hairball quickly - it's possible to have several logical databases for one physical 'database 'server' though.
- galkk 4y agoOne of first experiences in my professional career was situation when "one big server" that was serving the system that was making money actually failed on Friday, HP's warranty was like next or 2 business days to get a replacement. The entire situation ended up having conference call with multiple department directors who were deciding which server from other systems to cannibalize (even if it is underpowered) to get the system going. Since that time I'm quite skeptical about "one", and to me this is one of big benefits of cloud provides, as, most likely, there is another instance and stockouts are more rare.
- jmull 4y agoThe article is really talking about one big server plus a backup vs. cloud providers.
- lrvick 4y agoA consequence of one-big-server is decreased security. You become discouraged from applying patches because you must reboot. Also if one part of the system is compromised, every service is now compromised. Microservices on distinct systems offer damage control.
- yomkippur 4y agoWhat holds me back from doing this is how will I reduce latency from the calls coming from other side of the world when OVHcloud seemingly does not have datacenters all over the world? There is an noticeable lag when it comes to multiplayer games or even web applications.
- krn 4y agoOVH has datacenters in four continents: https://us.ovhcloud.com/about/company/data-centers https://us.ovhcloud.com/about/company/data-centers
- YetAnotherNick 4y agoThis post raises small issues like reliability, but missed lot of much bigger issues like testing, upgrades, reproducibility, backups and even deployments. Also, the author is comparing on demand pricing, which to me doesn't make sense if you are paying for the server with reserved pricing. Still I agree there would be a difference of 2-3x(unless your price is dominated by AWS egress fees), but most server with fixed workload, even for very popular but simple sites, it could be done in $1k/month in cloud, less than 10% of one developer salary. For non fixed workload like ML training, you would anyways need some cloudy setup.
- cxromos 4y agois this clickbait? although i do like the alternate version: use servers, but don’t be too serverly.
- arwhatever 4y agoRecent team I was on used one big server. Wound up spawning off a separate thread from our would-be stateless web api to run recurring bulk processing jobs. Then coupled our web api to the global singleton-esque bulk processing jobs thread in a stateful manner. The wrapped actors up on actors on top of everything to try to wring as much performance as possible out of the big server. Then decided they wanted to have a failover/backup server but it was too difficult due to the coupling to the global singleton-esque bulk processing job. [I resigned at this point.] So yeah color me skeptical. I know every project's needs are different, but I'm a huge fan of dumping my code into some cloud host that auto-scaled horizontally, and then getting back to writing more code that provides some freeeking busines value.
- deleted 4y ago[deleted]
- fleddr 4y agoOur industry summarized: Hardware engineers are pushing the absolute physical limits of getting state (memory/storage) as close as possible to compute. A monumental accomplishment as impactful as the invention of agriculture and the industrial revolution. Software engineers: let's completely undo all that engineering by moving everything apart as far as possible. Hmmm, still too fast. Let's next add virtualization and software stacks with shitty abstractions. Fast and powerful browser? Let's completely ignore 20 years of performance engineering and reinvent...rendering. Hmm, sucks a bit. Let's add back server rendering. Wait, now we have to render twice. Ah well, let's just call it a "best practice". The mouse that I'm using right now (an expensive one) has a 2GB desktop Electron app that seems to want to update itself twice a week. The state of us, the absolute garbage that we put out, and the creative ways in which we try to justify it. It's like a mind virus. I want my downvotes now.
- Foomf 4y agoYou've more or less described Wirth's Law: https://en.wikipedia.org/wiki/Wirth%27s_law https://en.wikipedia.org/wiki/Wirth%27s_law
- fleddr 4y agoI had no idea, thanks. Consider this a broken clock being sometimes right.
- smarks 4y agoHeh, there's a mention here to Andy and Bill's Law, "What Andy giveth, Bill taketh away," which is a reference to Andy Grove (Intel) and Bill Gates (Microsoft). Since I have a long history with Sun Microsystems, upon seeing "Andy and Bill's Law" I immediately thought this was a reference to Andy Bechtolsheim (Sun hardware guy) and Bill Joy (Sun software guy). Sun had its own history of software bloat, with the latest software releases not fitting into contemporary hardware.
- GuB-42 4y agoActually, for those who push for these cloudy solutions, they do that in part to make data close to you. I am talking mostly about CDNs, I don't thing YouTube and Netflix would have been possible without them. Google is a US company, but you don't want people in Australia to connect to the other side of the globe every time they need to access Google services, it would be an awful waste of intercontinental bandwidth. Instead, Google has data centers in Australia to serve people in Australia, and they only hit US servers when absolutely needed. And that's when you need to abstract things out. If something becomes relevant in Australia, move it in there, and move it out when it no longer matters. When something big happens, copy it everywhere, and replace the copies by something else as interest wanes. Big companies need to split everything, they can't centralize because the world isn't centralized. The problem is when small businesses try to do the same because "if Google is so successful doing that, it must be right". Scale matters.
- rcarmo 4y agoI once fired up an Azure instance with 4TB of RAM and hundreds of cores for a performance benchmark. htop felt incredibly roomy, and I couldn’t help thin how my three previous projects would fit in with room to spare (albeit lacking redundancy, of course).
- hot_gril 4y agoThe way I code now after 10 years: Use one big file. No executable I'm capable of writing on my own is complex enough to need 50 files spread across a 3-layers-deep directory tree. Doesn't matter if it's a backend, a UI, or what. There's no way your React or whatever tutorial example code needs that either. And you don't gain any meaningful organization splitting into files when there are already namespaces, classes, structs, comments, etc. I don't want to waste time reorganizing it, dealing with imports, or jumping around different files while I code. Oh, there's some custom lib I want to share between executables, like a Postgres client? Fine, it gets its own new file. Maybe I end up with 4 files in the end.
- EddySchauHai 4y agoI like simplicity but this sounds pretty awful if you work in a team - file structure can help with the domain design too.
- harperlee 4y agoI read it as satire.
- hot_gril 4y agoThis is sorta how our team does things, and so far it hasn't presented issues. Each service has the vast majority of its real logic in a single file. Worst case, one day this stops working, and someone takes 10 minutes to split things into a separate file. On the other side, I've seen people spend hours preemptively deciding on a file structure. It often stops making sense a month later, and every code review has a back and forth argument about what to name a new file.
- journey_16162 4y agoReminds me of a company I used to work at which took a similar approach. We used one file per person policy, each developer had their own file that contained functionality developed by them, named like firstName_lastName.ext - everyone owned their file so we didn't have to worry about merge conflicts.
- mrinterweb 4y agoOne advantage I didn't see in the article was the performance costs of network latency. If you're running everything on one server, every DB interaction, microservice interaction, etc. would not necessarily need to go over the network. I think it is safe to say, IO is generally the biggest performance bottleneck of most web applications. Minimizing/negating that should not be underestimated.
- Melatonic 4y agoI have a feeling building your own "private cloud" is gonna be the next big thing :-D
- rbanffy 4y agoNot to be nasty, but we used to call them mainframes. A mainframe is still a perfectly good solution if you need five nines of uptime, with transparent failover of pretty much every part of the machine, the absolute fastest single-thread performance and the most transaction throughput per million dollars in the market. I would not advise anyone to run them as a single machine, however, but to have it partitioned into smaller slices (they call them LPARs) and host lots of VMs in there (you can oversubscribe like crazy on those machines). Managing a single box is cheaper, even if you have a thousand little goldfish servers in there (remember: cattle, not pets) and this is something the article only touches lightly.
- neverartful 4y ago"absolute fastest single-thread performance" Can you provide citation?
- catern 4y agoOne can embrace this philosophy for one's personal computing too http://catern.com/computers.html http://catern.com/computers.html although it's not for everyone
- quickthrower2 4y agoI am using Firebase on a project and I regret it. There are some Firebase specific annoyances to put up with, like the local emulator is not as nice and "isomorphic" as say running postgresql locally. But the main problem (and I think this is shared by what I call loosely "distributed databases") is you have to think really hard about how the data is structured. You can't structure it as nicely from a logical perspective compared to a relational DB. Because you can't join without pulling data from all over the place. Because the data isn't in one place. It is hard to do joins both in terms of performance and in terms of developer ergonomics. I really miss SELECT A.X, B.Y FROM A JOIN B ON A.ID = B.AID; when using Firebase. You have to make data storage decisions early on, and it is hard to change you mind later. It is hard to migrate (and may be expensive if you have a lot of existing data). I picked Firebase for the wrong reason (I thought it would make MVP quicker to set up). But the conveniences it provides are outweighed by having to structure your data for distribution across servers. Instead next time I would go relational, then when I hit a problem do that bit distributed. Most tables have 1000s of records. Maybe millions. The table with billions might need to go out to something distributed. Market gap??: Let me rent real servers, but expose it in a "serverless" "cloud-like" way, so I don't have to upgrade the OS and all that kind of stuff.
- hot_gril 4y agoYes, it's very hard to beat Postgres IMO. You can use Firebase without using its database, and you can certainly run a service with a Postgres database without having to rent out physical servers.
- SicSemperUranus 4y ago> Let me rent real servers, but expose it in a "serverless" "cloud-like" way, so I don't have to upgrade the OS and all that kind of stuff. I think you're describing platform-as-a-service? It does exist, but it didn't eat cloud's lunch, rather the opposite I expect. It's hard to sell a different service when most technical people in medium-big companies are at the mercy of non-technical people who just want things to be as normal as possible. I recently encountered this problem where even using Kubernetes wasn't enough, we had to use one of the big three, even though even sustained outages wouldn't be very harmful to our business model. What can I say, boss want cloud.
- ClumsyPilot 4y agooh nice, we are about to rediscover the mainframe. I agree in spirit with much of the stuff said here.
- wheresmycraisin 4y agoOne thing that has helped me grow over the last few years building startups is: microservices software architecture and microservice deployment are two different things. You can logically break down your software into DDD bounded contexts and have each own its data, but that doesn't mean you need to do Kubernetes with Kafka and dozens of tiny database instances, communicating via json/grpc. You can have each "service" live in its own thread/process, have it's own database (in the "CREATE DATABASE" sense, not the instance sense), communicate via a simple in-memory message queue, and communicate through "interfaces" native to your programming language. Of course it has its disadvantages (need to commit to a single softare stack, still might need a distributed message queue if you want load balancing, etc) but for the "boring business applications" I've been implementing (where DDD/logical microservices makes sense) it has been very useful.
- samsquire 4y agoI am really interested in scalability problems. I recommend the whitepaper Scalability! But at what cost? My experience with Microservices is that they are very slow due to all the IO. We kind of want the development and developer scalability of decoupled services in addition to the computational and storage scalability in a disaggregated architecture.
- wiremine 4y agoI see these debates and wish there was an approach that scaled better. A single server (and a backup) really _is_ great. Until it's not, for whatever reason. We need more frameworks that scale from a single box to many boxes, without starting over from scratch. There are a lot of solid approaches: Erlang/Elxir and the actor model comes to mind. But that approach is not perfect, and it's far from common place.
- bluehatbrit 4y ago> We need more frameworks that scale from a single box to many boxes, without starting over from scratch. I'm not sure I really understand what you're saying here. I suppose most applications are some kind of CRUD app these days, not all sure, but an awful lot. If we take that as an example, how is it difficult to go from one box to multiple? It's not something you get for free, you need to put in time to provision any new infra (be it baremetal or some kind of cloud instance) but the act of scaling out is pretty straight forward. Perhaps you're talking about stateful applications?
- chx 4y agoYeah, I've been saying this for a long long time now, an early blog post of mine http://drupal4hu.com/node/305.html http://drupal4hu.com/node/305.html and this madness just got worse because of Kubernetes et al. Kubernetes is a Google solution. Are you sure Google-sized solutions are right for your organization? Also, an equally pseudo controversial viewpoint: it's almost always cheaper to be down than engineering a HA architecture. Take a realistic look at downtime causes outside of your control -- for example, your DDoS shield provider going down etc. etc. and then consider how much downtime a hardware failure adds and now think. Maybe a manual failover master-slave is enough or perhaps even that's overkill? How much money does the business lose by being down versus how much it costs to protect from it? And can you really protect from it? Are you going to have regular drills to practice the failover -- and absurdly, will the inevitable downtime from failing a few of those be larger than a single server downtime? I rarely see posts about weighing these while the general advice of avoiding single points of failure -- which is very hard -- is abundant.
- djmetzle 4y ago>Generally, the burstier your workload is, the more cloudy your architecture should be. Well, crap dude, that's the web!
- andsoitis 4y ago"In total, this server has 128 cores with 256 simultaneous threads. With all of the cores working together, this server is capable of 4 TFLOPs of peak double precision computing performance. This server would sit at the top of the top500 supercomputer list in early 2000. It would take until 2007 for this server to leave the top500 list. Each CPU core is substantially more powerful than a single core from 10 years ago, and boasts a much wider computation pipeline."
- langsoul-com 4y agoSo much of the latest tech news and solutions are from huge companies. Let's be real here, how many of us get 10-100 million users/requests etc? My blog, langsoul.com, has 2, myself and a bot . Simple dumb solutions seems best for 99% of cases, then, if you ever hit that 1%, well, you'll have shit tons of money to deal with it then!
- disqard 4y agoI didn't see the COST paper linked anywhere in this thread [0]. Excerpt from abstract: We offer a new metric for big data platforms, COST, or the Configuration that Outperforms a Single Thread. The COST of a given platform for a given problem is the hardware configuration required before the platform outperforms a competent single-threaded implementation. [0] https://www.usenix.org/conference/hotos15/workshop-program/presentation/mcsherry https://www.usenix.org/conference/hotos15/workshop-program/p...
- sanderjd 4y agoI didn't want to write a top-level comment and I'm sure few people will see this, but I scrolled down very far in this thread and didn't see this point made anywhere: The article focuses almost entirely on technical questions, but the technical considerations are secondary; the reason so many organizations prefer cloud services, VMs, and containers is to manage the challenges of scaling organizationally, not technically. Giving every team the tools necessary to spin up small or experimental services greases the skids of a large or quickly growing organization. It's possible to set this up on rented servers, but it's an up front cost in time. The article makes perfect sense for a mature public facing service with a lot of predictable usage, but the sweet spot for cloud services is sprawling organizations with lots of different teams doing lots of different mostly-internally facing things.
- gwbas1c 4y agoI agree with almost everything you said; except that the article offers extremely valuable advice for small startups going the cloud / rented VM route: Yearly payments, or approaching a salesperson, can lead to much lower costs. (I should point out that yesterday, in Azure, I added a VM in a matter of seconds and it took all of 15 minutes to boot up and start running our code. My employer is far too small to have dedicated ops; the cost of cloud VMs is much cheaper than hiring another ops / devops / whatever.)
- sanderjd 4y agoYep. To be clear, I thought it was a great article with lots of great advice, just too focused on the technical aspects of cloud benefits, whereas I think the real value is organizational.
- krnlpnc 4y agoUse two…
- xcambar 4y agoOne major selling point against One Big Server: VCs and enterprise customers prefer (sometimes demand) Cloud. For better or for worse. (Worse, IMO)
- fsdghrth3 4y agoReading these comments make me sad. It's like everyone has forgotten the cookie cutter server architecture pattern. https://dzone.com/articles/monoliths-cookie-cutter-or https://dzone.com/articles/monoliths-cookie-cutter-or I really don't understand microservices for most businesses. They're great if you put the effort into it but most business don't have the scale required. Big databases and big servers serve most businesses just fine. And past that NFS and other distributed filesystem approaches get you to the next phase by horizontally scaling your app servers without needing to decompose your business logic into microservices. The best approach I've ever seen is a monorepo codebase with non-micro services built into it all running the same way across every app server with a big loadbalancer in front of it all.
- gbro3n 4y agoBeing too cloudy without being too cloudy, as per the article, I've gone with a full stack in containers under Docker Compose one one EC2 server, including the database. Services are still logically separated and have a robust CI/CD set up but the cost is a 3rd of what an ECS set up with load balancers and RDS for the database would have been. It's also simpler. Have scripted the server set up, with regular back ups / snapshots but admit I would like db replication in there.
- KronisLV 4y agoCurrently using two old computers as servers in my homelab: 200 GE Athlons with 35 W TDP, ~20 GB of value RAM (can't afford ECC), a few 1TB HDDs. As CI servers and test nodes for running containers, they're pretty great, as well as nodes for pulling backups from any remote servers (apart from the ECC aspect), or even something to double as a NAS (on separate drives). I actually did some quick maths and it would appear that a similar setup on AWS would cost over 600$ per month, Azure, GCP and others also being similarly expensive, which I just couldn't afford. Currently running a few smaller VPSes on Time4VPS as well (though Hetzner is also great), for the stuff that needs better availability and better networking. Would I want everything on a single server? Probably not, because that would mean needing something a bit better than a homelab setup behind a residential Internet connection (even if parts of it can be exposed to the Internet through a cheap VPS as a proxy, a la Cloudflare). Either way, I appreciate the sentiment!
- _448 4y agoHow about using combination of CouchDB + Elixir for both horizontal and vertical scaling of backend? What will be the pros and cons of this combo for backend stack?
- koffiezet 4y agoLet's be clear here, everything you can do in a "cloudy" environment, you could do on big servers yourself - but at what engineering and human resource cost? Because that's something many - if not most - hardware and 'on-prem' infra focussed people seem to miss. While cloud might seem expensive, most of the times, humans will be even more expensive (unless you're in very niche markets like HPC)
- dividedbyzero 4y agoYou could also have those big servers in the cloud (I think this is what many are doing; I certainly have). That gives you a lot of the cloud services e.g. for monitoring, but you get to not have to scale horizontally or rebuild for serverless just yet. Works great for Kubernetes workloads, too – have a single super beefy node (i.e. single-node node pool) and target just your resource-heavy workload onto that node. As far as costs are concerned, however, I've found that for medium+ sized orgs, cloud doesn't actually save money in the HR department, the HR spend just shifts to devops people, who tend to be expensive and you can't really leave those roles empty since then you'll likely get an ungovernable mess of unsecured resources that waste a huge ton of money and may expose you to GDPR fines and all sorts of nasty breaches. If done right, you get a ton of execution speed. Engineers have a lot of flexibility in terms of the services they use (which they'd otherwise have to buy through processes that tend to be long and tedious), scale as needed when needed, shift work to the cloud provider, while the devops/governance/security people have some pretty neat tools to make sure all that's done in a safe and compliant manner. That tends to be worth it many times over for a lot of orgs, if done effectively with that aim, though it may not do much for companies with relatively stagnant or very simple products. If you want to reduce HR costs, cloud is probably not going to help much.
- calrain 4y agoDesign systems such that eventual completion/consistency is a core tenant. When it gets too slow, improve only the parts that are currently the slowest.
- roeles 4y ago> Populated with specialized high-capacity DIMMs (which are generally slower than the smaller DIMMs), this server supports up to 8 TB of memory total. At work we're building a measurement system for wind tunnel experiments, which should be able to sustain 500 MB/sec for minutes on end, preferably while simultaneously reading/writing from/to disk for data format conversion. We bought a server with 1TB of RAM, but I wonder how much slower these high-capacity DIMMs are. Can anyone point me to information regarding latency and throughput? More RAM for disk caching might be something to look at.
- AzzieElbab 4y agoOne big sever, one big program, and one big 10x developer. Deploy websphere when you need isolation. The industry truly is going in spiral. Although, I must admit cloud providers really overplayed their hand when it comes to performance/buck and complexity.