9 ms·

A terrible way to jump into colocating your own stuff

238 points by ingve 2y ago

subjectsigma 2y ago
Why is the tone of the article so negative and grouchy? On one hand the author doesn’t owe anybody fake kindness and it’s their blog. On the other hand - if you don’t like the topic or giving advice, then why write a grumpy article about it?
N8works 2y ago
This is very 2004. I like it. RIP Alchemy.
yjftsjthsd-h 2y ago
> 1. Install Linux on the box. Turn everything off but sshd. Turn off password access to sshd. Also, test that it's properly disabled with something like `ssh -v yourserver : 2>&1 | grep continue`, because there are a surprising number of ways for that to go wrong (did you know that sshd lets you Include multiple config files together in a way that can override the main one? I know that now.)
- Maledictus 2y ago
  and `sshd -T | grep -i password`
- a-french-anon 2y ago
  This. OVH's VPS had two .confs reenabling passwords. Now I know too.
  - a-french-anon 2y ago
    I should specify that this was with AlmaLinux 9. The offending files were a 50-redhat.conf and a 50-cloud-init.conf.
  - omgtehlion 2y ago
    default ubuntu 22.04 has at least one, too
- omgtehlion 2y ago
  I always remove all 'includes' in sshd config, just to be sure I will not get any surprises later...
  - zokier 2y ago
    I think doing the opposite is better usually, never making changes to the main /etc/ssh/sshd_config and always add your own customizations to /etc/ssh/sshd_config.d/; that way you have clear separation of your own and distro configs, which makes life easier especially when upgrading sshd. Although I'll readily admit that I don't myself follow that advise all the time.
    - xyse53 2y ago
      This. And give your drop-in a prefix like 00-*.conf. Fwiw I don't think SSH adds the include line upstream. Most distros add it now.
      - jgalt212 2y ago
        Yes. I had a hard time figuring out if a low numbered prefix got final say, or the high numbered one.
- hedora 2y ago
  In situations where I have more than one box, one (usually a little embedded SoC) gets openbsd on it, and is used as a bastion host, for exactly the reasons outlined here. The problems I hit with using Linux for this were different ten years ago, but, based on this thread, things got worse on that side of the fence.
- ycombinatrix 2y ago
  iirc the double config file nonsense is another genius move brought to you by debian maintainers. it seriously bothered me that an update automatically re-enabled password authentication. i ended up switching to a different OS.
  - taskforcegemini 2y ago
    on updates you are asked if your conf file should be kept or overwritten, with further options like diff-comparison
cuu508 2y ago
What's the importance of having a switch/hub? Is it because you may want to add more servers later, but the colo host only provides one port in their router?
- re-thc 2y ago
  Also if you want to manage the IP ranges provided amongst >1 server.
- pests 2y ago
  Basically. Need to pay for a bigger pipe. Same applies to power as well, gotta pay to get more capacity.
- kelnos 2y ago
  So you can plug your laptop into it at the colo when you set it up so you can verify that the server is working right and can get to the internet while you are still there. And if it isn't working right, it'll be a lot easier to keep everything plugged into the switch, rather than swapping between your laptop and the colo's network every time you try something to fix it. But yes, if you think you might want to add other machines later, preemptively putting a switch in there will make it tons easier.
moandcompany 2y ago
Documenting and testing the hard reset/reboot procedure, as well as any expectations, for your colocated gear sounds like a good thing to add to this list.
- bigfatkitten 2y ago
  Test to make sure it'll actually come back after a hard reset. Don't do what I did and put a Cisco router on a remote mountaintop radio site with the config register still set to 0x2142, and then go home.
  - pferde 2y ago
    Hey, that's just an excuse to go for another hike! :)
    - bigfatkitten 2y ago
      3 hour drive in warm weather, or an expensive helicopter ride in winter!
  - fanf2 2y ago
    Also, check what happens when mains power disappears and returns.
  - moandcompany 2y ago
    Systems Administration the Hard Way... it's an uphill hike, both ways, and in the snow :)
krab 2y ago
A bit less terrible way in my opinion: Find a dedicated server provider and rent the hardware. These companies rent some part of the datacenter (or sometimes build their own). Bonus points if they offer KVM - as in remote console, not the Linux hypervisor. Also ask if they do hardware monitoring and proactively replace the failed parts. All of this is still way cheaper than cloud. Usually with unmetered networking. Way less hassle. They'll even take your existing stuff and put it into the same rack with the rented hardware. The difference from cloud, apart from the price, is mainly that they have a sales rep instead of an API. And getting a server may take a from few hours to a few days. But in the end you get the same SSH login details you would get from a cloud provider. Or, if you really want to just collocate your boxes, the providers offer "remote hands" service, so you can have geo-redundancy or just choose a better deal instead of one that's physically close to your place.
- rsanheim 2y ago
  One hurdle that many companies who have only known cloud hosting will face here is significant: how do you find a reliable, trustworthy datacenter? One who actually monitors the hardware and also has a real human if your networking access gets screwed or if you need a critical component swapped at 2 am on a Saturday. I used to have a short list of trustworthy companies like this I'd recommend to clients ~20 years ago when doing consulting. I think 3/4 of them have been gobbled up by private equity chop shops or are just gone. Nowadays noone gets fired for going with AWS, or resold AWS with a 100% markup from a 'private enterprise cloud' provider.
  - krab 2y ago
    You're right you need to find a company you can trust. And for a lot of startups it really makes sense to use AWS. But if you do something resource or bandwidth intensive (and I'm not even talking about Llama now), the costs add up quickly. In our case, switching to AWS would increase our costs by an equivalent of 4 - 8 devs salaries. After AWS discounts. That's a hard sell in a 15-person team even though half of our infra costs already are with AWS (S3).
    - 2y ago
ggm 2y ago
Remote power management can be a godsend. If you can get an ipmi console in, you want it.
- michaelt 2y ago
  It is indeed very helpful when it's needed - but most IPMI is horrifically insecure, so usually it's not connected directly to the public internet. Instead it's on a separate network with special VPN access. As this document focuses on keeping things simple, they don't have the isolated network / VPN needed to use IPMI.
- qhwudbebd 2y ago
  Ipmi is a bit of a double-edged sword. Network-connected access to a serial console (including uefi/bios console redirection) and the reset button can be a total lifesaver, I agree. I wouldn't want to be without a serial console and remote reset button either. But ipmi cards are little network-attached linux boxes which run prehistoric kernels and userspace, exposing as many services as the half-wits who put together the firmware image can shovel in, and are rarely if ever patched by the vendor unless there's some really public scandal. The standard thing to do is to isolate them on some kind of private management network in an attempt to shield the wider internet from the full majesty of the firmware engineers' dazzling skills, but that might be harder to do in the simple 'beginner' scenario Rachel describes. One good simple version when you get up to two servers instead of just one is to cross-connect so each machine has ipmi access to the other, but neither ipmi service is exposed to the wider world.
  - omgtehlion 2y ago
    Even if your server does not have proper IPMI, most colo providers have an assortment of ip-kvms, you can ask for those and access your server from any modern browser.
    - qhwudbebd 2y ago
      I'm pretty suspicious of trusting the firmware on things like that either, to be honest. (Personally, I tend to use serial cables and a little custom board to turn serial breaks into resets, but I know my NIH instinct is probably a little on the high side. It would be unreasonable for me to suggest someone new to colo start making random extra boards to stick inside their machines!)
mjevans 2y ago
They're absolutely correct: " 1. Install Linux on the box. Turn everything off but sshd. Turn off password access to sshd. If you just locked yourself out of sshd because you didn't install ssh keys first, STOP HERE. You are not ready for this. " If you blindly followed the directions and got locked out, you can do exactly the same thing with other directions. You were not ready.
- kijin 2y ago
  The great thing about having unfettered physical access to hardware is that you can easily recover from mistakes like this. No need to rebuild that EC2 instance. No need to beg a hosting company for IPMI access. You can just pull the plug and try again as if were your own PC.
  - jiggunjer 2y ago
    Isn't it the same with something like idrac?
  - bigiain 2y ago
    On the other hand, it's nice to be able to take a no-reboot AMI of the machine you're futzing with, risk breaking it, then just re provision from that AMI if you fucked up. And if you're chasing three or four nines of uptime you do the same but provision a new instance and futz with that one while your working one keeps on keeping on. Even big instances are cheap enough by the hour that you can afford to run and extra one while testing potentially breaking changes.
- nasretdinov 2y ago
  At least it doesn't say to set PermitRootLogon and remove the root password :)
  - Foobar8568 2y ago
    AKA, "We are compatible with passwordless authentication mechanism." Or SSO, depending of the use case.... Le sight.
behringer 2y ago
Forget the switch and pi and get a ubiquity router. Much more powerful and simple to setup. Does require some knowhow. Also you could see if your local hackers space has a server rack
seszett 2y ago
The most difficult step I find is just barely mentioned, finding colocation space at reasonable price is difficult these days.
- c0l0 2y ago
  I faced the same problem, until I found a non-profit community-operated housing provider (a registered computer club/society that has been operating since at least the early 2000 who is also a LIR (https://en.wikipedia.org/wiki/Regional_Internet_registry#Local_Internet_registry https://en.wikipedia.org/wiki/Regional_Internet_registry#Loc...), and they give shelter, bandwidth and IP address space to anyone who participates in covering their operational costs. I gladly do, and it's the best hosting experience I've had so far, and I used to have rented dedicated iron from Server4You, Hetzner, Webtropia and the like from 2005 on. Maybe there's a similar hidden gem in the area you live in, and you just do not know about it? :) Mine flew under my radar for nearly 20 years, and even though I knew they existed, I was not aware they'd colo other peoples' boxen at very fair rates.
  - hiatus 2y ago
    How did you find your local non-profit? Is there a resource you can share?
    - c0l0 2y ago
      There is, unfortunately, no directory of services like this that I am aware of. If you have a local hackerspace, asking specific questions there would probably give you answers quickly, I imagine :)
  - oarsinsync 2y ago
    Is this provider for local people only, or can people from afar also participate in contributing towards costs in exchange for services? If so, can you share their details? I need someone to house some RIPE objects for me.
- acatton 2y ago
emptiestplace 2y ago
What audience was this written for?
- schmookeeg 2y ago
  I was gonna ask the same thing, but then she called me a usual broken people and I decided "not me" was a good enough answer :) This was a tortured way to setup a colo. I bring rackable gear, no switch, and use the colo's "setup station" to do final config, then I rack the sucker. They can always VLAN me and my laptop together somewhere for a final check before I leave. I'd be worried if they couldn't.
kkfx 2y ago
I suggest a very different reasoning: we know WFH works, we also agree that in large part of the world we have enough sunlight for p.v., battery storage could be cheap [1] and FTTH connections start be be very good for modern time. Who have already understood where I want to go? Well, we could run decentralized systems at home for SMEs/Startup common needs from modern homes of remote workers, just renting them a room with proper local equipment the modern remote worker probably already have (VMC, A/C, "the big UPS" from domestic p.v.) with not much more expenses than using someone else computer also named "the cloud". Then if/when needs mount some sheds with p.v. + local storage and good connections can pops up. Who need a CDN in such setup? Then/if a real small datacenter could born. Are you horrified? Well, now try to see the recent history, when Amazon choose not to buy expensive and reliable SUN servers choosing cheap and crappy PCs or Google choose to save money with rackmounted velcro-assembled PCs with a built-in UPS, i.e. a naked velcro strapped one. See the history of buying cheap low quality ram because we have ECC, lo quality storage because we have checksums anyway and so on. Then think about some real recent datacenter fires and what they told about "how brilliant and safe" was their design. Long story short: as we ditch big iron in the past, we ditch mainframes, it's about time to think ditching dataceter model as well for a spread set of small unreliable but many machine rooms at homelabs scale. This, like FLOSS gives us reliability, cheapness and ownership as well. This ALSO decouple hw and sw giants, witch is a very good things in terms of free markets. It will also force he OEMs to be flexible in their design and ditching slowly from modern crappy fats-tech also some giants deprecate [2] as unsustainable, for more balanced tech because well, such homelabs last much longer than typical ready-made PCs and craptops or NAS from the mass distribution market. [1] buying from China directly instead of feeding local importers who skyrocket the price instead of lowering it when Chinese supplier lower them [2] https://unctad.org/system/files/official-document/der2024_en.pdf https://unctad.org/system/files/official-document/der2024_en... of course suggesting the wrong answer
omgtehlion 2y ago
It is also useful to configure mutiple IPs on the same interface, one for Colo, and one for your lab. You can do this on all OSes, even on windows. And if you failed to do this and all advices in the article: then kindly ask colo provider to attach ip-KVM (if your "server" is not actual server with ipmi/bmc), 99.9% of places have these. And most of them will provide it free of charge for limited amount of time.
neomantra 2y ago
Don't forget hearing protection! All the fans in there make it crazy loud. Facilities often supply earplugs, but consider using it with over-the-ear protection.
elric 2y ago
Back in the late 90s/early 00s when I was a precocious teenager, I ran a somewhat popular website. At some point it made sense to just buy a 1U rack moubtable server and having it colocated (commercial webhosting was expensive then). I couldn't find anyone to give me a ride to the datacenter, so I took a bus. By the time I got there my arms were numb from carrying the bloody thing. There was a single security guard, I signed in and he gave me directions and a big keychain. The keys opened most of the rooms and most of the cages around the racks. To this day I remain mystified at the level of trust (or nonchalance) that security guard had in a spotty teenager.
- qhwudbebd 2y ago
  Having run hosting companies from the mid 90s as well, from memory this kind of thing was pretty normal, even in allegedly secure places like London Telehouse. Quite a few of us in that era were juggling it with being students, so it wouldn't surprise me if the security staff were used to it and expected you to look young enough to be their kid!
  - anal_reactor 2y ago
    Modern society is absolutely paranoid. Zero-trust policy, everyone is treated like a potential terrorist.
    - thelastparadise 2y ago
      You're treated lile a terrorist when you are a law abiding citizen. Actual criminal? Society will bend over to accommodate you.
    - phil21 2y ago
      It has nothing really to do with that. To this day I can get into pretty much any rack or room I feel like at datacenters everyone here has heard of. It just takes experience these days and a bit of charm. Plus having a million keys and staff rack combination codes doesn't hurt. These were freely given and simply added to my collection over time, nothing stolen or social engineered. I've never done anything nefarious with these abilities, and no one I know has either. It's simply a matter of practicality when you staff a 150,000 square foot facility with 2 security guards who have no idea what they are doing. If I (and many others) had wanted to, we could have caused multi-week/month outages you'd be reading about on the news with 5 minutes of effort. This is basically the status quo for any sensitive industry. The world turns because 99.9999% of people want to give you a hug vs. hit you. Society falls when that ratio goes much lower.
andyjohnson0 2y ago
Interesting article. > Plug everything in and turn it on. Most server racks will have C13 or C19 outlets for power distribution. For consumer-type devices designed to plug into a wall socket, suitable cables or an adapter strip would probably be required.
methou 2y ago
IPMI/BMC is a must if plan to keep the server running longer than a release cycle of your Linux distribution. Also remember ask your provider to assign you a private vlan and vpn to it for your own access only.
- rpcope1 2y ago
  A working BMC is great, but I sure wouldn't want it facing the open web. The colo would either need some sort of private network, or you'd need to add a router and firewall of some sort to lock down access to the BMC.
efortis 2y ago
My tip is adding hot spare disks so you don't have to replace broken ones. For example, in ZFS you can mirror disks 1 and 2, while having 3 and 4 as hot spares with: zpool create pool mirror $d1 $d2 spare $d3 $d4
- thelastparadise 2y ago
  How about make it so you can smash the whole rack with a sledgehammer or datacenter burn down and system hums along like nothing happened.
  - iphoneisbetter 2y ago
    No; because that costs at least an order of magnitude more money than a simple colo setup.
ForHackernews 2y ago
Why would you do this instead of having the hardware at your house? For better/faster connection?
- gsck 2y ago
  Reliability, and depending on your work load the ability to use it commercially. Your house isn't going to have any meaningful reliability for either the network or power. Your house will have one connection to the outside work, the DC should have multiple for redundancy (In case a digger goes through your network connection), and same for power any good DC will have their own redundant power supplies (Batteries, generators etc). And if your server is serving a lot of traffic, your consumer home ISP might not be too happy about it. Theres a reason they split plans between home and business.
  - nucleardog 2y ago
    > Your house isn't going to have any meaningful reliability for either the network or power. Not really a serious argument, but maybe worth looking at this these days. Mostly because I think as engineers we all have the desire to hit 100% uptime and reliability when in a lot of cases "close enough" might be a lot further than you think. On the power end, a lot of people are getting to some form of local generation and storage (e.g., solar + batteries). Small backup generators aren't entirely unheard of. And on the network end, I'm probably not the only person that went to multiple connections when my ability to earn income became tied to my ability to be online from home. As it sits right now, my power's not the most reliable in the world but because of that I already have a fair sized generator and a lot of fuel stored on-site (~5 days of gasoline, ~10 days of propane). I'm directly off of a fairly regionally important power line, so I'm usually fairly high up on the priority list during any major outage. (I mean, not "hospital" but if my power's out so are tens of thousands of others generally.) My grid power's rarely out for that long, and my local power even less since I have the generator. If I finally got around to adding some solar and storage to bridge between grid going down and the generator firing up I'd be able to pretty comfortable ignore power outages for over a week before I started needing to worry about making trips out to dump fuel in the generator or calling for some propane. As far as connectivity, I've got three connections across two providers. All my traffic hits a real data center first so I can bond the connections (because residential) then flows to and from my house. One of the providers is wireless so the only single point of failure for cut lines is a few feet in front of my router. I'm well aware this gets nowhere close to the reliability of a data center. I wouldn't run anything safety or life critical this way. I wouldn't run anything making a substantial amount of money this way. But there's a lot of stuff that fits in below "successful SaaS" and "life critical" that I think could tolerate an hour of downtime here or there that would get along just fine like this. And even if you think "well, you're clearly an insane person no one else is going to do that" I think there's still lots yet that could tolerate a day or two of downtime every year or two just fine. There are a lot of smaller services out there that businesses rely on that are hosted in AWS but aren't much more reliable than if they were running in someone's basement and they survive just fine.
theideaofcoffee 2y ago
Missing step -1: find someone to get on your staff that knows what they are doing, or at least find a contractor with a good history of doing this kind of thing. After a certain point, you really shouldn't wing it. But if you're at that point, you should know if you actually need your own hardware, or if public cloud or other options are good for you. Some need their own, most don't. There's a lot of nuance there that is difficult to express in a short HN comment.
- imperialdrive 2y ago
  Agreed. It blows my mind a bit that there is a generation of folks that only know Cloud vs Colo. Colo is a walk in the park compared to cloud, assuming you've done basic PC building and networking. Heck, Cloud can be a walk in the park too for most use cases. If you don't have time to learn it yourself over a month or two, there should be folks available to consult. A lot can be brainstormed in an hour.
  - AtlasBarfed 2y ago
    Cloud has documentation (usually) and can be lather-rinse-repeat as needed. You think it's hard compared to colo? The "clowns" are annoying as hell in their costs and dominance of CIOs, but there are distinct advantages of experimentation in clouds. Um, as long as it isn't your credit card...
jareklupinski 2y ago
> the bare minimum required to run your own hardware in a colocation environment i remember the look in the admin's eyes when they asked "alright, what kind of hardware are you looking to install?" and I said "oh i have it right here" and pulled two Intel NUCs out of my backpack > Consider bringing a screwdriver and a flashlight (any halfway decent place will provide those for you, but you never know). two multitools minimum, sometimes you need to hold a nut with one while loosening the bolt with the other the best space is the one that is right next to a Frys/Microcenter/whathaveyou
- theideaofcoffee 2y ago
  > "oh i have it right here" and pulled two Intel NUCs out of my backpack NUCs would be like nirvana after some of the jerry-rigged crap I've seen dragged into facilities. Maybe I'd like them to have a secondary power supply, but then again you had two, but they'd make a fantastic little router for something not too traffic heavy. Lots of utility use cases for something like that in a proper facility.
  - jareklupinski 2y ago
    > I'd like them to have a secondary power supply, but then again you had two yup! it was cheaper to bring two NUCs than try to get N+1 power redundancy set up for "atypical" (not u-rack) systems :) then they started making 1U NUC rack mounts...
- AtlasBarfed 2y ago
  Frys still exists? Are they "back"?
brodouevencode 2y ago
Is this cloud repatriation?
zamadatix 2y ago
Get something like a PiKVM and drop all of the stuff about being very local since you can find a cheaper overall provider elsewhere and use smart hands once a year either for free or still cheaper than picking the local place you can drive to. Even if you do the things in this guide perfectly it'll break/hang/get misconfigured at some point and the PiKVM (or like) lets you remotely hard boot a box instantly without having to drive or open a ticket. It also enables you to easily reinstall the entire OS remotely if you need to. If your server/device has an IPMI... get a PiKVM (or like) anyways. Not only will you last more than 2 seconds without being hacked but it'll have more functionality and be much faster. If you're in the US there are lots of places in the Kansas City area that have ridiculously cheap pricing and it's decently centrally located in the country.
- cheschire 2y ago
  https://pikvm.org/ https://pikvm.org/ Thanks, I had never heard of PiKVM!
- skybrian 2y ago
  What do you mean by “smart hands?” Will they replace hardware that failed?
  - Nux 2y ago
    DC's staff, yes they can replace hw if you have spares or can mail/order one in.
- alexdunmow 2y ago
  If you don't want to assemble a PiKVM yourself there's always Tiny Pilot: https://tinypilotkvm.com/ https://tinypilotkvm.com/
  - zamadatix 2y ago
    I don't want people to think TinyPilot is a bad option but this is where their marketing really grinds my gears. PiKVM and TinyPilot both have preassembled ordering option. TinyPilot's compare and contrast point of "To exercise the full functionality of PiKVM, users must install a custom circuit board on top of their motherboard and re-route their power supply's ATX pins." is a complete farce (it's not required) and worded in an intentionally scary way (the only reason TinyPilot doesn't have this requirement it doesn't offer the feature while PiKVM does). The "custom circuit board" is literally a PCB that, optionally, allows you to jumper the inline the remote power controls in a way that the normal power buttons still work too rather than rely on ACPI signalling over USB. It honestly makes my blood boil to see this underhanded approach works so well... the TinyPilot device is good though, as are some other options. Just keep in mind if you opt to go with ACPI only remote power controls via USB things may still go bad if your system hangs/crashes/gets in a weird power state whereas plugging in the wires to the PiKVM will be no different than holding the power button.
renewiltord 2y ago
Imho, the easiest way is to get a Mikrotik switch, set up Wireguard on it, allow only Wireguard in, then use the Wireguard Mac App. Build your machines with AsRock Rack or Gigabyte motherboards. If you have existing, then add a PiKVM. You'll get an IPv4 with a half-cab in most DCs. HE will give you a full cabinet for $600/month https://he.net/colocation.html?a=707336661752&n=g&m=e&k=hurricane%20electric%20colocation&gad_source=1 https://he.net/colocation.html?a=707336661752&n=g&m=e&k=hurr...
vfclists 2y ago
Is colo necessary of you have a 1000Mbit symmetrical provider at home?
- doubled112 2y ago
  I selfhost a bunch of personal stuff at home and have 1500M/940M at home, and I don’t think bandwidth would ever be the issue. Frequent IP changes and the residential IP block could make mail hard. If I was looking to host something that mattered, and was a little more public, I would move that more for my own sanity. Not being able to find me via IP, network uptime guarantees, more robust backup power, etc. If you were some cloud admin and put your website on a resume or something, and it was down, I might use that to jump to conclusions.
  - speckx 2y ago
    Remember that running any internet-facing servers could violate your terms of service unless you have a business-class connection.
johnklos 2y ago
I'm writing up a how-to about colocating using one of those Pi colocation offers. Glad to see colocation get some more attention!
mastazi 2y ago
She wrote a "part 2" just today https://rachelbythebay.com/w/2024/09/23/colo/ https://rachelbythebay.com/w/2024/09/23/colo/
throwaway984393 2y ago
[dead]
divbzero 2y ago
Does anyone have a sense of how much colocation costs? Assuming you have a modest setup that would otherwise run on a home server or VPS?
- Ir0nMan 2y ago
  A VPS will be cheaper, easier, and better in /almost/ all ways. 1U rack space, including power, a 1Gb network uplink, and at least 1 dedicated IPv4 IP, is going to be at least $60-80/month, and even that would from small discount colo providers you have never heard of.
- bluedino 2y ago
  It's hard to say. Any of the big places are going to be an 'email us for a quote' pricing model. https://ussignal.com/uploads/general/Grand-Rapids-East-Data-Center-Data-Sheet.pdf https://ussignal.com/uploads/general/Grand-Rapids-East-Data-... I haven't rented any space in a while but I'm going to assume $400/month is about the cheapest that you would be able to get a half a cabinet. And of course, that depends on how much bandwidth you need, how much power you need, are you running top-tier connectivity or something like Cogent/he.net...
shrubble 2y ago
If someone asked me "right now" and they didn't have unusual hardware needs, and they were dipping their toes into it, I would suggest wholesaleinternet.net and get 1 or more cheap servers to practice with. I rent something there for about $25/month, though you can get a better setup for a bit more. When you have a better handle on what works for you, then go with colo (meaning you don't rent servers, but buy and setup your own servers and Ethernet switches, that go into your rack space).
thatsit 2y ago
„If you just locked yourself out of sshd because you didn't install ssh keys first, STOP HERE. You are not ready for this.“ that one had me chuckling
whalesalad 2y ago
I have 2x R720's that I am thinking of dropping into a colo here in Michigan. They'll be running Proxmox in a cluster (no HA as you need 3+ nodes for that). Can someone talk me out of this? It'll be $200/mo for both at Smarthost (looking glass here, http://mi1.smarthost.net/ http://mi1.smarthost.net/, I get ~12ms ping to this datacenter) I guess I could leave them here at home and probably pay less in power than the total colo cost, but the bandwidth here at home is not great.
- justsomehnguy 2y ago
  R720 is too old, slow and with no paths for the upgrades. I know, I have one in the DC. If they aren't packed there are probably some other options, more performant and having more storage now.

238 points by ingve 2y ago

Why is the tone of the article so negative and grouchy? On one hand the author doesn’t owe anybody fake kindness and it’s their blog. On the other hand - if you don’t like the topic or giving advice, then why write a grumpy article about it?

N8works 2y ago

This is very 2004. I like it. RIP Alchemy.

yjftsjthsd-h 2y ago

> 1. Install Linux on the box. Turn everything off but sshd. Turn off password access to sshd. Also, test that it's properly disabled with something like `ssh -v yourserver : 2>&1 | grep continue`, because there are a surprising number of ways for that to go wrong (did you know that sshd lets you Include multiple config files together in a way that can override the main one? I know that now.)

Maledictus 2y ago
and `sshd -T | grep -i password`
a-french-anon 2y ago
This. OVH's VPS had two .confs reenabling passwords. Now I know too.
- a-french-anon 2y ago
  I should specify that this was with AlmaLinux 9. The offending files were a 50-redhat.conf and a 50-cloud-init.conf.
- omgtehlion 2y ago
  default ubuntu 22.04 has at least one, too
omgtehlion 2y ago
I always remove all 'includes' in sshd config, just to be sure I will not get any surprises later...
- zokier 2y ago
  I think doing the opposite is better usually, never making changes to the main /etc/ssh/sshd_config and always add your own customizations to /etc/ssh/sshd_config.d/; that way you have clear separation of your own and distro configs, which makes life easier especially when upgrading sshd. Although I'll readily admit that I don't myself follow that advise all the time.
  - xyse53 2y ago
    This. And give your drop-in a prefix like 00-*.conf. Fwiw I don't think SSH adds the include line upstream. Most distros add it now.
    - jgalt212 2y ago
      Yes. I had a hard time figuring out if a low numbered prefix got final say, or the high numbered one.
hedora 2y ago
In situations where I have more than one box, one (usually a little embedded SoC) gets openbsd on it, and is used as a bastion host, for exactly the reasons outlined here. The problems I hit with using Linux for this were different ten years ago, but, based on this thread, things got worse on that side of the fence.
ycombinatrix 2y ago
iirc the double config file nonsense is another genius move brought to you by debian maintainers. it seriously bothered me that an update automatically re-enabled password authentication. i ended up switching to a different OS.
- taskforcegemini 2y ago
  on updates you are asked if your conf file should be kept or overwritten, with further options like diff-comparison

cuu508 2y ago

What's the importance of having a switch/hub? Is it because you may want to add more servers later, but the colo host only provides one port in their router?

re-thc 2y ago
Also if you want to manage the IP ranges provided amongst >1 server.
pests 2y ago
Basically. Need to pay for a bigger pipe. Same applies to power as well, gotta pay to get more capacity.
kelnos 2y ago
So you can plug your laptop into it at the colo when you set it up so you can verify that the server is working right and can get to the internet while you are still there. And if it isn't working right, it'll be a lot easier to keep everything plugged into the switch, rather than swapping between your laptop and the colo's network every time you try something to fix it. But yes, if you think you might want to add other machines later, preemptively putting a switch in there will make it tons easier.

moandcompany 2y ago

Documenting and testing the hard reset/reboot procedure, as well as any expectations, for your colocated gear sounds like a good thing to add to this list.

bigfatkitten 2y ago
Test to make sure it'll actually come back after a hard reset. Don't do what I did and put a Cisco router on a remote mountaintop radio site with the config register still set to 0x2142, and then go home.
- pferde 2y ago
  Hey, that's just an excuse to go for another hike! :)
  - bigfatkitten 2y ago
    3 hour drive in warm weather, or an expensive helicopter ride in winter!
- fanf2 2y ago
  Also, check what happens when mains power disappears and returns.
- moandcompany 2y ago
  Systems Administration the Hard Way... it's an uphill hike, both ways, and in the snow :)

krab 2y ago

A bit less terrible way in my opinion: Find a dedicated server provider and rent the hardware. These companies rent some part of the datacenter (or sometimes build their own). Bonus points if they offer KVM - as in remote console, not the Linux hypervisor. Also ask if they do hardware monitoring and proactively replace the failed parts. All of this is still way cheaper than cloud. Usually with unmetered networking. Way less hassle. They'll even take your existing stuff and put it into the same rack with the rented hardware. The difference from cloud, apart from the price, is mainly that they have a sales rep instead of an API. And getting a server may take a from few hours to a few days. But in the end you get the same SSH login details you would get from a cloud provider. Or, if you really want to just collocate your boxes, the providers offer "remote hands" service, so you can have geo-redundancy or just choose a better deal instead of one that's physically close to your place.

rsanheim 2y ago
One hurdle that many companies who have only known cloud hosting will face here is significant: how do you find a reliable, trustworthy datacenter? One who actually monitors the hardware and also has a real human if your networking access gets screwed or if you need a critical component swapped at 2 am on a Saturday. I used to have a short list of trustworthy companies like this I'd recommend to clients ~20 years ago when doing consulting. I think 3/4 of them have been gobbled up by private equity chop shops or are just gone. Nowadays noone gets fired for going with AWS, or resold AWS with a 100% markup from a 'private enterprise cloud' provider.
- krab 2y ago
  You're right you need to find a company you can trust. And for a lot of startups it really makes sense to use AWS. But if you do something resource or bandwidth intensive (and I'm not even talking about Llama now), the costs add up quickly. In our case, switching to AWS would increase our costs by an equivalent of 4 - 8 devs salaries. After AWS discounts. That's a hard sell in a 15-person team even though half of our infra costs already are with AWS (S3).
  - 2y ago

ggm 2y ago

Remote power management can be a godsend. If you can get an ipmi console in, you want it.

michaelt 2y ago
It is indeed very helpful when it's needed - but most IPMI is horrifically insecure, so usually it's not connected directly to the public internet. Instead it's on a separate network with special VPN access. As this document focuses on keeping things simple, they don't have the isolated network / VPN needed to use IPMI.
qhwudbebd 2y ago
Ipmi is a bit of a double-edged sword. Network-connected access to a serial console (including uefi/bios console redirection) and the reset button can be a total lifesaver, I agree. I wouldn't want to be without a serial console and remote reset button either. But ipmi cards are little network-attached linux boxes which run prehistoric kernels and userspace, exposing as many services as the half-wits who put together the firmware image can shovel in, and are rarely if ever patched by the vendor unless there's some really public scandal. The standard thing to do is to isolate them on some kind of private management network in an attempt to shield the wider internet from the full majesty of the firmware engineers' dazzling skills, but that might be harder to do in the simple 'beginner' scenario Rachel describes. One good simple version when you get up to two servers instead of just one is to cross-connect so each machine has ipmi access to the other, but neither ipmi service is exposed to the wider world.
- omgtehlion 2y ago
  Even if your server does not have proper IPMI, most colo providers have an assortment of ip-kvms, you can ask for those and access your server from any modern browser.
  - qhwudbebd 2y ago
    I'm pretty suspicious of trusting the firmware on things like that either, to be honest. (Personally, I tend to use serial cables and a little custom board to turn serial breaks into resets, but I know my NIH instinct is probably a little on the high side. It would be unreasonable for me to suggest someone new to colo start making random extra boards to stick inside their machines!)

mjevans 2y ago

They're absolutely correct: " 1. Install Linux on the box. Turn everything off but sshd. Turn off password access to sshd. If you just locked yourself out of sshd because you didn't install ssh keys first, STOP HERE. You are not ready for this. " If you blindly followed the directions and got locked out, you can do exactly the same thing with other directions. You were not ready.

kijin 2y ago
The great thing about having unfettered physical access to hardware is that you can easily recover from mistakes like this. No need to rebuild that EC2 instance. No need to beg a hosting company for IPMI access. You can just pull the plug and try again as if were your own PC.
- jiggunjer 2y ago
  Isn't it the same with something like idrac?
- bigiain 2y ago
  On the other hand, it's nice to be able to take a no-reboot AMI of the machine you're futzing with, risk breaking it, then just re provision from that AMI if you fucked up. And if you're chasing three or four nines of uptime you do the same but provision a new instance and futz with that one while your working one keeps on keeping on. Even big instances are cheap enough by the hour that you can afford to run and extra one while testing potentially breaking changes.
nasretdinov 2y ago
At least it doesn't say to set PermitRootLogon and remove the root password :)
- Foobar8568 2y ago
  AKA, "We are compatible with passwordless authentication mechanism." Or SSO, depending of the use case.... Le sight.

behringer 2y ago

Forget the switch and pi and get a ubiquity router. Much more powerful and simple to setup. Does require some knowhow. Also you could see if your local hackers space has a server rack

seszett 2y ago

The most difficult step I find is just barely mentioned, finding colocation space at reasonable price is difficult these days.

c0l0 2y ago
I faced the same problem, until I found a non-profit community-operated housing provider (a registered computer club/society that has been operating since at least the early 2000 who is also a LIR (https://en.wikipedia.org/wiki/Regional_Internet_registry#Local_Internet_registry https://en.wikipedia.org/wiki/Regional_Internet_registry#Loc...), and they give shelter, bandwidth and IP address space to anyone who participates in covering their operational costs. I gladly do, and it's the best hosting experience I've had so far, and I used to have rented dedicated iron from Server4You, Hetzner, Webtropia and the like from 2005 on. Maybe there's a similar hidden gem in the area you live in, and you just do not know about it? :) Mine flew under my radar for nearly 20 years, and even though I knew they existed, I was not aware they'd colo other peoples' boxen at very fair rates.
- hiatus 2y ago
  How did you find your local non-profit? Is there a resource you can share?
  - c0l0 2y ago
    There is, unfortunately, no directory of services like this that I am aware of. If you have a local hackerspace, asking specific questions there would probably give you answers quickly, I imagine :)
- oarsinsync 2y ago
  Is this provider for local people only, or can people from afar also participate in contributing towards costs in exchange for services? If so, can you share their details? I need someone to house some RIPE objects for me.
acatton 2y ago

emptiestplace 2y ago

What audience was this written for?

schmookeeg 2y ago
I was gonna ask the same thing, but then she called me a usual broken people and I decided "not me" was a good enough answer :) This was a tortured way to setup a colo. I bring rackable gear, no switch, and use the colo's "setup station" to do final config, then I rack the sucker. They can always VLAN me and my laptop together somewhere for a final check before I leave. I'd be worried if they couldn't.

kkfx 2y ago

I suggest a very different reasoning: we know WFH works, we also agree that in large part of the world we have enough sunlight for p.v., battery storage could be cheap [1] and FTTH connections start be be very good for modern time. Who have already understood where I want to go? Well, we could run decentralized systems at home for SMEs/Startup common needs from modern homes of remote workers, just renting them a room with proper local equipment the modern remote worker probably already have (VMC, A/C, "the big UPS" from domestic p.v.) with not much more expenses than using someone else computer also named "the cloud". Then if/when needs mount some sheds with p.v. + local storage and good connections can pops up. Who need a CDN in such setup? Then/if a real small datacenter could born. Are you horrified? Well, now try to see the recent history, when Amazon choose not to buy expensive and reliable SUN servers choosing cheap and crappy PCs or Google choose to save money with rackmounted velcro-assembled PCs with a built-in UPS, i.e. a naked velcro strapped one. See the history of buying cheap low quality ram because we have ECC, lo quality storage because we have checksums anyway and so on. Then think about some real recent datacenter fires and what they told about "how brilliant and safe" was their design. Long story short: as we ditch big iron in the past, we ditch mainframes, it's about time to think ditching dataceter model as well for a spread set of small unreliable but many machine rooms at homelabs scale. This, like FLOSS gives us reliability, cheapness and ownership as well. This ALSO decouple hw and sw giants, witch is a very good things in terms of free markets. It will also force he OEMs to be flexible in their design and ditching slowly from modern crappy fats-tech also some giants deprecate [2] as unsustainable, for more balanced tech because well, such homelabs last much longer than typical ready-made PCs and craptops or NAS from the mass distribution market. [1] buying from China directly instead of feeding local importers who skyrocket the price instead of lowering it when Chinese supplier lower them [2] https://unctad.org/system/files/official-document/der2024_en.pdf https://unctad.org/system/files/official-document/der2024_en... of course suggesting the wrong answer

omgtehlion 2y ago

It is also useful to configure mutiple IPs on the same interface, one for Colo, and one for your lab. You can do this on all OSes, even on windows. And if you failed to do this and all advices in the article: then kindly ask colo provider to attach ip-KVM (if your "server" is not actual server with ipmi/bmc), 99.9% of places have these. And most of them will provide it free of charge for limited amount of time.

neomantra 2y ago

Don't forget hearing protection! All the fans in there make it crazy loud. Facilities often supply earplugs, but consider using it with over-the-ear protection.

elric 2y ago

Back in the late 90s/early 00s when I was a precocious teenager, I ran a somewhat popular website. At some point it made sense to just buy a 1U rack moubtable server and having it colocated (commercial webhosting was expensive then). I couldn't find anyone to give me a ride to the datacenter, so I took a bus. By the time I got there my arms were numb from carrying the bloody thing. There was a single security guard, I signed in and he gave me directions and a big keychain. The keys opened most of the rooms and most of the cages around the racks. To this day I remain mystified at the level of trust (or nonchalance) that security guard had in a spotty teenager.

qhwudbebd 2y ago
Having run hosting companies from the mid 90s as well, from memory this kind of thing was pretty normal, even in allegedly secure places like London Telehouse. Quite a few of us in that era were juggling it with being students, so it wouldn't surprise me if the security staff were used to it and expected you to look young enough to be their kid!
- anal_reactor 2y ago
  Modern society is absolutely paranoid. Zero-trust policy, everyone is treated like a potential terrorist.
  - thelastparadise 2y ago
    You're treated lile a terrorist when you are a law abiding citizen. Actual criminal? Society will bend over to accommodate you.
  - phil21 2y ago
    It has nothing really to do with that. To this day I can get into pretty much any rack or room I feel like at datacenters everyone here has heard of. It just takes experience these days and a bit of charm. Plus having a million keys and staff rack combination codes doesn't hurt. These were freely given and simply added to my collection over time, nothing stolen or social engineered. I've never done anything nefarious with these abilities, and no one I know has either. It's simply a matter of practicality when you staff a 150,000 square foot facility with 2 security guards who have no idea what they are doing. If I (and many others) had wanted to, we could have caused multi-week/month outages you'd be reading about on the news with 5 minutes of effort. This is basically the status quo for any sensitive industry. The world turns because 99.9999% of people want to give you a hug vs. hit you. Society falls when that ratio goes much lower.

andyjohnson0 2y ago

Interesting article. > Plug everything in and turn it on. Most server racks will have C13 or C19 outlets for power distribution. For consumer-type devices designed to plug into a wall socket, suitable cables or an adapter strip would probably be required.

methou 2y ago

IPMI/BMC is a must if plan to keep the server running longer than a release cycle of your Linux distribution. Also remember ask your provider to assign you a private vlan and vpn to it for your own access only.

rpcope1 2y ago
A working BMC is great, but I sure wouldn't want it facing the open web. The colo would either need some sort of private network, or you'd need to add a router and firewall of some sort to lock down access to the BMC.

efortis 2y ago

My tip is adding hot spare disks so you don't have to replace broken ones. For example, in ZFS you can mirror disks 1 and 2, while having 3 and 4 as hot spares with: zpool create pool mirror $d1 $d2 spare $d3 $d4

thelastparadise 2y ago
How about make it so you can smash the whole rack with a sledgehammer or datacenter burn down and system hums along like nothing happened.
- iphoneisbetter 2y ago
  No; because that costs at least an order of magnitude more money than a simple colo setup.

ForHackernews 2y ago

Why would you do this instead of having the hardware at your house? For better/faster connection?

gsck 2y ago
Reliability, and depending on your work load the ability to use it commercially. Your house isn't going to have any meaningful reliability for either the network or power. Your house will have one connection to the outside work, the DC should have multiple for redundancy (In case a digger goes through your network connection), and same for power any good DC will have their own redundant power supplies (Batteries, generators etc). And if your server is serving a lot of traffic, your consumer home ISP might not be too happy about it. Theres a reason they split plans between home and business.
- nucleardog 2y ago
  > Your house isn't going to have any meaningful reliability for either the network or power. Not really a serious argument, but maybe worth looking at this these days. Mostly because I think as engineers we all have the desire to hit 100% uptime and reliability when in a lot of cases "close enough" might be a lot further than you think. On the power end, a lot of people are getting to some form of local generation and storage (e.g., solar + batteries). Small backup generators aren't entirely unheard of. And on the network end, I'm probably not the only person that went to multiple connections when my ability to earn income became tied to my ability to be online from home. As it sits right now, my power's not the most reliable in the world but because of that I already have a fair sized generator and a lot of fuel stored on-site (~5 days of gasoline, ~10 days of propane). I'm directly off of a fairly regionally important power line, so I'm usually fairly high up on the priority list during any major outage. (I mean, not "hospital" but if my power's out so are tens of thousands of others generally.) My grid power's rarely out for that long, and my local power even less since I have the generator. If I finally got around to adding some solar and storage to bridge between grid going down and the generator firing up I'd be able to pretty comfortable ignore power outages for over a week before I started needing to worry about making trips out to dump fuel in the generator or calling for some propane. As far as connectivity, I've got three connections across two providers. All my traffic hits a real data center first so I can bond the connections (because residential) then flows to and from my house. One of the providers is wireless so the only single point of failure for cut lines is a few feet in front of my router. I'm well aware this gets nowhere close to the reliability of a data center. I wouldn't run anything safety or life critical this way. I wouldn't run anything making a substantial amount of money this way. But there's a lot of stuff that fits in below "successful SaaS" and "life critical" that I think could tolerate an hour of downtime here or there that would get along just fine like this. And even if you think "well, you're clearly an insane person no one else is going to do that" I think there's still lots yet that could tolerate a day or two of downtime every year or two just fine. There are a lot of smaller services out there that businesses rely on that are hosted in AWS but aren't much more reliable than if they were running in someone's basement and they survive just fine.

theideaofcoffee 2y ago

Missing step -1: find someone to get on your staff that knows what they are doing, or at least find a contractor with a good history of doing this kind of thing. After a certain point, you really shouldn't wing it. But if you're at that point, you should know if you actually need your own hardware, or if public cloud or other options are good for you. Some need their own, most don't. There's a lot of nuance there that is difficult to express in a short HN comment.

imperialdrive 2y ago
Agreed. It blows my mind a bit that there is a generation of folks that only know Cloud vs Colo. Colo is a walk in the park compared to cloud, assuming you've done basic PC building and networking. Heck, Cloud can be a walk in the park too for most use cases. If you don't have time to learn it yourself over a month or two, there should be folks available to consult. A lot can be brainstormed in an hour.
- AtlasBarfed 2y ago
  Cloud has documentation (usually) and can be lather-rinse-repeat as needed. You think it's hard compared to colo? The "clowns" are annoying as hell in their costs and dominance of CIOs, but there are distinct advantages of experimentation in clouds. Um, as long as it isn't your credit card...

jareklupinski 2y ago

> the bare minimum required to run your own hardware in a colocation environment i remember the look in the admin's eyes when they asked "alright, what kind of hardware are you looking to install?" and I said "oh i have it right here" and pulled two Intel NUCs out of my backpack > Consider bringing a screwdriver and a flashlight (any halfway decent place will provide those for you, but you never know). two multitools minimum, sometimes you need to hold a nut with one while loosening the bolt with the other the best space is the one that is right next to a Frys/Microcenter/whathaveyou

theideaofcoffee 2y ago
> "oh i have it right here" and pulled two Intel NUCs out of my backpack NUCs would be like nirvana after some of the jerry-rigged crap I've seen dragged into facilities. Maybe I'd like them to have a secondary power supply, but then again you had two, but they'd make a fantastic little router for something not too traffic heavy. Lots of utility use cases for something like that in a proper facility.
- jareklupinski 2y ago
  > I'd like them to have a secondary power supply, but then again you had two yup! it was cheaper to bring two NUCs than try to get N+1 power redundancy set up for "atypical" (not u-rack) systems :) then they started making 1U NUC rack mounts...
AtlasBarfed 2y ago
Frys still exists? Are they "back"?

brodouevencode 2y ago

Is this cloud repatriation?

zamadatix 2y ago

Get something like a PiKVM and drop all of the stuff about being very local since you can find a cheaper overall provider elsewhere and use smart hands once a year either for free or still cheaper than picking the local place you can drive to. Even if you do the things in this guide perfectly it'll break/hang/get misconfigured at some point and the PiKVM (or like) lets you remotely hard boot a box instantly without having to drive or open a ticket. It also enables you to easily reinstall the entire OS remotely if you need to. If your server/device has an IPMI... get a PiKVM (or like) anyways. Not only will you last more than 2 seconds without being hacked but it'll have more functionality and be much faster. If you're in the US there are lots of places in the Kansas City area that have ridiculously cheap pricing and it's decently centrally located in the country.

cheschire 2y ago
https://pikvm.org/ https://pikvm.org/ Thanks, I had never heard of PiKVM!
skybrian 2y ago
What do you mean by “smart hands?” Will they replace hardware that failed?
- Nux 2y ago
  DC's staff, yes they can replace hw if you have spares or can mail/order one in.
alexdunmow 2y ago
If you don't want to assemble a PiKVM yourself there's always Tiny Pilot: https://tinypilotkvm.com/ https://tinypilotkvm.com/
- zamadatix 2y ago
  I don't want people to think TinyPilot is a bad option but this is where their marketing really grinds my gears. PiKVM and TinyPilot both have preassembled ordering option. TinyPilot's compare and contrast point of "To exercise the full functionality of PiKVM, users must install a custom circuit board on top of their motherboard and re-route their power supply's ATX pins." is a complete farce (it's not required) and worded in an intentionally scary way (the only reason TinyPilot doesn't have this requirement it doesn't offer the feature while PiKVM does). The "custom circuit board" is literally a PCB that, optionally, allows you to jumper the inline the remote power controls in a way that the normal power buttons still work too rather than rely on ACPI signalling over USB. It honestly makes my blood boil to see this underhanded approach works so well... the TinyPilot device is good though, as are some other options. Just keep in mind if you opt to go with ACPI only remote power controls via USB things may still go bad if your system hangs/crashes/gets in a weird power state whereas plugging in the wires to the PiKVM will be no different than holding the power button.

renewiltord 2y ago

Imho, the easiest way is to get a Mikrotik switch, set up Wireguard on it, allow only Wireguard in, then use the Wireguard Mac App. Build your machines with AsRock Rack or Gigabyte motherboards. If you have existing, then add a PiKVM. You'll get an IPv4 with a half-cab in most DCs. HE will give you a full cabinet for $600/month https://he.net/colocation.html?a=707336661752&n=g&m=e&k=hurricane%20electric%20colocation&gad_source=1 https://he.net/colocation.html?a=707336661752&n=g&m=e&k=hurr...

vfclists 2y ago

Is colo necessary of you have a 1000Mbit symmetrical provider at home?

doubled112 2y ago
I selfhost a bunch of personal stuff at home and have 1500M/940M at home, and I don’t think bandwidth would ever be the issue. Frequent IP changes and the residential IP block could make mail hard. If I was looking to host something that mattered, and was a little more public, I would move that more for my own sanity. Not being able to find me via IP, network uptime guarantees, more robust backup power, etc. If you were some cloud admin and put your website on a resume or something, and it was down, I might use that to jump to conclusions.
- speckx 2y ago
  Remember that running any internet-facing servers could violate your terms of service unless you have a business-class connection.

johnklos 2y ago

I'm writing up a how-to about colocating using one of those Pi colocation offers. Glad to see colocation get some more attention!

mastazi 2y ago

She wrote a "part 2" just today https://rachelbythebay.com/w/2024/09/23/colo/ https://rachelbythebay.com/w/2024/09/23/colo/

throwaway984393 2y ago

[dead]

divbzero 2y ago

Does anyone have a sense of how much colocation costs? Assuming you have a modest setup that would otherwise run on a home server or VPS?

Ir0nMan 2y ago
A VPS will be cheaper, easier, and better in /almost/ all ways. 1U rack space, including power, a 1Gb network uplink, and at least 1 dedicated IPv4 IP, is going to be at least $60-80/month, and even that would from small discount colo providers you have never heard of.
bluedino 2y ago
It's hard to say. Any of the big places are going to be an 'email us for a quote' pricing model. https://ussignal.com/uploads/general/Grand-Rapids-East-Data-Center-Data-Sheet.pdf https://ussignal.com/uploads/general/Grand-Rapids-East-Data-... I haven't rented any space in a while but I'm going to assume $400/month is about the cheapest that you would be able to get a half a cabinet. And of course, that depends on how much bandwidth you need, how much power you need, are you running top-tier connectivity or something like Cogent/he.net...

shrubble 2y ago

If someone asked me "right now" and they didn't have unusual hardware needs, and they were dipping their toes into it, I would suggest wholesaleinternet.net and get 1 or more cheap servers to practice with. I rent something there for about $25/month, though you can get a better setup for a bit more. When you have a better handle on what works for you, then go with colo (meaning you don't rent servers, but buy and setup your own servers and Ethernet switches, that go into your rack space).

thatsit 2y ago

„If you just locked yourself out of sshd because you didn't install ssh keys first, STOP HERE. You are not ready for this.“ that one had me chuckling

whalesalad 2y ago

I have 2x R720's that I am thinking of dropping into a colo here in Michigan. They'll be running Proxmox in a cluster (no HA as you need 3+ nodes for that). Can someone talk me out of this? It'll be $200/mo for both at Smarthost (looking glass here, http://mi1.smarthost.net/ http://mi1.smarthost.net/, I get ~12ms ping to this datacenter) I guess I could leave them here at home and probably pay less in power than the total colo cost, but the bandwidth here at home is not great.

justsomehnguy 2y ago
R720 is too old, slow and with no paths for the upgrades. I know, I have one in the DC. If they aren't packed there are probably some other options, more performant and having more storage now.