3 ms·

I run a few services from my home but still have to rely on aws/fly.io for some portions of my infrastructure. I really want is to learn how to rent rack space

by rank0 5y ago

I run a few services from my home but still have to rely on aws/fly.io for some portions of my infrastructure. I really want is to learn how to rent rack space from a colocation. The documentation available does not make it easy to learn. Can I just buy an old 1U blade, throw xen on it and show up at my nearest colo? What do I need to preconfigure to ensure I have remote access without giving remote access to the colo as well? Do I get physical access to the data center? Wish I could find some guides on this topic. 95% of blog post tutorials are just ads for the latest trendy cloud startup/language framework.

kjs3 5y ago
I've never worked with a colo vendor that once you contacted them didn't have exhaustive support for "how to we get to the point where we can start billing you", usually including an actual human that you can ask questions.
eddieroger 5y ago
I did this once. Don't overthink it too much - yes, it is as simple as finding a rack with sufficient space, power and network, plugging it in and going. You'll most likely get a public IP and have no access to your neighbors, so they won't really care what you do with it as long as it's not illegal or against the Terms of Service for your host. So yeah, if you want to do it, just do it. Get an OS you know, install an SSH server or Remote Desktop, and rack it up. If you can get to it on your LAN, you'll be able to get to it on the public Internet. Also, quickly learn about good auth and firewalls and fail2ban. That all said (and said with the clarity of age and knowing I was a stubborn kid who did things "because I could"), the experience of spinning up a VPS today on Linode or Digital Ocean is effectively the same, infinitely cheaper, and a lot more fun than racking a server somewhere. I can script up a fleet of servers from my bed at 1am just because, and can't tell the difference between SSH'ing to them versus that one box I did 15 years ago. If you want to do it, go nuts and have fun, but you really aren't really missing much over conventional VPSes these days.
- rank0 5y ago
  Thanks for the response! I gotta disagree with you though on cost. You can get a beefy refurbished dual Xeon blade for a couple hundred bucks. Rack space where I live is like $50/month for 1U and gets much cheaper/machine as you scale up. $50 on aws will get me maybe 1 medium ec2 instance and an s3 bucket. With a used blade I get 20x the compute for the same price.
  - dangus 5y ago
    You're overestimating AWS/cloud costs by a decent amount. t3a.medium is under $30/month and that price only goes down when you reserve for the year. Save even more if you can run your service on ARM/Gravitron. A VPS service like Linode will have even better pricing than AWS. Driving over to a data center isn't free (time and cost), either. Those used Xeon blades are cheap for good reason – the companies that originally owned them consider them EOL. There's "no such thing" as dealing with hardware failure (except for occasional stop/starts) in the cloud.
    - rank0 5y ago
      You’re right about the time not being free and certainly hardware failure. Although HW is way more reliable than people let on. But aws is extremely expensive per unit of raw compute performance. Obviously I concede that the stability and availability is unmatched compared to DIY solutions. For my specific workload I need lots of memory and cpu cores. (Lots meaning my single 32c/64t 32Gb Xeon tower in my home). I almost fully utilize my resources and STIL need to pay aws for storage/tunneling/DNS/etc. Consuming equivalent compute in aws would be hundreds of dollars per month. Hybrid is just my preference!
  - eddieroger 5y ago
    You're quite welcome. I'm not trying to dissuade you, just provide a point of view I've got from having felt the same way. I'm definitely not comparing to AWS, because yeah those can get super expensive, super fast. What you're paying for with AWS is Amazon-tier stability (whatever that's worth these days), but the difference in uptime between them and a Linode is more than fine for my needs. With your $50/mo, make sure that includes power - Xeons eat watts. Also, be sure to compare apples-to-apples on bandwidth. By comparison, Linode's dedicated CPU plan (not shared, closer to bare metal, but still not) starts at $30 for 2 CPUs and 4GB of RAM and 4TB of transfer, and they'll take care of keeping you on the latest hardware. Again, I don't want to dissuade you, because colo is fun and it's cool to think about your own box out in the world. If anything, I'm envious of how easy it is nowadays compared to when I drove 3 hours to South Bend, Indiana to colo a box of my own, or the first time I needed to engage remote hands because the box got in an irreparable state.
landemva 5y ago
If you have a cabinet, and neighbors are caged to prevent your access to those, then you may get physical access. Call a small provider near you and ask.
lbriner 5y ago
Sadly the answer is, as often, it depends! Many rack space rentals will not permit you to just install whatever PC you fancy because it is potentially a risk to the neighbours in terms of fire or bad hardware, most will happily quote you to buy one their approved ones! It is pretty easy to get a rack space provider where the provider cannot access the machine but this can be good or bad. In some cases, I would rather they could shutdown the host if, say, the RAM is broken and replace it but if you would prefer to do this yourself, that is fine. In most cases, you will be given a public IP address directly mapping to your machine via a router/nat lookup so whatever services you open on your machine are open on that public IP address so pretty easy to setup RDP/ssh/whatever. Probably the biggest issue though is the extra work or hassle if something goes wrong. I remember at a previous company where some guy would frequently have to drive for 30 minutes each way to go to a data centre to perform certain updates that couldn't be done remotely. YMMV
- vel0city 5y ago
  > Many rack space rentals will not permit you to just install whatever PC you fancy because it is potentially a risk to the neighbours in terms of fire or bad hardware, most will happily quote you to buy one their approved ones! I have _never_ experienced this. The only restrictions I've seen on colo contracts I've gone after were related to UPSes and things with large batteries in them. So a big stack of laptops would be a no, but if I wanted to put Atari ST's or Dell PowerEdges or white box builds or bitcoin miners it doesn't matter. I guess I've always done things at at least a half or full cab, never single Us at a time.
benedikt 5y ago
you’re not all that far off * you’d have to sign up with a colo provider first. since data centers in physical buildings, this just depends on where you live * when you sign up with them they provide you with info like ip addresses or how to connect to their network (they might have dhcp, or you might have to configure static ips). usually there is a initial setup fee, around 1 month of rent. * if you just rent a a 1U space you usually can get physical access to it while accompanied by someone working for the data center. usually this is during business hours, but each data center will have its own rules. if you rent larger units, such as a full rack (42U) or half a rack you usually get a key card and can access it 24/7 (this usually involves a phone call for them to remotely open a lock)
Moru 5y ago
With the ones I have used you just click around on the homepage selecting what you want on the server and then pay. Some sell second hand repurposed servers on auction that they will set up for you. A while later you get an SSH login on the server and that's it, your server is running somewhere in a basement/bunker/old mine and you can go visit it if you want but in general you can do everything remote. There is even stuff that can let you see the bootup in bios from remote (Called KVM I believe). Some help you set up backups on the server and help you with setting up programs on the server but then it starts to get expensive. You can also just rent a space to place your own server but I haven't tried that.
- rank0 5y ago
  In your experience did you have to sign up with a partner ISP at the colo? Or is that done for me and just part of my colo bill? Is power use included as well?
  - procombo 5y ago
    Colocation provider will bring the circuits to provide best-path connectivity based on packet destination. There shouldn't be an additional charge for this. They are incentivized to manage their bandwidth so data transfers fast, as they are likely charged wholesale for fiber availability. You will likely be charged 95th percentile mbps based on your usage. (Again, "pipe space required" to your needs.) Basically, whenever you're busiest -- 4pm-9pm are popular times for us in the USA. Some customers limit their bandwidth themselves (like, only allow max 12mbps file downloads, etc.) especially when they have the hardware to support huge bandwidth. Or your colocation provider can perhaps limit max connection to 100mbs or 1gbps if you want. Power is usually leased in amps. If you go over amps the circuit will break -- at worst case scenario. But typically they get in touch with you and tell you to upgrade. Also, they do want to know vaguely what your service is. Because you'll likely lease their IPs, they will question you if you do a lot of email (caution for spam), or run a Tor exit node (legal hassles for them in many cases).

by rank0 5y ago

kjs3 5y ago

I've never worked with a colo vendor that once you contacted them didn't have exhaustive support for "how to we get to the point where we can start billing you", usually including an actual human that you can ask questions.

eddieroger 5y ago

I did this once. Don't overthink it too much - yes, it is as simple as finding a rack with sufficient space, power and network, plugging it in and going. You'll most likely get a public IP and have no access to your neighbors, so they won't really care what you do with it as long as it's not illegal or against the Terms of Service for your host. So yeah, if you want to do it, just do it. Get an OS you know, install an SSH server or Remote Desktop, and rack it up. If you can get to it on your LAN, you'll be able to get to it on the public Internet. Also, quickly learn about good auth and firewalls and fail2ban. That all said (and said with the clarity of age and knowing I was a stubborn kid who did things "because I could"), the experience of spinning up a VPS today on Linode or Digital Ocean is effectively the same, infinitely cheaper, and a lot more fun than racking a server somewhere. I can script up a fleet of servers from my bed at 1am just because, and can't tell the difference between SSH'ing to them versus that one box I did 15 years ago. If you want to do it, go nuts and have fun, but you really aren't really missing much over conventional VPSes these days.

rank0 5y ago
Thanks for the response! I gotta disagree with you though on cost. You can get a beefy refurbished dual Xeon blade for a couple hundred bucks. Rack space where I live is like $50/month for 1U and gets much cheaper/machine as you scale up. $50 on aws will get me maybe 1 medium ec2 instance and an s3 bucket. With a used blade I get 20x the compute for the same price.
- dangus 5y ago
  You're overestimating AWS/cloud costs by a decent amount. t3a.medium is under $30/month and that price only goes down when you reserve for the year. Save even more if you can run your service on ARM/Gravitron. A VPS service like Linode will have even better pricing than AWS. Driving over to a data center isn't free (time and cost), either. Those used Xeon blades are cheap for good reason – the companies that originally owned them consider them EOL. There's "no such thing" as dealing with hardware failure (except for occasional stop/starts) in the cloud.
  - rank0 5y ago
    You’re right about the time not being free and certainly hardware failure. Although HW is way more reliable than people let on. But aws is extremely expensive per unit of raw compute performance. Obviously I concede that the stability and availability is unmatched compared to DIY solutions. For my specific workload I need lots of memory and cpu cores. (Lots meaning my single 32c/64t 32Gb Xeon tower in my home). I almost fully utilize my resources and STIL need to pay aws for storage/tunneling/DNS/etc. Consuming equivalent compute in aws would be hundreds of dollars per month. Hybrid is just my preference!
- eddieroger 5y ago
  You're quite welcome. I'm not trying to dissuade you, just provide a point of view I've got from having felt the same way. I'm definitely not comparing to AWS, because yeah those can get super expensive, super fast. What you're paying for with AWS is Amazon-tier stability (whatever that's worth these days), but the difference in uptime between them and a Linode is more than fine for my needs. With your $50/mo, make sure that includes power - Xeons eat watts. Also, be sure to compare apples-to-apples on bandwidth. By comparison, Linode's dedicated CPU plan (not shared, closer to bare metal, but still not) starts at $30 for 2 CPUs and 4GB of RAM and 4TB of transfer, and they'll take care of keeping you on the latest hardware. Again, I don't want to dissuade you, because colo is fun and it's cool to think about your own box out in the world. If anything, I'm envious of how easy it is nowadays compared to when I drove 3 hours to South Bend, Indiana to colo a box of my own, or the first time I needed to engage remote hands because the box got in an irreparable state.

landemva 5y ago

If you have a cabinet, and neighbors are caged to prevent your access to those, then you may get physical access. Call a small provider near you and ask.

lbriner 5y ago

Sadly the answer is, as often, it depends! Many rack space rentals will not permit you to just install whatever PC you fancy because it is potentially a risk to the neighbours in terms of fire or bad hardware, most will happily quote you to buy one their approved ones! It is pretty easy to get a rack space provider where the provider cannot access the machine but this can be good or bad. In some cases, I would rather they could shutdown the host if, say, the RAM is broken and replace it but if you would prefer to do this yourself, that is fine. In most cases, you will be given a public IP address directly mapping to your machine via a router/nat lookup so whatever services you open on your machine are open on that public IP address so pretty easy to setup RDP/ssh/whatever. Probably the biggest issue though is the extra work or hassle if something goes wrong. I remember at a previous company where some guy would frequently have to drive for 30 minutes each way to go to a data centre to perform certain updates that couldn't be done remotely. YMMV

vel0city 5y ago
> Many rack space rentals will not permit you to just install whatever PC you fancy because it is potentially a risk to the neighbours in terms of fire or bad hardware, most will happily quote you to buy one their approved ones! I have _never_ experienced this. The only restrictions I've seen on colo contracts I've gone after were related to UPSes and things with large batteries in them. So a big stack of laptops would be a no, but if I wanted to put Atari ST's or Dell PowerEdges or white box builds or bitcoin miners it doesn't matter. I guess I've always done things at at least a half or full cab, never single Us at a time.

benedikt 5y ago

you’re not all that far off * you’d have to sign up with a colo provider first. since data centers in physical buildings, this just depends on where you live * when you sign up with them they provide you with info like ip addresses or how to connect to their network (they might have dhcp, or you might have to configure static ips). usually there is a initial setup fee, around 1 month of rent. * if you just rent a a 1U space you usually can get physical access to it while accompanied by someone working for the data center. usually this is during business hours, but each data center will have its own rules. if you rent larger units, such as a full rack (42U) or half a rack you usually get a key card and can access it 24/7 (this usually involves a phone call for them to remotely open a lock)

Moru 5y ago

With the ones I have used you just click around on the homepage selecting what you want on the server and then pay. Some sell second hand repurposed servers on auction that they will set up for you. A while later you get an SSH login on the server and that's it, your server is running somewhere in a basement/bunker/old mine and you can go visit it if you want but in general you can do everything remote. There is even stuff that can let you see the bootup in bios from remote (Called KVM I believe). Some help you set up backups on the server and help you with setting up programs on the server but then it starts to get expensive. You can also just rent a space to place your own server but I haven't tried that.

rank0 5y ago
In your experience did you have to sign up with a partner ISP at the colo? Or is that done for me and just part of my colo bill? Is power use included as well?
- procombo 5y ago
  Colocation provider will bring the circuits to provide best-path connectivity based on packet destination. There shouldn't be an additional charge for this. They are incentivized to manage their bandwidth so data transfers fast, as they are likely charged wholesale for fiber availability. You will likely be charged 95th percentile mbps based on your usage. (Again, "pipe space required" to your needs.) Basically, whenever you're busiest -- 4pm-9pm are popular times for us in the USA. Some customers limit their bandwidth themselves (like, only allow max 12mbps file downloads, etc.) especially when they have the hardware to support huge bandwidth. Or your colocation provider can perhaps limit max connection to 100mbs or 1gbps if you want. Power is usually leased in amps. If you go over amps the circuit will break -- at worst case scenario. But typically they get in touch with you and tell you to upgrade. Also, they do want to know vaguely what your service is. Because you'll likely lease their IPs, they will question you if you do a lot of email (caution for spam), or run a Tor exit node (legal hassles for them in many cases).