4 ms·

Dedicated hosting providers. I'm so amazed that somehow people completely forget that for literally decades, web host provided dedicated hosting options at fan

Dedicated hosting providers. I'm so amazed that somehow people completely forget that for literally decades, web host provided dedicated hosting options at fantastic prices. Yes, loooong time ago - to get your dedicated server might have taken a few hours to provision and the instant server access that AWS brought should not be discredited. But large numbers of web host today allow you to programmatically spin up a dedicated web host instantaneously and at a fraction of the cost.

that_guy_iain 4y ago
> Yes, loooong time ago - to get your dedicated server might have taken a few hours to provision and the instant server access that AWS brought should not be discredited. At one point it took often days for a dedicated server to be set up. We also didn't have such nice provisioning tools. Now it just seems like cargo cult to use cloud providers as the only option. People just completely discount dedicated servers.
- frognumber 4y ago
  In most businesses I've worked in: developer costs >> infrastructure costs An AWS large server is around $500/year, which is about 1-2 developer hours (with taxes, overhead, etc) at the cost scales last time I priced this out. That's crazy expensive in the absolute, but if it saves a couple of hours, it makes sense. PaaS providers cost even more. I've gone with those in the past, since it basically eliminated dev-ops. The major downside wasn't cost, so much as flexibility. Dedicated servers start to make sense for: - The very low end (e.g. personal use, or hosting something long-running for a small business) - The very high end (e.g. once cloud costs start to hit hundred of thousands of dollars per year) On the very high end, cloud providers will often cut a deal, though. My problem with AWS, recently, has been reliability. Servers crash or have performance degradation a bit too often. That leads to developer costs, and might be what pushes me back to dedicated.
  - isoprophlex 4y ago
    1-2 developer hours?! Anyone looking to hire an ML guy for 250-500$/hr flat fee, hourly rate, no taxes, no healthcare, cancel any time... get in touch! For that money I'll gladly bark like a dog, walk on all fours and fetch your slippers.
    - frognumber 4y ago
      You're confusing how much money you take home from how much money you cost the organization. You should see how much lawyers cost versus how much lawyers earn. The sticker price for SWEs isn't as well-advertised, but it's how businesses work.
    - elephantum 4y ago
      I have to mention, that what you described is not what ML guy is supposed to be doing.
      - simfree 4y ago
        Well, it's either that or tuning/training/trimming a model to fit your intended use case...
      - yjftsjthsd-h 4y ago
        Surely they mean to get training data for the robot;)
    - deleted 4y ago
      [deleted]
    - dehrmann 4y ago
      That number would actually be the opportunity cost of developer time.
    - Cwizard 4y ago
      To add a datapoint, a company I worked at budgeted developer time at ~700euro/day. However almost no one made more than 300€/day. The rest came from office space, your manager’s salary, laptop, extra benefits, heating, consultants to tell your managers what you already know, idk what else. In the US salaries are a lot higher so in that context I can see 300-500$/day be realistic. Really surprised me… Always made me wonder if all that money was being spent well
goodpoint 4y ago
Not to mention having your own datacenter, which is even cheaper (unless you do something stupid). EDIT: to the naysayers: I worked in various companies (from tiny to huge) maintaining their own DCs and had access to financial data. And I was involved in the setting up new DCs.
- nithril 4y ago
  Might not be so obvious when summing all the cost (apex, opex). Not mentioning the difficulty to find experienced people to maintain it. Unless you are really big of course
- MajimasEyepatch 4y ago
  Having your own data center is almost never cheaper nowadays unless you have very specific hardware needs. And it is far, far, far slower to build your own data center than to just start using a cloud provider.
kaptainscarlet 4y ago
Hosting providers are a pain. My server was down for full day because the provider was being DDoSed. AWS has DDoS protection by default. If I was on AWS, my server would not go down due to network DDoS attacks. That is just one of many things AWS does for you that many hosting providers don't.
- mwcampbell 4y ago
  There are also affordable dedicated server providers with DDoS protection, like OVHcloud.
- doublerabbit 4y ago
  My hosting provider does and successfully too. Maybe it's time to move providers, I would.
throwaway858 4y ago
The one big thing missing from dedicated hosts is an S3 equivalent. Sure, you can get a huge hard drive for cheap but this will not have the durability requirements for storing your precious data. And if you try to use AWS just for S3 then you will pay a lot extra for the bandwidth charges of bringing the data from S3 to your server (something that is free if you were to use EC2 or other AWS services).
- alberth 4y ago
  Lots of web host also have S3 compatible equivalent offerings. They typically market it as "object storage".
- pwinnski 4y ago
  That same logic works up and down the line. If what you need is disk storage, then you are limited to dedicated hosts that provide something akin to S3. There are some! But that's also true when you want a database that isn't sqlite. Now you need a dedicated host that provides something akin to DynamoDB, and will manage it for you. Then you decide you need queueing, and you can either install and manage that yourself or look for a dedicate host that provides something akin to SNS/SQS. And so on...
- theptip 4y ago
  Right -- that's the one implementation detail in the OP that was interesting. It sounds like they ultimately used MinIO to replace S3. I've seen people use Ceph, but it's apparently a nightmare to operate a Ceph cluster. If you're on k8s I think the "cloud native" way might be Rook, haven't looked into that. Anyway, running an object store is painful. Their notes here are a bit vague: > When the migration reached mid-June, we had 300 servers running very smoothly with a total 200 million cached pages. We used Apache Cassandra nodes on each of the servers that were compatible with AWS S3. > We broke the online migration into four steps, each a week or two apart. After testing whether Prerender pages could be cached in both S3 and minio, we slowly diverted traffic away from AWS S3 and towards minio. When the writes to S3 had been stopped completely, Prerender saved $200 a day on S3 API costs and signaled we were ready to start deleting data already cached in our Cassandra cluster. > However, the big reveal came at the end of this phase around June 24th. In the last four weeks, we moved most of the cache workload from AWS S3 to our own Cassandra cluster. The daily cost of AWS was reduced to $1.1K per day, projecting to 35K per month, and the new servers’ monthly recurring cost was estimated to be around 14K. It says (briefly, in passing) that they used Cassandra to implement the S3 API for their nodes, but maybe just to replicate the S3 API that they were previously using? That's an interesting choice I'd not heard of before. Perhaps all of their individual files are quite small? Then they moved to MinIO, which would be the S3 equivalent that you are looking for.

that_guy_iain 4y ago

> Yes, loooong time ago - to get your dedicated server might have taken a few hours to provision and the instant server access that AWS brought should not be discredited. At one point it took often days for a dedicated server to be set up. We also didn't have such nice provisioning tools. Now it just seems like cargo cult to use cloud providers as the only option. People just completely discount dedicated servers.

frognumber 4y ago
In most businesses I've worked in: developer costs >> infrastructure costs An AWS large server is around $500/year, which is about 1-2 developer hours (with taxes, overhead, etc) at the cost scales last time I priced this out. That's crazy expensive in the absolute, but if it saves a couple of hours, it makes sense. PaaS providers cost even more. I've gone with those in the past, since it basically eliminated dev-ops. The major downside wasn't cost, so much as flexibility. Dedicated servers start to make sense for: - The very low end (e.g. personal use, or hosting something long-running for a small business) - The very high end (e.g. once cloud costs start to hit hundred of thousands of dollars per year) On the very high end, cloud providers will often cut a deal, though. My problem with AWS, recently, has been reliability. Servers crash or have performance degradation a bit too often. That leads to developer costs, and might be what pushes me back to dedicated.
- isoprophlex 4y ago
  1-2 developer hours?! Anyone looking to hire an ML guy for 250-500$/hr flat fee, hourly rate, no taxes, no healthcare, cancel any time... get in touch! For that money I'll gladly bark like a dog, walk on all fours and fetch your slippers.
  - frognumber 4y ago
    You're confusing how much money you take home from how much money you cost the organization. You should see how much lawyers cost versus how much lawyers earn. The sticker price for SWEs isn't as well-advertised, but it's how businesses work.
  - elephantum 4y ago
    I have to mention, that what you described is not what ML guy is supposed to be doing.
    - simfree 4y ago
      Well, it's either that or tuning/training/trimming a model to fit your intended use case...
    - yjftsjthsd-h 4y ago
      Surely they mean to get training data for the robot;)
  - deleted 4y ago
    [deleted]
  - dehrmann 4y ago
    That number would actually be the opportunity cost of developer time.
  - Cwizard 4y ago
    To add a datapoint, a company I worked at budgeted developer time at ~700euro/day. However almost no one made more than 300€/day. The rest came from office space, your manager’s salary, laptop, extra benefits, heating, consultants to tell your managers what you already know, idk what else. In the US salaries are a lot higher so in that context I can see 300-500$/day be realistic. Really surprised me… Always made me wonder if all that money was being spent well

goodpoint 4y ago

Not to mention having your own datacenter, which is even cheaper (unless you do something stupid). EDIT: to the naysayers: I worked in various companies (from tiny to huge) maintaining their own DCs and had access to financial data. And I was involved in the setting up new DCs.

nithril 4y ago
Might not be so obvious when summing all the cost (apex, opex). Not mentioning the difficulty to find experienced people to maintain it. Unless you are really big of course
MajimasEyepatch 4y ago
Having your own data center is almost never cheaper nowadays unless you have very specific hardware needs. And it is far, far, far slower to build your own data center than to just start using a cloud provider.

kaptainscarlet 4y ago

Hosting providers are a pain. My server was down for full day because the provider was being DDoSed. AWS has DDoS protection by default. If I was on AWS, my server would not go down due to network DDoS attacks. That is just one of many things AWS does for you that many hosting providers don't.

mwcampbell 4y ago
There are also affordable dedicated server providers with DDoS protection, like OVHcloud.
doublerabbit 4y ago
My hosting provider does and successfully too. Maybe it's time to move providers, I would.

throwaway858 4y ago

The one big thing missing from dedicated hosts is an S3 equivalent. Sure, you can get a huge hard drive for cheap but this will not have the durability requirements for storing your precious data. And if you try to use AWS just for S3 then you will pay a lot extra for the bandwidth charges of bringing the data from S3 to your server (something that is free if you were to use EC2 or other AWS services).

alberth 4y ago
Lots of web host also have S3 compatible equivalent offerings. They typically market it as "object storage".
pwinnski 4y ago
That same logic works up and down the line. If what you need is disk storage, then you are limited to dedicated hosts that provide something akin to S3. There are some! But that's also true when you want a database that isn't sqlite. Now you need a dedicated host that provides something akin to DynamoDB, and will manage it for you. Then you decide you need queueing, and you can either install and manage that yourself or look for a dedicate host that provides something akin to SNS/SQS. And so on...
theptip 4y ago
Right -- that's the one implementation detail in the OP that was interesting. It sounds like they ultimately used MinIO to replace S3. I've seen people use Ceph, but it's apparently a nightmare to operate a Ceph cluster. If you're on k8s I think the "cloud native" way might be Rook, haven't looked into that. Anyway, running an object store is painful. Their notes here are a bit vague: > When the migration reached mid-June, we had 300 servers running very smoothly with a total 200 million cached pages. We used Apache Cassandra nodes on each of the servers that were compatible with AWS S3. > We broke the online migration into four steps, each a week or two apart. After testing whether Prerender pages could be cached in both S3 and minio, we slowly diverted traffic away from AWS S3 and towards minio. When the writes to S3 had been stopped completely, Prerender saved $200 a day on S3 API costs and signaled we were ready to start deleting data already cached in our Cassandra cluster. > However, the big reveal came at the end of this phase around June 24th. In the last four weeks, we moved most of the cache workload from AWS S3 to our own Cassandra cluster. The daily cost of AWS was reduced to $1.1K per day, projecting to 35K per month, and the new servers’ monthly recurring cost was estimated to be around 14K. It says (briefly, in passing) that they used Cassandra to implement the S3 API for their nodes, but maybe just to replicate the S3 API that they were previously using? That's an interesting choice I'd not heard of before. Perhaps all of their individual files are quite small? Then they moved to MinIO, which would be the S3 equivalent that you are looking for.