27 ms·

Pretty bullish, anyone who has tried to setup and manage their own compute knows the pain they're solving. Plus I predict more companies will exit the cloud on

Pretty bullish, anyone who has tried to setup and manage their own compute knows the pain they're solving. Plus I predict more companies will exit the cloud once they realize how thick the margins have become or want better guarantees over sovereignty.

master_crab 1y ago
I don’t disagree that there are some fat margins in the cloud, but how is vendor lock-in any different here? Companies could end up paying fat margins to oxide too while still managing physical gear and plant.
- TZubiri 1y ago
  Well, their servers are mostly hypervisors, so the interface is mostly any virtual machine. You can "just" migrate by exporting or importing the vms.
  - yencabulator 1y ago
    You're literally ignoring that the Oxide management stack is very much custom and effectively vendor-locking the purchase to be maintained by them. They are not general purpose PC servers. You can "just" migrate away from Oxide but that would mean throwing away the hardware you now own. That's the grandparent's point; if you're migrating out of a cloud to avoid the margins demanded by the cloud vendor, now you're at the mercy of whatever Oxide thinks your support contract is worth. Sure, the convenience may be worth it, but watch how many companies are now struggling to get off of VMWare after Broadcom moves.
    - TZubiri 1y ago
      This reminds me of people complaining about github being closed source and moving to gitlab, or people obsessing over terraform to avoid cloud locking. Sure you will have vendor locking at the periferies, but the core is what's important, the guest vms. The hypervisor is whatever. If you have 100 vms running on ec2, you have done a great job of designing portable software, don't obsess over the last 1%.
turnsout 1y ago
I've run my own servers for over 20 years now, but I guess I don't understand the pain point. Can you elaborate? They write: > Our system delivers all the hardware and software you need to run cloud… To "run cloud?" Does this mean treating your own servers like "serverless?" Does it mean running Kubernetes? Is this primarily for people who want to self-host LLMs? I'm literally so old that I write programs that run on a server and never think about infrastructure.
- throw0101c 1y ago
  > To "run cloud?" Does this mean treating your own servers like "serverless?" Does it mean running Kubernetes? Is this primarily for people who want to self-host LLMs? Machine/system/service deployment via API. See §Essential Characteristics > On-demand self-service; Broad network access; Resource pooling; Rapid elasticity; Measured service * https://en.wikipedia.org/wiki/Cloud_computing https://en.wikipedia.org/wiki/Cloud_computing None are generally applicable to standalone pizza boxes as managed individual (as opposed to being herded by (e.g.) OpenStack).
- steveklabnik 1y ago
  > To "run cloud?" I agree that's a bit awkwardly phrased, let me send in a patch for that. > Does this mean treating your own servers like "serverless?" Does it mean running Kubernetes? Is this primarily for people who want to self-host LLMs? Not exactly any of that. We let you treat an entire rack as a single pool of resources, spinning up virtual machines that our control plane manages for you. Think "VPS provider but you own it." There's an API, but if you want to see what our console looks like, you can poke around with a demo here: https://console-preview.oxide.computer/ https://console-preview.oxide.computer/
  - turnsout 1y ago
    That's helpful, thanks!
- skybrian 1y ago
  No GPU’s yet, so it’s not good for LLM’s. But they have a lot of funding now, so perhaps that will change?
jillesvangurp 1y ago
I actually think that having more cloud providers might deflate a lot of the pricing. If you think about it, companies like Amazon buy server hardware and then rent it out by the vcpu (with throttling if they can get away with it) per month. Add memory and IO and you are looking at bills that pay for the server in mere months/weeks several tenants carving up all the hardware and each paying tens/hundreds per month. There are of course benefits to using cloud based VMs and I use them as well. But you are paying a very steep premium for what is a pitiful amount of compute and memory. There's a lot of wiggle room for price decreases and the only thing preventing that is a lack of competition. There's a reason Amazon is so rich: nobody seems to challenge them on AWS pricing. There's value in having them do all the faffing about with hardware of course. That's why companies use them. I'm in GCP; but same principle. I don't want to have to worry about replacing hard disks in the middle of the night, deal with network routers that are misbehaving, cooling issues, etc. That's why I pay them the big bucks. But I'm well aware that it's not that great of a deal. I used Hetzner a decade ago and paid something like 50 euros per month for a quad core xeon with a raid 1 disks, 32 GB, etc. Bare metal of course. But also, 50 euro. We had five of those. Forget about getting anything close to that with modern cloud providers for anything resembling a reasonable price. Your first monthly bill might actually add up to enough to buy your own hardware. Very tempting. They have beefed up their specs since then. You now get more for less. And they also do VMs now.
- throwup238 1y ago
  I knew it was bad but I didn’t realize just how bad the pricing spread can be until I started dealing with the GPU instances (8x A100 or H100 pods). Last I checked the on-demand pricing was $40/hr and the 1-year reserved instances were $25/hr. That’s over $200k/yr for the reserved instances so within two years I’d spend enough to buy my own 8x H100 pod (based on LambdaLabs pricing) plus enough to pay an engineer to babysit five pods at a time. It’s insane. With on-demand pricing the pod would pay for itself (and the cost to manage it) within a year.
  - wordpad 1y ago
    That's just hardware. If you need to build and maintain your own devops tooling it can balloon in complexity and cost real quick. It would still likely be much cheaper to do everything in house, but you would be assuming a lot of risks and locking yourself in losing flexibility. There is a reason people go with AWS over many competing cheaper cloud providers.
geodel 1y ago
Agree. But those fat margins can get starting to be shared with CIOs, CTOs and other managers with purchasing power. IMO the constant hectoring at workplace about migration to cloud or cloud native crap is not coming from some deep technical principles. It is more of Do it before you get fired for non-compliance
AndrewKemendo 1y ago
Having done web applications for oh…30 years now the key pain is network routing That’s the only thing I can tell is useful about “the cloud” I can build racks and servers easily, but the challenge is availability and getting past everyone’s firewalls So the real win is any service that allows for instant DNS table updates and availability of DNS whitelisting. This is why Google, Msft etc win in email because they have trusted endpoints Alternative routes with self signed DKIM etc is more or less blocked by default forcing you onto a provider We need more cloud flare tunnel and local hosting via commercial ISP routes and less new centralized data centers
- chrisweekly 1y ago
  +1 Insightful. I've also been doing web-related work for a living for over 25y and yours is the most spot-on take I've seen in this discussion.
  - AndrewKemendo 1y ago
    Thank you :)
- davinoishi 1y ago
  [dead]
bambax 1y ago
At the (very) low end it's pretty easy to build your own "cloud" with a NAS, containers, and reverse proxies and tunnels to the outside world. And this will get you suprisingly far. But at the high end, I think the market is litterally infinite. Every large company should want this, and want it now. Cloud providers are extremely expensive and, outside of the 1-tier where prices are really outrageous, they perform poorly and often offer little support. This really feels like the future.
- naikrovek 1y ago
  like us old assholes were saying when the cloud really started to take off: "this is nuts, it's just someone else's computer! and they're making a profit off of this service, meaning it's more expensive than what we were doing!" Now a lot of the things that were done pre-cloud were done in bad ways, and I'm not saying that we were right about those things. Having APIs for provisioning and monitoring are far better than submitting a request to some queue and having your VM provisioned manually 1 week later by someone who gets a key detail wrong. APIs and granular permissions are how this should be done, and "the cloud" taught everyone that very early. But a lot of companies are really stuck in the cloud mindset now, and won't let go of it. I think companies like Oxide and product lines like theirs are going to start becoming common. Microsoft, of course, completely fumbled the ball with Azure Stack, and I've never even heard of anyone deploying AWS Outpost, both for the same reason: the costs for these are absolutely insane for what they provide. What most folks really want is their own infrastructure running their own stuff using APIs that are either written in-house or provided by some vendor. Oxide is betting that they can sell you a working scalable system for less money than it would take to hire a team to write the APIs that would allow a company to do the same with off-the-shelf hardware. I think that they're probably right about that.
  - jeffbee 1y ago
    > they're making a profit off of this service, meaning it's more expensive than what we were doing! I hope you can see that this is a logical fallacy known as "zero-sum thinking". It is not only possible for a business to profit while lowering prices, it is universal throughout the economy. Tomato farmers make a profit selling tomatoes at a price much lower than the cost to grow tomatoes at home. Bakeries radically undercut the cost of home baking. It is obviously cheaper to buy motor fuel at the gas station than it would be to buy crude and refine it yourself. The main reason people think their on-prem is cheaper than cloud is that they are bad at accounting.

master_crab 1y ago

I don’t disagree that there are some fat margins in the cloud, but how is vendor lock-in any different here? Companies could end up paying fat margins to oxide too while still managing physical gear and plant.

TZubiri 1y ago
Well, their servers are mostly hypervisors, so the interface is mostly any virtual machine. You can "just" migrate by exporting or importing the vms.
- yencabulator 1y ago
  You're literally ignoring that the Oxide management stack is very much custom and effectively vendor-locking the purchase to be maintained by them. They are not general purpose PC servers. You can "just" migrate away from Oxide but that would mean throwing away the hardware you now own. That's the grandparent's point; if you're migrating out of a cloud to avoid the margins demanded by the cloud vendor, now you're at the mercy of whatever Oxide thinks your support contract is worth. Sure, the convenience may be worth it, but watch how many companies are now struggling to get off of VMWare after Broadcom moves.
  - TZubiri 1y ago
    This reminds me of people complaining about github being closed source and moving to gitlab, or people obsessing over terraform to avoid cloud locking. Sure you will have vendor locking at the periferies, but the core is what's important, the guest vms. The hypervisor is whatever. If you have 100 vms running on ec2, you have done a great job of designing portable software, don't obsess over the last 1%.

turnsout 1y ago

I've run my own servers for over 20 years now, but I guess I don't understand the pain point. Can you elaborate? They write: > Our system delivers all the hardware and software you need to run cloud… To "run cloud?" Does this mean treating your own servers like "serverless?" Does it mean running Kubernetes? Is this primarily for people who want to self-host LLMs? I'm literally so old that I write programs that run on a server and never think about infrastructure.

throw0101c 1y ago
> To "run cloud?" Does this mean treating your own servers like "serverless?" Does it mean running Kubernetes? Is this primarily for people who want to self-host LLMs? Machine/system/service deployment via API. See §Essential Characteristics > On-demand self-service; Broad network access; Resource pooling; Rapid elasticity; Measured service * https://en.wikipedia.org/wiki/Cloud_computing https://en.wikipedia.org/wiki/Cloud_computing None are generally applicable to standalone pizza boxes as managed individual (as opposed to being herded by (e.g.) OpenStack).
steveklabnik 1y ago
> To "run cloud?" I agree that's a bit awkwardly phrased, let me send in a patch for that. > Does this mean treating your own servers like "serverless?" Does it mean running Kubernetes? Is this primarily for people who want to self-host LLMs? Not exactly any of that. We let you treat an entire rack as a single pool of resources, spinning up virtual machines that our control plane manages for you. Think "VPS provider but you own it." There's an API, but if you want to see what our console looks like, you can poke around with a demo here: https://console-preview.oxide.computer/ https://console-preview.oxide.computer/
- turnsout 1y ago
  That's helpful, thanks!
skybrian 1y ago
No GPU’s yet, so it’s not good for LLM’s. But they have a lot of funding now, so perhaps that will change?

jillesvangurp 1y ago

I actually think that having more cloud providers might deflate a lot of the pricing. If you think about it, companies like Amazon buy server hardware and then rent it out by the vcpu (with throttling if they can get away with it) per month. Add memory and IO and you are looking at bills that pay for the server in mere months/weeks several tenants carving up all the hardware and each paying tens/hundreds per month. There are of course benefits to using cloud based VMs and I use them as well. But you are paying a very steep premium for what is a pitiful amount of compute and memory. There's a lot of wiggle room for price decreases and the only thing preventing that is a lack of competition. There's a reason Amazon is so rich: nobody seems to challenge them on AWS pricing. There's value in having them do all the faffing about with hardware of course. That's why companies use them. I'm in GCP; but same principle. I don't want to have to worry about replacing hard disks in the middle of the night, deal with network routers that are misbehaving, cooling issues, etc. That's why I pay them the big bucks. But I'm well aware that it's not that great of a deal. I used Hetzner a decade ago and paid something like 50 euros per month for a quad core xeon with a raid 1 disks, 32 GB, etc. Bare metal of course. But also, 50 euro. We had five of those. Forget about getting anything close to that with modern cloud providers for anything resembling a reasonable price. Your first monthly bill might actually add up to enough to buy your own hardware. Very tempting. They have beefed up their specs since then. You now get more for less. And they also do VMs now.

throwup238 1y ago
I knew it was bad but I didn’t realize just how bad the pricing spread can be until I started dealing with the GPU instances (8x A100 or H100 pods). Last I checked the on-demand pricing was $40/hr and the 1-year reserved instances were $25/hr. That’s over $200k/yr for the reserved instances so within two years I’d spend enough to buy my own 8x H100 pod (based on LambdaLabs pricing) plus enough to pay an engineer to babysit five pods at a time. It’s insane. With on-demand pricing the pod would pay for itself (and the cost to manage it) within a year.
- wordpad 1y ago
  That's just hardware. If you need to build and maintain your own devops tooling it can balloon in complexity and cost real quick. It would still likely be much cheaper to do everything in house, but you would be assuming a lot of risks and locking yourself in losing flexibility. There is a reason people go with AWS over many competing cheaper cloud providers.

geodel 1y ago

Agree. But those fat margins can get starting to be shared with CIOs, CTOs and other managers with purchasing power. IMO the constant hectoring at workplace about migration to cloud or cloud native crap is not coming from some deep technical principles. It is more of Do it before you get fired for non-compliance

AndrewKemendo 1y ago

Having done web applications for oh…30 years now the key pain is network routing That’s the only thing I can tell is useful about “the cloud” I can build racks and servers easily, but the challenge is availability and getting past everyone’s firewalls So the real win is any service that allows for instant DNS table updates and availability of DNS whitelisting. This is why Google, Msft etc win in email because they have trusted endpoints Alternative routes with self signed DKIM etc is more or less blocked by default forcing you onto a provider We need more cloud flare tunnel and local hosting via commercial ISP routes and less new centralized data centers

chrisweekly 1y ago
+1 Insightful. I've also been doing web-related work for a living for over 25y and yours is the most spot-on take I've seen in this discussion.
- AndrewKemendo 1y ago
  Thank you :)
davinoishi 1y ago
[dead]

bambax 1y ago

At the (very) low end it's pretty easy to build your own "cloud" with a NAS, containers, and reverse proxies and tunnels to the outside world. And this will get you suprisingly far. But at the high end, I think the market is litterally infinite. Every large company should want this, and want it now. Cloud providers are extremely expensive and, outside of the 1-tier where prices are really outrageous, they perform poorly and often offer little support. This really feels like the future.

naikrovek 1y ago
like us old assholes were saying when the cloud really started to take off: "this is nuts, it's just someone else's computer! and they're making a profit off of this service, meaning it's more expensive than what we were doing!" Now a lot of the things that were done pre-cloud were done in bad ways, and I'm not saying that we were right about those things. Having APIs for provisioning and monitoring are far better than submitting a request to some queue and having your VM provisioned manually 1 week later by someone who gets a key detail wrong. APIs and granular permissions are how this should be done, and "the cloud" taught everyone that very early. But a lot of companies are really stuck in the cloud mindset now, and won't let go of it. I think companies like Oxide and product lines like theirs are going to start becoming common. Microsoft, of course, completely fumbled the ball with Azure Stack, and I've never even heard of anyone deploying AWS Outpost, both for the same reason: the costs for these are absolutely insane for what they provide. What most folks really want is their own infrastructure running their own stuff using APIs that are either written in-house or provided by some vendor. Oxide is betting that they can sell you a working scalable system for less money than it would take to hire a team to write the APIs that would allow a company to do the same with off-the-shelf hardware. I think that they're probably right about that.
- jeffbee 1y ago
  > they're making a profit off of this service, meaning it's more expensive than what we were doing! I hope you can see that this is a logical fallacy known as "zero-sum thinking". It is not only possible for a business to profit while lowering prices, it is universal throughout the economy. Tomato farmers make a profit selling tomatoes at a price much lower than the cost to grow tomatoes at home. Bakeries radically undercut the cost of home baking. It is obviously cheaper to buy motor fuel at the gas station than it would be to buy crude and refine it yourself. The main reason people think their on-prem is cheaper than cloud is that they are bad at accounting.