8 ms·

Moore's Scofflaws

139 points by steveklabnik 3y ago

geerlingguy 3y ago
Per core licensing is pretty dumb, especially considering commodity systems now have 32, 64, or more cores, and server cores vary widely in capabilities. Per core maybe made a tiny bit of sense when a server was one or two cores. But that era's long gone.
- bee_rider 3y ago
  Also it is a bizarre unit to break things up over. Actually, how do per-core programs account for things like SMT? What about Bulldozer? Can I get a discount license if I only run it on an “efficiency” core? And we won’t even get into GPUs… What next? Let’s charge users based on how many instructions they have in flight for our application. These ILP cheaters have been mooching off the rest of us for ages!
  - ghaff 3y ago
    It made a lot more sense back in the day. Then a slot of things switched to per-socket.
  - wmf 3y ago
    Oracle and IBM have detailed pricing tables with dozens of cases like "one mainframe core is worth three Intel cores". But then you negotiate the price anyway so it seems kind of pointless.
- bombcar 3y ago
  Cloud, for better or worse, seems to be sold in vCPUs (read: cores), RAM (gigglebites), and disk. I wish there was a much better way to compare vCPUs than just "count" because one core of some five year old server is not the same as a core of a modern performance beast.
  - wmf 3y ago
    It's not like they charge the same price for a current gen vCPU as a five-year-old vCPU. The benefit of self-service is that you can benchmark it yourself.
- nyrikki 3y ago
  IMHO per socket was extractive most of the time, especially when memory capacity was the limit on performance. The 'Oracle doesn't have customers, it has hostages' The main purpose shifted from a hypothetical development cost offset to an effort to extract more fees from customers who were locked into your product. Broadcom's changes to VMware, where they have publicly stated that increasing revenue while not attempting to build new revenue streams or through any value adds for the customers is happening now. I get that any technology moves to an extraction phase after growth slows. But for me, moves like per core licenses are an indicator we need to consider vendor mitigation. Unfortunately capital markets seem to prefer these lower long term but predictable gains through extraction vs long term dividends etc...
andrewstuart 3y ago
The cloud is where Moores Law goes to die.
- CatWChainsaw 3y ago
  Clouds = heaven, it fits.
readthenotes1 3y ago
Isn't this just an ad?
- hello_computer 3y ago
  KVM is free. ZFS is free. Linux & BSD are free. The margins on server kit are tiny. If I've grown to the point that I need to buy my own hardware, the application software is probably something written in-house, so no licensing there either. Oxide gets plugged here non-stop, but the audience here still struggles to see what the point is. Sounds like a vanity project to keep some of the old Solaris talent in-pocket. edit: and the current growth-area is GPU/ML processing, where Oxide has bupkis.
  - away271828 3y ago
    One can simultaneously be impressed by the engineering while wondering if this level of custom implementation at what will probably be pretty low scale really makes sense.
    - hello_computer 3y ago
      Bezos had a clear proposition: "I can handle your workload better than you can for X cents per hour." Cantrill doesn't.
      - linksnapzz 3y ago
        Bezos's proposition needed to be qualified for each customer; in some cases, he was wrong. Yes, sometimes renting other people's hardware makes sense, for a time, until it doesn't.
        hello_computer 3y ago
        I wrote 'proposition', not 'fact'. Jeff made a simple proposal, then I, as the customer, can evaluate it, accept it, or reject it. 0xide doesn't have a simple proposal. They have a constellation of nitpicks and tastes, but that's not good enough. The typical consumer in this space wants quantifiables.
neom 3y ago
Bryan Cantrill is a million times smarter than me so I'm probably missing something. I thought people bought into the enterprise cloud ecosystems because of cost savings elsewhere? If you go into a platform like AWS/GCP/Azure shouldn't you be designing your applications to take advantage of the flexibility in the wide range of services offered? If you don't need that and you're not buying into all the abstractions they provide, you could use any other cloud-lite (Vultr, OVH, Linode etc)? Maybe I'm stuck in 2015 but are people really out here rolling their own clouds these days?
- bcantrill 3y ago
  Our big belief is that elastic infrastructure (that is, cloud computing) should be orthogonal to the economic model (that is, own versus rent). Today, that is not the case: if you want cloud computing, you more or less have to rent it. But we know that there are many who wish (or need) to run cloud computing but on an owned asset. There are many factors that deter that today, not least the odious per-core licensing that we highlight here. (For a particular brazen example of this, see VMware's infamous "AMD tax" ca. 2020.[0]) [0] https://news.vmware.com/company/cpu-pricing-model-update-feb-2020 https://news.vmware.com/company/cpu-pricing-model-update-feb...
  - ghaff 3y ago
    I actually find it interesting how many things we (which, in part, I mean I) got wrong about cloud computing early on. Various other things too.
  - dsizzle 3y ago
    Why aren't there prices on the website? (Seems weird that there isn't given that blog post, where costs are pretty central.)
    - steveklabnik 3y ago
      (not Bryan, not actively part of the decision to put prices on the website or not, just my own take.) Different audiences have different expectations. In the place we are, that is, enterprise sales, the expectation is not that you can go to a website, get a price, and click a button. The expectation is that the two organizations will communicate over a period of time ("the sales cycle") to work through everything that goes into a deal, and eventually come to an agreement or not. This includes so many variables that putting the price on the website wouldn't make sense, as you're never going to end up at that exact price. I used to find this attitude frustrating, as an engineer, but the longer I have been a professional, and worked at various organizations, the more I come around to that being a good thing, not a bad thing.
ChuckMcM 3y ago
I really like 0xide's insight here. I don't know if Bryan has an MBA but this article reminds me of how I learned that good systems engineers make for good product people. A friend of mine from Sun went to Santa Clara university for their MBA and I was always interested in running a company and we talked about what core knowledge the school felt that a "Master" of business administration should know. Not too surprisingly there was a lot about how you could organize a bunch of unreliable elements (employees) into a structure that reliably delivered results (products). And in one sense that is exactly what cloud computing did as well, organizing unreliable "cheap" PC type servers, into a fabric of reliable service delivery. It is the system of organizing the parts that is the solution. Greg Lindahl was one of the founders of Blekko and his experience of managing hundreds of machines for the super computer types with big machine budgets and low IT budgets, was essential to Blekko's ability to deliver its search engine. With a DevOps group of six (one manager and five SREs) Blekko managed 2000+ machines in two data centers. IBM was astonished when they bought us that we could manage with so few devops engineers. We had also done the "What would this cost us on AWS" calculation many times with many different variables and the 'break even' point, where the cost to self host at a Colocation facility fell below the cost to host with AWS was 120 servers. And that included reducing the DevOps group size to 3. So about 10 racks worth of servers. There was a really funny experience when IBM insisted we move our infrastructure to SoftLayer (an IBM business) when IBM acquired us and we pointed out that the "soft money" cost of running our infrastructure on SoftLayer was about $9M a month versus $120K a month. Which their finance group shut down that talk and just renewed the lease in the Colo. But to successfully run a big distributed thing like that, you needed to both put your servers in a colocation facility and have Greg's software which allowed you to manage them with a small team. The total cost of that infrastructure depended on it. I got a chance to go visit the 0xide team and they absolutely "got" that requirement. If I were running Engineering and Operations in Blekko today we totally would have kicked Supermicro to the curb and replaced them with the 0xide solution. I don't know if they have teamed up with Antithesis for testing their stuff but man, that combo of management software that never breaks and integrated server cabinets with all the things? That's a pretty good integration.
- mixmastamyk 3y ago
  What was the secret sauce?
frankfrank13 3y ago
I am constantly impressed by Oxide. Their writing is top notch, I think they're exceptional engineers and product thinkers, and clearly they care about the ecosystem they inhabit. One day I would love to get all of their OSS up and running locally. Truly, why not try to run your own private cloud? How many old laptops, desktops, PIs, etc must we all have lying around?
- taneq 3y ago
  > why not try to run your own private cloud? Because until you're big enough to have local weather systems evolving inside your offices, that's called 'servers'. I'm not saying not to do it, it's great, and cost effective, and much safer, but calling a small-to-mid business's servers 'cloud' is like saying I have a 'private Uber' in my garage that I can drive myself.
  - metaxy2 3y ago
    Right or wrong, I think the notion of "the cloud" that Oxide is selling is the "elastic" part--the ability to request resources by quantity instead of having to think about the exact servers responsible for serving your request. So physical servers become an implementation detail rather than a developer concern. That's a concept that can apply to how you provision two on-site servers (or even one, really) as much as a whole multi-data-center cloud computing service.
    - angarg12 3y ago
      People have been doing that for 10+ years solid, even more with rudimentary middleware such as grid computing.
      - metaxy2 3y ago
        I don't think Oxide is trying to say no one's done this before, more that they're the best and most comprehensive offering to buy an onsite setup for this fully set up from a third party.
daxaxelrod 3y ago
Tangent but "Your margin is my opportunity" was originally a Sam Walton quote when he was building the 5 and dime stores. Bezos was deeply inspired by early Walmart and Sam. https://www.acquired.fm/episodes/walmart https://www.acquired.fm/episodes/walmart
- bcantrill 3y ago
  That's interesting -- the quote is very much attributed to Bezos. Did the quote actually come from Sam Walton, or merely the sentiment? Either way, it's clear that the Waltons were an inspiration to Bezos!
  - ghaff 3y ago
    Quote Investigator comes down pretty hard on the Bezos side and doesn't mention Walton. https://quoteinvestigator.com/2019/01/13/margin/ https://quoteinvestigator.com/2019/01/13/margin/
zbrozek 3y ago
Case is inverted with javascript disabled. I've never seen that before!
- dcre 3y ago
  I can't reproduce in Firefox on macOS. Tell us more about your setup!
  - zbrozek 3y ago
    For that visit, Windows 11 with Firefox developer. Ublock Origin and Noscript, with some combination of previously allowed and default-blocked script sources. I didn't really investigate what was missing to mess with the case, but allowing all scripts on the page cleared it. Just a weird novelty impact that I hadn't seen before, not really a big deal. Also I retract "invert". I went back and looked again and it's definitely not inverted. Check it out! https://zlnp.net/serve/weird-case.png https://zlnp.net/serve/weird-case.png
    - dcre 3y ago
      Ooh, very cool. Looks like you're overriding the font at the browser or OS level? We suspect that has something to do with it. If you look at the raw HTML response, which is what should be getting rendered when you have scripts turned off, you can see that the casing is normal. The mystery to me is why turning scripts back on would fix it. Maybe that overrides your font override and changes it back to our font.
      - zbrozek 3y ago
        I'm not aware of any font overrides (that's not something I typically fuss with) but now you have me suspicious that I don't know my own setup!
mattbee 3y ago
It's a great value product, and I would definitely be excited if I was still filling my own data centre with VMs. But you can't buy integrated like this without software support. So when 0xide have the (nice) problem of topping out whatever market share they can win, their shareholders will want to extract more revenue from existing customers. That can only come in the form of licensing updates (or selling new sleds) according to _how_ they're used: by core, by power, AI model, number of customers, or whatever else. Either 0xide are such a roaring success that Amazon reduce their cloud prices to compete. Or they save their select customers so much money on the cloud that they can start raising the rent. (not a negative judgment by the way, customers win either way. Just putting this kind of marketing in context and a reminder that you don't own anything any more, haha)
- nxobject 3y ago
  You're right: their shareholders (especially the less imaginative ones) could pressure them to take a page out of IBM's playbook – the parallels between an 0xide cabinet and an IBM big iron cabinet hosting multiple VMs/LPARs is too tempting. Or, on a positive note, they could offer customers the option to invest in the "go the extra step" customer service engineering that IBM offers.
pclmulqdq 3y ago
Thank you for correctly citing Wright's law, but I will say that Moore's law is very much over-cited. Moore spoke only on number of transistors per chip, not anything about transistors per dollar or transistors per unit area.
vajrabum 3y ago
Looking at Oxide, aren't hyperconverged computing solutions a product category? Looks like Oxide's yearly revenue is on the order of $15M/year with about $45M of Series VC cash. So these folks might be smart but they've not shipped very much product yet. At scale (1-10K servers) I'm not sure why I would tell my CTO or CIO to buy from them instead of a larger, more established vendor? Definitely agree that you can easily pay too much for cloud services and that the hyperscalers have every incentive to put as much margin in their pockets as they can get away with.
- steveklabnik 3y ago
  I wrote a bit about "hyperconverged" vs "hyperscale" and how we're the latter here https://news.ycombinator.com/item?id=30688865 https://news.ycombinator.com/item?id=30688865 > So these folks might be smart but they've not shipped very much product yet. We just started shipping late last year, announcing our first two customers then: https://oxide.computer/blog/the-cloud-computer https://oxide.computer/blog/the-cloud-computer So yes, still early :)
- roughly 3y ago
  I don’t know Oxide’s solutions personally, but the fact that the economics have always favored self-hosting and yet AWS is a gazillion dollar business suggests that while they may be building in a pre-existing product category, nobody’s actually gotten the product right yet.
deleted 3y ago
[deleted]
nromiun 3y ago
I know AWS has a reputation for being expensive overall. But it is crazy that it has the cheapest VMs. See here: https://cloudoptimizer.io/ https://cloudoptimizer.io/ But of course other things like egress, serverless services more than make up for it.
pkz 3y ago
This was a great blog post highlighting some of the weird business models that plague cloud computing. I tried to find competitors to suppliers like Oxide in the EU where there should be a huge amount of business cases for "datacenter in a box". Surprisingly there are very few options available (yes, Azure Stack is available but there is still licensing per cpu for that). Why aren't there more players in this area?
- wmf 3y ago
  VxRack, Nutanix, Outposts, Azure Stack as you said, probably something from Oracle Cloud...
- bcantrill 3y ago
  We have definitely had the same question! I think the answer is twofold: it's technically hard -- and it's cross-domain in that you need both hardware AND software to pull it off. On the one hand, it's not like such expertise doesn't exist (and we are not doing our own silicon here!), but on the other, even with the right team, it is time-consuming (and therefore expensive). If the difficulty and cost were the only challenge, this would be a candidate to do at a larger company, but the cross-domain nature of it makes it really thorny: you would need a lot of internal alignment to succeed -- and (more challenging) you need to maintain that internal alignment for a protracted period of time. I had done something not wholly dissimilar (though frankly much less ambitious) at Sun back in the day[0] -- and even though Sun was much more amenable to this kind of disruptive endeavor than any company of its size[1], we barely pulled it off. Indeed, some of the worst behavior I ever saw at Sun was from the people who trying to prevent us from succeeding because they felt it threatened them. I simply cannot imagine doing something more ambitious at Sun -- or as ambitious anywhere else. We came to the conclusion that it really has to be new company formation -- which means raising money to do it, which means finding the right investors. Even though the upside is extraordinary, finding the right investors in hard tech is really, really tough[2]. So yes, there should be more options -- but there aren't, and there are frankly unlikely to be in the foreseeable future... [0] https://bcantrill.dtrace.org/2008/11/10/fishworks-now-it-can-be-told/ https://bcantrill.dtrace.org/2008/11/10/fishworks-now-it-can... [1] https://bcantrill.dtrace.org/2011/07/12/in-defense-of-intrapreneurialism/ https://bcantrill.dtrace.org/2011/07/12/in-defense-of-intrap... [2] https://oxide-and-friends.transistor.fm/episodes/deep-tech-investing https://oxide-and-friends.transistor.fm/episodes/deep-tech-i...
mad_vill 3y ago
I am really curious if part of the reason why oxide is leaning into racksize is that joyent has scared them from attempting a 1U/2U solution.

139 points by steveklabnik 3y ago

Per core licensing is pretty dumb, especially considering commodity systems now have 32, 64, or more cores, and server cores vary widely in capabilities. Per core maybe made a tiny bit of sense when a server was one or two cores. But that era's long gone.

bee_rider 3y ago
Also it is a bizarre unit to break things up over. Actually, how do per-core programs account for things like SMT? What about Bulldozer? Can I get a discount license if I only run it on an “efficiency” core? And we won’t even get into GPUs… What next? Let’s charge users based on how many instructions they have in flight for our application. These ILP cheaters have been mooching off the rest of us for ages!
- ghaff 3y ago
  It made a lot more sense back in the day. Then a slot of things switched to per-socket.
- wmf 3y ago
  Oracle and IBM have detailed pricing tables with dozens of cases like "one mainframe core is worth three Intel cores". But then you negotiate the price anyway so it seems kind of pointless.
bombcar 3y ago
Cloud, for better or worse, seems to be sold in vCPUs (read: cores), RAM (gigglebites), and disk. I wish there was a much better way to compare vCPUs than just "count" because one core of some five year old server is not the same as a core of a modern performance beast.
- wmf 3y ago
  It's not like they charge the same price for a current gen vCPU as a five-year-old vCPU. The benefit of self-service is that you can benchmark it yourself.
nyrikki 3y ago
IMHO per socket was extractive most of the time, especially when memory capacity was the limit on performance. The 'Oracle doesn't have customers, it has hostages' The main purpose shifted from a hypothetical development cost offset to an effort to extract more fees from customers who were locked into your product. Broadcom's changes to VMware, where they have publicly stated that increasing revenue while not attempting to build new revenue streams or through any value adds for the customers is happening now. I get that any technology moves to an extraction phase after growth slows. But for me, moves like per core licenses are an indicator we need to consider vendor mitigation. Unfortunately capital markets seem to prefer these lower long term but predictable gains through extraction vs long term dividends etc...

andrewstuart 3y ago

The cloud is where Moores Law goes to die.

CatWChainsaw 3y ago
Clouds = heaven, it fits.

readthenotes1 3y ago

Isn't this just an ad?

hello_computer 3y ago
KVM is free. ZFS is free. Linux & BSD are free. The margins on server kit are tiny. If I've grown to the point that I need to buy my own hardware, the application software is probably something written in-house, so no licensing there either. Oxide gets plugged here non-stop, but the audience here still struggles to see what the point is. Sounds like a vanity project to keep some of the old Solaris talent in-pocket. edit: and the current growth-area is GPU/ML processing, where Oxide has bupkis.
- away271828 3y ago
  One can simultaneously be impressed by the engineering while wondering if this level of custom implementation at what will probably be pretty low scale really makes sense.
  - hello_computer 3y ago
    Bezos had a clear proposition: "I can handle your workload better than you can for X cents per hour." Cantrill doesn't.
    - linksnapzz 3y ago
      Bezos's proposition needed to be qualified for each customer; in some cases, he was wrong. Yes, sometimes renting other people's hardware makes sense, for a time, until it doesn't.
      - hello_computer 3y ago
        I wrote 'proposition', not 'fact'. Jeff made a simple proposal, then I, as the customer, can evaluate it, accept it, or reject it. 0xide doesn't have a simple proposal. They have a constellation of nitpicks and tastes, but that's not good enough. The typical consumer in this space wants quantifiables.

neom 3y ago

Bryan Cantrill is a million times smarter than me so I'm probably missing something. I thought people bought into the enterprise cloud ecosystems because of cost savings elsewhere? If you go into a platform like AWS/GCP/Azure shouldn't you be designing your applications to take advantage of the flexibility in the wide range of services offered? If you don't need that and you're not buying into all the abstractions they provide, you could use any other cloud-lite (Vultr, OVH, Linode etc)? Maybe I'm stuck in 2015 but are people really out here rolling their own clouds these days?

bcantrill 3y ago
Our big belief is that elastic infrastructure (that is, cloud computing) should be orthogonal to the economic model (that is, own versus rent). Today, that is not the case: if you want cloud computing, you more or less have to rent it. But we know that there are many who wish (or need) to run cloud computing but on an owned asset. There are many factors that deter that today, not least the odious per-core licensing that we highlight here. (For a particular brazen example of this, see VMware's infamous "AMD tax" ca. 2020.[0]) [0] https://news.vmware.com/company/cpu-pricing-model-update-feb-2020 https://news.vmware.com/company/cpu-pricing-model-update-feb...
- ghaff 3y ago
  I actually find it interesting how many things we (which, in part, I mean I) got wrong about cloud computing early on. Various other things too.
- dsizzle 3y ago
  Why aren't there prices on the website? (Seems weird that there isn't given that blog post, where costs are pretty central.)
  - steveklabnik 3y ago
    (not Bryan, not actively part of the decision to put prices on the website or not, just my own take.) Different audiences have different expectations. In the place we are, that is, enterprise sales, the expectation is not that you can go to a website, get a price, and click a button. The expectation is that the two organizations will communicate over a period of time ("the sales cycle") to work through everything that goes into a deal, and eventually come to an agreement or not. This includes so many variables that putting the price on the website wouldn't make sense, as you're never going to end up at that exact price. I used to find this attitude frustrating, as an engineer, but the longer I have been a professional, and worked at various organizations, the more I come around to that being a good thing, not a bad thing.

ChuckMcM 3y ago

I really like 0xide's insight here. I don't know if Bryan has an MBA but this article reminds me of how I learned that good systems engineers make for good product people. A friend of mine from Sun went to Santa Clara university for their MBA and I was always interested in running a company and we talked about what core knowledge the school felt that a "Master" of business administration should know. Not too surprisingly there was a lot about how you could organize a bunch of unreliable elements (employees) into a structure that reliably delivered results (products). And in one sense that is exactly what cloud computing did as well, organizing unreliable "cheap" PC type servers, into a fabric of reliable service delivery. It is the system of organizing the parts that is the solution. Greg Lindahl was one of the founders of Blekko and his experience of managing hundreds of machines for the super computer types with big machine budgets and low IT budgets, was essential to Blekko's ability to deliver its search engine. With a DevOps group of six (one manager and five SREs) Blekko managed 2000+ machines in two data centers. IBM was astonished when they bought us that we could manage with so few devops engineers. We had also done the "What would this cost us on AWS" calculation many times with many different variables and the 'break even' point, where the cost to self host at a Colocation facility fell below the cost to host with AWS was 120 servers. And that included reducing the DevOps group size to 3. So about 10 racks worth of servers. There was a really funny experience when IBM insisted we move our infrastructure to SoftLayer (an IBM business) when IBM acquired us and we pointed out that the "soft money" cost of running our infrastructure on SoftLayer was about $9M a month versus $120K a month. Which their finance group shut down that talk and just renewed the lease in the Colo. But to successfully run a big distributed thing like that, you needed to both put your servers in a colocation facility and have Greg's software which allowed you to manage them with a small team. The total cost of that infrastructure depended on it. I got a chance to go visit the 0xide team and they absolutely "got" that requirement. If I were running Engineering and Operations in Blekko today we totally would have kicked Supermicro to the curb and replaced them with the 0xide solution. I don't know if they have teamed up with Antithesis for testing their stuff but man, that combo of management software that never breaks and integrated server cabinets with all the things? That's a pretty good integration.

mixmastamyk 3y ago
What was the secret sauce?

frankfrank13 3y ago

I am constantly impressed by Oxide. Their writing is top notch, I think they're exceptional engineers and product thinkers, and clearly they care about the ecosystem they inhabit. One day I would love to get all of their OSS up and running locally. Truly, why not try to run your own private cloud? How many old laptops, desktops, PIs, etc must we all have lying around?

taneq 3y ago
> why not try to run your own private cloud? Because until you're big enough to have local weather systems evolving inside your offices, that's called 'servers'. I'm not saying not to do it, it's great, and cost effective, and much safer, but calling a small-to-mid business's servers 'cloud' is like saying I have a 'private Uber' in my garage that I can drive myself.
- metaxy2 3y ago
  Right or wrong, I think the notion of "the cloud" that Oxide is selling is the "elastic" part--the ability to request resources by quantity instead of having to think about the exact servers responsible for serving your request. So physical servers become an implementation detail rather than a developer concern. That's a concept that can apply to how you provision two on-site servers (or even one, really) as much as a whole multi-data-center cloud computing service.
  - angarg12 3y ago
    People have been doing that for 10+ years solid, even more with rudimentary middleware such as grid computing.
    - metaxy2 3y ago
      I don't think Oxide is trying to say no one's done this before, more that they're the best and most comprehensive offering to buy an onsite setup for this fully set up from a third party.

daxaxelrod 3y ago

Tangent but "Your margin is my opportunity" was originally a Sam Walton quote when he was building the 5 and dime stores. Bezos was deeply inspired by early Walmart and Sam. https://www.acquired.fm/episodes/walmart https://www.acquired.fm/episodes/walmart

bcantrill 3y ago
That's interesting -- the quote is very much attributed to Bezos. Did the quote actually come from Sam Walton, or merely the sentiment? Either way, it's clear that the Waltons were an inspiration to Bezos!
- ghaff 3y ago
  Quote Investigator comes down pretty hard on the Bezos side and doesn't mention Walton. https://quoteinvestigator.com/2019/01/13/margin/ https://quoteinvestigator.com/2019/01/13/margin/

zbrozek 3y ago

Case is inverted with javascript disabled. I've never seen that before!

dcre 3y ago
I can't reproduce in Firefox on macOS. Tell us more about your setup!
- zbrozek 3y ago
  For that visit, Windows 11 with Firefox developer. Ublock Origin and Noscript, with some combination of previously allowed and default-blocked script sources. I didn't really investigate what was missing to mess with the case, but allowing all scripts on the page cleared it. Just a weird novelty impact that I hadn't seen before, not really a big deal. Also I retract "invert". I went back and looked again and it's definitely not inverted. Check it out! https://zlnp.net/serve/weird-case.png https://zlnp.net/serve/weird-case.png
  - dcre 3y ago
    Ooh, very cool. Looks like you're overriding the font at the browser or OS level? We suspect that has something to do with it. If you look at the raw HTML response, which is what should be getting rendered when you have scripts turned off, you can see that the casing is normal. The mystery to me is why turning scripts back on would fix it. Maybe that overrides your font override and changes it back to our font.
    - zbrozek 3y ago
      I'm not aware of any font overrides (that's not something I typically fuss with) but now you have me suspicious that I don't know my own setup!

mattbee 3y ago

It's a great value product, and I would definitely be excited if I was still filling my own data centre with VMs. But you can't buy integrated like this without software support. So when 0xide have the (nice) problem of topping out whatever market share they can win, their shareholders will want to extract more revenue from existing customers. That can only come in the form of licensing updates (or selling new sleds) according to _how_ they're used: by core, by power, AI model, number of customers, or whatever else. Either 0xide are such a roaring success that Amazon reduce their cloud prices to compete. Or they save their select customers so much money on the cloud that they can start raising the rent. (not a negative judgment by the way, customers win either way. Just putting this kind of marketing in context and a reminder that you don't own anything any more, haha)

nxobject 3y ago
You're right: their shareholders (especially the less imaginative ones) could pressure them to take a page out of IBM's playbook – the parallels between an 0xide cabinet and an IBM big iron cabinet hosting multiple VMs/LPARs is too tempting. Or, on a positive note, they could offer customers the option to invest in the "go the extra step" customer service engineering that IBM offers.

pclmulqdq 3y ago

Thank you for correctly citing Wright's law, but I will say that Moore's law is very much over-cited. Moore spoke only on number of transistors per chip, not anything about transistors per dollar or transistors per unit area.

vajrabum 3y ago

Looking at Oxide, aren't hyperconverged computing solutions a product category? Looks like Oxide's yearly revenue is on the order of $15M/year with about $45M of Series VC cash. So these folks might be smart but they've not shipped very much product yet. At scale (1-10K servers) I'm not sure why I would tell my CTO or CIO to buy from them instead of a larger, more established vendor? Definitely agree that you can easily pay too much for cloud services and that the hyperscalers have every incentive to put as much margin in their pockets as they can get away with.

steveklabnik 3y ago
I wrote a bit about "hyperconverged" vs "hyperscale" and how we're the latter here https://news.ycombinator.com/item?id=30688865 https://news.ycombinator.com/item?id=30688865 > So these folks might be smart but they've not shipped very much product yet. We just started shipping late last year, announcing our first two customers then: https://oxide.computer/blog/the-cloud-computer https://oxide.computer/blog/the-cloud-computer So yes, still early :)
roughly 3y ago
I don’t know Oxide’s solutions personally, but the fact that the economics have always favored self-hosting and yet AWS is a gazillion dollar business suggests that while they may be building in a pre-existing product category, nobody’s actually gotten the product right yet.

deleted 3y ago

[deleted]

nromiun 3y ago

I know AWS has a reputation for being expensive overall. But it is crazy that it has the cheapest VMs. See here: https://cloudoptimizer.io/ https://cloudoptimizer.io/ But of course other things like egress, serverless services more than make up for it.

pkz 3y ago

This was a great blog post highlighting some of the weird business models that plague cloud computing. I tried to find competitors to suppliers like Oxide in the EU where there should be a huge amount of business cases for "datacenter in a box". Surprisingly there are very few options available (yes, Azure Stack is available but there is still licensing per cpu for that). Why aren't there more players in this area?

wmf 3y ago
VxRack, Nutanix, Outposts, Azure Stack as you said, probably something from Oracle Cloud...
bcantrill 3y ago
We have definitely had the same question! I think the answer is twofold: it's technically hard -- and it's cross-domain in that you need both hardware AND software to pull it off. On the one hand, it's not like such expertise doesn't exist (and we are not doing our own silicon here!), but on the other, even with the right team, it is time-consuming (and therefore expensive). If the difficulty and cost were the only challenge, this would be a candidate to do at a larger company, but the cross-domain nature of it makes it really thorny: you would need a lot of internal alignment to succeed -- and (more challenging) you need to maintain that internal alignment for a protracted period of time. I had done something not wholly dissimilar (though frankly much less ambitious) at Sun back in the day[0] -- and even though Sun was much more amenable to this kind of disruptive endeavor than any company of its size[1], we barely pulled it off. Indeed, some of the worst behavior I ever saw at Sun was from the people who trying to prevent us from succeeding because they felt it threatened them. I simply cannot imagine doing something more ambitious at Sun -- or as ambitious anywhere else. We came to the conclusion that it really has to be new company formation -- which means raising money to do it, which means finding the right investors. Even though the upside is extraordinary, finding the right investors in hard tech is really, really tough[2]. So yes, there should be more options -- but there aren't, and there are frankly unlikely to be in the foreseeable future... [0] https://bcantrill.dtrace.org/2008/11/10/fishworks-now-it-can-be-told/ https://bcantrill.dtrace.org/2008/11/10/fishworks-now-it-can... [1] https://bcantrill.dtrace.org/2011/07/12/in-defense-of-intrapreneurialism/ https://bcantrill.dtrace.org/2011/07/12/in-defense-of-intrap... [2] https://oxide-and-friends.transistor.fm/episodes/deep-tech-investing https://oxide-and-friends.transistor.fm/episodes/deep-tech-i...

mad_vill 3y ago

I am really curious if part of the reason why oxide is leaning into racksize is that joyent has scared them from attempting a 1U/2U solution.