19 ms·

Servers as they should be – shipping early 2022

876 points by ykl 5y ago

warrenm 5y ago
wish I was a good fit for the company!
zapita 5y ago
Who is the target customer for this?
- caeril 5y ago
  Cantrill has always said it's for people who want Facebook class on-premise infrastructure but don't have a $900B market cap and a hundred engineers designing and building custom boxes. Oh, and open firmware.
  - jeffbee 5y ago
    Can definitely see it for a company size of Dropbox, big enough to already be working with ODMs, big enough to be sensitive to the kind of headaches you get from a heterogeneous fleet of ILOM processors designed by deranged engineers.
    - qbasic_forever 5y ago
      Dropbox is big enough that they could just acquire Oxide now before they even get to market. That might even be the plan all along. I can't imagine there are more than like a dozen companies that are their target market, i.e. big enough to need Facebook-level datacenters but not big enough (yet) to have that engineering team.
    - yjftsjthsd-h 5y ago
      > ILOM processors designed by deranged engineers. As opposed to what? But seriously, why do all ILOMs suck and are there any exceptions?
      - jeffbee 5y ago
        The people behind OpenBMC, RunBMC, LibreBMC are hoping to fix this.
      - daddylongstroke 5y ago
        They don't all suck. I always liked how Sun/Oracle and Fujitsu's did their LOMs.
- lumost 5y ago
  This level of integration surely won't come cheap, from what I recall of Server purchasing a target price of ~200-500k per rack would be expected on a TCO of roughly 2x the rack price over 3 years. (Assuming you are buying from Quanta/SuperMicro or other commodity integrator) It's possible the prices are different now, but you would need customers looking to drop > 1 million dollars in CapEx for the management capabilities they are providing. Possibly non-cloud Fortune-500?
  - riking 5y ago
    The core customer profile appears to be aging startups that know how their software behaves and are currently way over 2k cores on AWS.
    - benlivengood 5y ago
      Startups with no significant use of anything from AWS other than EC2+EBS with an existing well-tested storage migration procedure.
  - dcolkitt 5y ago
    That reminds me a lot of the Sun Microsystem mega servers from 20 years ago. Those were kind of the cure-all solutions to high scalability web services before Google et al. pioneered cloud-like services on commodity software.
    - nickik 5y ago
      Its kind of like if you took those servers and put a cloud like software layer on top and made many of them work together.
  - kraig 5y ago
    They'd have to sell these at a significant loss to make up for the risk any company would have to take to build out a DC on first generation hardware from a startup.
    - 5y ago
- atonse 5y ago
  I can see this also being appealing for large orgs with sensitive data they don't want to put in a public cloud.
- convolvatron 5y ago
  good question. not someone too large to want to pay the per-node margins. not someone too small to want to pay .. not someone who is satisfied with using VMs on a cloud provider. not someone who is selling these as part of a turnkey solution for whatever segment is left. that said, I do feel persistently sad that we can't fix structural problems because the market is such a gradient descent world.
- rapsey 5y ago
  Plenty of companies live outside the cloud on their own hardware.
  - daddylongstroke 5y ago
    The vast majority of companies live on their own hardware for business-critical workloads. It's not even close.
- salmo 5y ago
  I get the target of HyperConverged infrastructure. It's a pretty big market: potentially all private/colo datacenters. And there are only a few players left. Dell/EMC/VMWare, Nutanix, Cisco's schizophrenic offerings, a waning HP, cloud providers trying to make a 'hybrid' play, etc. And most don't buy one or two of these things. It's rows and rows. But most of those are so entrenched and wrapped up in their customers. I imagine the target here is actually acquisition, it would just be too hard to get a foothold as an up-and-comer. Also it usually means giving loaner gear to companies for an extended period for them to evaluate pre-purchase, showing your support, etc. That's a lot of up-front cost for someone without a warchest. I'm also kind of surprised by the site. It sells to geeks well, but isn't the normal "look at our customers in key industry segments!", "something something Gartner magic quadrant", "whitepapers!" thing. Selling to execs on these things is usually a matter of convincing them they're not making a "bad" decision. They're "cool", but enough industry people agree with them that it's not career limiting if it doesn't pan out. I like the idea of the product, and it would be nice to have another player. But it's like starting a new car company, and I feel like they're selling to mechanics.
fsociety 5y ago
I kind of want one of these in my home, I wonder what the price range will be.
- gwbrooks 5y ago
  I'm adding one of these to the if-I-win-the-lottery list. Nifty home media server AND it'll probably keep the house warm during the winter.
- softfalcon 5y ago
  “2048 x86 cores per rack” Probably a lot more than any sane person would pay for a home server. But I won’t stop you from trying! Wouldn’t it be cool to have that plugged into your local network?
  - bayindirh 5y ago
    I'd get one. Would connect hot side to home HVAC and just run scalability tests of my code. That'd be hot, nice and expensive. And possibly noisy. Yes, noisy.
- ptomato 5y ago
  for what they're advertising there, on the order of $1mm, based on underlying hardware costs.
  - kaliszad 5y ago
    Actually, standard Azure rack is on the order of $1.1 mm of hardware depending on SKU if I am not mistaken. So I would guess, it could be more like $2 mm. There is the aspect of management and other vendors Dell/EMC + VMware like you to pay way more than the hardware costs for e.g. VXRail/ vSphere licences. That is the real target.
    - salmo 5y ago
      And maintenance. I like the idea of having to give them a key to your house to replace a disk when the SMART alert phones home. :)
discardable_dan 5y ago
I don't know if I have a use for something like this, but the website aesthetics are just plain awesome.
- tyingq 5y ago
  Yeah, I noticed that too. The green wireframe looking stuff is actually text in spans/divs next to, or overlayed on pictures. The little "nodes" are this character, for example: ⎕. The effect is pretty unique.
  - goodpoint 5y ago
    Looks like simple ascii art, typically used by CLI tools.
- megous 5y ago
  I like them too but just viewing the page on Firefox loads 5950X to 250%. A bit too much.
  - mcdevilkiller 5y ago
    Wow. It loads almost instantly and doesn't slow down on my phone (LG G7). Must be something else?
    - megous 5y ago
      You have to scroll all the way through the page to activate all the gimmicks. Then it never stops, and loads 2.5 cores permanently to 100%, which makes CPU fan spin to the max.
      - kfajdsl 5y ago
        Oh my god, that's hilarious. I was wondering why my lap was warm all of a sudden. Htop said firefox was the culprit so I closed out all my tabs except this one. Then I read your comment, opened the page, and scrolled all the way to the bottom—my cpu temp just steadily rose till it throttled. All the animations are smooth, though. Note: I'm typing this from a 9 year old thin-and-light, so that's probably part of the problem.
atonse 5y ago
This looks interesting (although I'm not in the target market, too small)... But if I were looking at this, judging from the quality of people they've amassed in their engineering team, is there any chance they won't be acquired in 6 months? To anyone looking to take a bet on this, what is the answer to "what's your plan for when your stellar team gets acquired?" And what answer will satisfy that buyer? Update: Adding another question, does this "environment" (where any really great product with great talent in it can be acquired very quickly) have a chilling effect on purchases for products like this? Hopefully some Oxide people can answer :-)
- bcantrill 5y ago
  Hi! So, at every step -- from conception to funding to building the team and now building the product -- we have done so to build a big, successful public company. Not only do we (the founders) share that conviction, but it is shared by our investors and employees as well. For better or for ill, we are -- as we memorably declared to one investor -- ride or die. Also, if it's of any solace, I really don't think any of the existing players would be terribly interested in buying a company that has so thoroughly and unequivocally rejected so many of their accrued decisions! ;) I'm pretty sure their due diligence would reveal that we have taken a first principles approach here that is anathema to the iterative one they have taken for decades -- and indeed, these companies have shown time and time again that they don't want to risk their existing product lines to a fresh approach, no matter how badly customers want it.
  - tw04 5y ago
    Isn't that what some of your old Sun compatriots thought with DSSD? :) Congrats on the announcement, here's hoping you're right! This looks too interesting to be swallowed by Oracle or HPe.
    - bcantrill 5y ago
      Fortunately, some of those same DSSD folks have joined us at Oxide -- and let's just say that they are of like mind with respect to Oxide's approach. ;)
newaccount2021 5y ago
Bravo! Better servers for people who want to own their infra. Too many people seek out cloud services just to get a modern control plane. Server "UI" has been long neglected. And finally, its nice to see people with brains building real things with nary a mention of "blockchain".
transfire 5y ago
Estimated price?
- wmf 5y ago
  Cheaper than AWS Outposts I guess. (Cheaper per core, not necessarily per rack.)
- kondro 5y ago
  Guessing at $400 - $1000 per core.
diwu1989 5y ago
Looks like a white labeled viking VDS2249R running custom software.
- ChrisMarshallNY 5y ago
  This is not my area of expertise, but it does look like that[0]. That "custom software," though, is where the magic often lies. As a software person that worked at hardware companies for most of my career, I know all too well, how disrespectful hardware people are of software. If they have a good software-respecting management chain, then it might be pretty awesome. [0] https://www.prnewswire.com/news-releases/viking-enterprise-solutions-introduces-new-nvme-over-fabrics--enterprise-storage-solution-for-high-performance-applications-301297571.html https://www.prnewswire.com/news-releases/viking-enterprise-s...
  - selectodude 5y ago
    Well if Oxide's software stack is going to be open source, I guess we'll get a good look into their secret sauce.
  - dralley 5y ago
    Let's be honest, software people are pretty disrespectful about hardware, too.
    - ChrisMarshallNY 5y ago
      Not me. I started as an EE. Wanna check out my first professional engineering project ever? https://littlegreenviper.com/TF30194/TF30194-Manual-1987.pdf https://littlegreenviper.com/TF30194/TF30194-Manual-1987.pdf (Downloads a PDF)
loudmax 5y ago
As I understand it, Oxide is going to have deep software integration into their hardware. So the expectation isn't that the servers in this rack will be running Windows or a generic Linux distribution. In case anyone from Oxide is here, is my understanding correct? And if so, will there be a way to run a smaller version of an Oxide system, say for testing or development, without purchasing an entire rack at a time? Anyway, glad to finally get a glimpse of what Oxide has to offer. Looking forward to seeing a lot more.
- 2trill2spill 5y ago
  My understanding is you will use a API to provision virtual machines on top of the Oxide hypervisor/software stack, which is bhyve running on Illumos. So you can still just run your favorite Linux distro or windows or a BSD if you want[1]. [1]: https://soundcloud.com/user-760920229/why-your-servers-suck-and-how-oxide-computer-plans-to-make-this-better https://soundcloud.com/user-760920229/why-your-servers-suck-...
  - SSLy 5y ago
    bhyve? what happened to the KVM port?
    - 2trill2spill 5y ago
      Apparently bhyve will be replacing KVM going forward, this article has a list of reasons[1]. [1]: https://omnios.org/info/bhyve https://omnios.org/info/bhyve
- jhickok 5y ago
  Agreed, I would love to hear more about the management plane. I'm glad it's API-driven, but I still have some questions about things like which hypervisor they are using. If it's a custom software stack, might be nice to get a miniature dev-kit!
  - deleted 5y ago
    [deleted]
stefan_ 5y ago
Why are we hard coupling the hardware to the software? The whole secret of the success of M1 and ARM in servers is that lots of software has long ago stopped being hyper-aware of what hardware it is running on. What software are we talking about anyways? It's all incredibly vague, but it seems to reach all the way into the Kubernetes sphere. Why would I run this over something I can use on my next job?
- monocasa 5y ago
  It's probs cheaper than AWS if you already have on prem infra. AWS has pretty damn good margins. And the idea of "these racks are my kubernetes cluster and are supported by the OEM as such" has a lot of value to a lot of the medium sized IT departments I've run across. Can you expand on what you mean on "coupling the hardware to the software"?
- alanbernstein 5y ago
  Is Apple really a good counterexample for the success of integrating software and hardware?
- tediousdemise 5y ago
  > Why are we hard coupling the hardware to the software? The whole secret of the success of M1 and ARM in servers is that lots of software has long ago stopped being hyper-aware of what hardware it is running on. The software running on M1 is a bespoke fit for it. That's why the performance in macOS on M1 is phenomenal. It was custom made to execute optimally on it.
- rcxdude 5y ago
  It's basically a mini-ec2 in your server room. The software and hardware give you a platform to deploy VMs configured however you want.
boulos 5y ago
First, congrats! But second, I'd love to understand the compute vs storage tradeoff chosen here. Looking at the (pretty!) picture [1], I was shocked to see "Wow, it's mostly storage?". Is that from going all flash? Heading to https://oxide.computer/product https://oxide.computer/product for more details, lists: - 2048 cores - 30 TB of memory - 1024 TB of flash (1 PiB) Given how much of the rack is storage, I'm not sure which Milan was chosen (and so whether that's 2048 threads or 4196 [edit: real cores, 4196 threads]), but it seems like visually 4U is compute? [edit: nope] Is that a mistake on my part, because dual-socket Milan at 128 threads per socket is 256 threads per server, so you need at least 8 servers to hit 2048 "somethings", or do the storage nodes also have Milans [would make sense] and their compute is included [also fine!] -- and so similarly that's how you get a funky 30 TiB of memory? [Top-level edit from below: the green stuff are the nodes, including the compute. The 4U near the middle is the fiber] P.S.: the "NETWORK SPEED 100 GB/S" in all caps / CSS loses the presumably 100 Gbps (though the value in the HTML is 100 gb/s which is also unclear). [1] https://oxide.computer/_next/image?url=%2Fimages%2Frenders%2Foxide-rack-front.png&w=2048&q=75 https://oxide.computer/_next/image?url=%2Fimages%2Frenders%2...
- ptomato 5y ago
  It looks like they're doing 2u half width nodes, so I'd strongly suspect each node is 1TB of ram, one epyc 7713p, and 10 3.2TB u.2/u.3 drives. eta: also suspect 30TB total just means they're leaving 64GB ram for the hypervisor OS on each node.
  - kaliszad 5y ago
    Leaving that RAM for ZFS L2 ARC perhaps? I don't think they would use Illumos as the hypervisor OS without also using OpenZFS with it. They also need some for management, the control UI, a DB for metrics and more. Btw. if I count correctly, they have 20 SSD slots per node (if a node is full width) and 16 nodes. They would need 2 TB to reach 1 PB of "raw" capacity with the obvious redundancy overhead of ~ 20%. It is also quite possible, they don't use ZFS at all and use e.g. Ceph or something like it but I don't think that is the case, because that wouldn't be cantrillian. :-) E.g. using Minio, they can provide something S3 like on top of a cluster of ZFS storage nodes too but they most likely get better latency with local ZFS and not a distributed filesystem. Financial institutions especially seem to be part of the target here and there latency can be king.
hlandau 5y ago
"Attests the software version is the version that is valid and shipped by the Oxide Computer Company" So in other words these servers will implement restrictive code signing practices and will be vendor-controlled, not owner-controlled? This is not my idea of "secure", and really in the wake of things like the Solarwinds or RSA hacks it shouldn't be anyone's idea of secure. Vendor-holds-the-keys is not an acceptable security model. A comment below mentions open firmware, open firmware is useless without the right to deploy modified versions of it. Happy to take clarification on this.
- gruez 5y ago
  "Attest" refers to https://en.wikipedia.org/wiki/Trusted_Computing#REMOTE-ATTESTATION https://en.wikipedia.org/wiki/Trusted_Computing#REMOTE-ATTES..., not forced code signing
  - hlandau 5y ago
    I'm familiar with the concept. Does this mean that attestation to a different root of trust than Oxide will also be feasible, and that this is just a default?
    - hobofan 5y ago
      It's an "enterprise" product, so you can be certain that some amount of extra money will buy you that capability (if it's not already included).
    - gruez 5y ago
      Oxide makes the hardware, so it makes sense to use them as the root of trust since you already have to trust them to not make backdoored hardware. Why bother adding more parties? Also, for remote attestation to make sense, it needs to be done in the hardware itself (ie. keys burned into the silicon). I'm not sure how that's supposed to work if you add your own keys, or whether that would even make sense.
      - 5y ago
deleted 5y ago
[deleted]
renewiltord 5y ago
I'm an idiot. I thought this was like the SGI UV300 where you'd view the whole thing as a single computer and everything would be NUMA'd away. It looks like it's not like that, though.
rbanffy 5y ago
Looks interesting. I wonder if the high integration makes it behave as a single-image machine with that many cores.
- wmf 5y ago
  No, single system image is really expensive (see Superdome Flex) and most workloads can't justify that cost.
  - zozbot234 5y ago
    Could one reimplement SSI at the OS layer, similar to existing distributed OS's? Distributed shared memory is usually dismissed because of the overhead involved in typical scenarios, but this kind of hardware platform might make it feasible.
    - wmf 5y ago
      There was Mosix at the OS layer in the 1990s and Virtual Iron at the hypervisor layer in the aughts. I think the cost and performance of software SSI just doesn't intersect with demand anywhere.
      - e12e 5y ago
        Plan9 on an oxide rack might get you close to an SSI work-a-like - but I think it's unlikely it'd be a practical use of the hw?
        zozbot234 5y ago
        AIUI, Plan9 is not quite fully SSI. It is a distributed OS, and gets quite close, but it's missing support for distributed memory (i.e. software-implemented shared memory exposing a single global address space); it also does not do process checkpointing and auto-migration, without which you don't really have a "global" system image.
psanford 5y ago
Congrats Oxide team! More competition in this space is always a good thing. I'm curious about management. Can the rack operate completely standalone? I assume when you have multiple there will be some management abstraction above the rack layer? The closest direct equivalent that I can think of to this is AWS outposts. Are there any others that I'm forgetting?
- wmf 5y ago
  There are a bunch of enterprisey "private cloud" aka "converged infrastructure" racks like VxRack, Nutanix, etc.
- _delirium 5y ago
  The density they're getting here is significantly higher than AWS Outposts, which is interesting. The top-end (~$600k) AWS Outposts seem to max out at around 1k CPUs and 4.5 TB RAM in a rack (e.g. 12x m5.24xlarge = 12x 384 GB), while this rack can house 2k CPUs and 30 TB (!) RAM.
  - psanford 5y ago
    Outposts seem like a solution to the problem, "for regulatory or compliance reasons we are required for data to reside and be processed within a physical space we control." For that problem, an organization that is otherwise on AWS might find Outposts to be appealing. I can imagine an engineering team's response to such a requirement as "Oh yeah? Fine, its but its going to cost you $600k per year per rack!". I believe Oxide is attempting to capture a much broader market than that.
  - ptomato 5y ago
    it's better than that, actually, as an AWS vCPU is a hyperthread, not a full core, and these would have 4096 hyperthreads.
  - kaliszad 5y ago
    Yes, well it isn't that dense either. As I have written, it's 32 CPUs (16x 2 CPUs). 1 TB of RAM per CPU is not that huge a deal, it's perhaps 16x 64 GB (Milan uses 8 channels, 2 DIMMs per channel is reasonable), if you consider that is 16 GB of RAM per core. In HPC, you would probably shrink it to 1/4 of the volume (half width, 1 U dual socket server). Oxide probably focuses on optimal thermal efficiency since their limit isn't the space so much as the power density/ max. power per rack in existing DCs, which they are already pushing hard. (Of course they have lower power options too but they probably will not use 2048 cores.)
alberth 5y ago
Who is the target buyer? Is this to compete against Nutanix / VCE?
- fuzzylightbulb 5y ago
  I think it would more play in the space of a NetApp/Cisco FlexPod or VCE's Vblock, but what those customers are really purchasing is the certified validation of core enterprise apps on a particular hardware/software stack, as well as the massive engineering and support organizations that those companies can bring to bear to validate firmware upgrade and to swoop in in the event of an issue. You also seem to get a LOT more flexibility. I am not a hater in the least but I really am failing to understand what is unique about this offering. It seems like you have no options regarding the internals, and so scaling compute separately from storage doesn't seem possible. I also am very suspect about offerings like this that have not yet released a second version of their key parts. Everyone says that they are going to be backwards compatible, but then the reality of managing heterogenous generations of gear in a homogenous fashion strikes and you get weird behavior all around. Long story short, I would love to know what a customer of this scale of physical infrastructure is getting with Oxide that they would not be better served by going to one of the major vendors.
  - Tuna-Fish 5y ago
    Firmware that does not suck? Because that's something the current "major vendors" really are irredeemably terrible at.
- etcet 5y ago
  We use Nutanix where I work and this has made everyone very excited. Though they would need something similar to Nutanix CE to make us switch entirely (i.e. the ability to run non-production unsupported on commodity hardware).
kaliszad 5y ago
"Only" 2048 CPU cores per rack is actually not that much by nowadays standards - its 16 U of 2x 64 core CPUs. Perhaps is could be more U if they used the lower core counts but e.g. higher frequency per core SKUs but I don't think they do. (And the picture kind of confirms it). They use 2U servers though so they are able to use lower speed but bigger fans and have more expansion cards and 2,5" form factor drives perhaps. The of course have to fit storage, which needs lots of CPU PCIe lanes for all the NVMe storage and networking (probably 2 or 4 U) and power conversion to power the bus bar and more somewhere. They probably use the 42 U+ standard 19" racks to fit in standard customers DCs. They also don't have such a high power budget as custom DCs for cloud providers do. 1 PB of flash is quite a bit but you could get perhaps 5x as much with HDDs probably (even with a relatively low density of 40x 12 x 12 TB). The problem really is I think, they wouldn't be able to write the HDD firmware in Rust in time (or at all, because no HDD manufacturer would sell an HDD to them without making sure their proprietary firmware is used). SSDs don't necessarily have this property as they are much more like the other components of a modern server.
mindfulplay 5y ago
Congrats to Oxide computer! Excited to see tech startups do actual tech instead of chasing VC funded growth hacking. I wonder what sort of enterprise customers this targets.. (definitely not for individual devs)
rcarmo 5y ago
Well, congrats then! I've been waiting for news (and listening to the "On The Metal" podcast) for a long while now, and this seems like a great way to push the envelope on server hardware. (plus I suspect there will be more to come...)
kaliszad 5y ago
As for the website, some of the animations could use a spring like movement profile to feel more physical. The website also isn't reachable over IPv6, so I would be very careful with the promised IPv6 capabilities of the server too ;-)
_jal 5y ago
It sure looks pretty, but appears to be - - dedicated to virtualization, done their way - rather inflexible in hardware specs - vendor-locked at the rack - if you have hardware from someone else, it can't live in the same cabinet I guess if you just want a pretty data center in a box and look like what they consider a 'normal' enterprise to be, it might appeal. But I'm not sure how many people asked for Apple-style hardware in the DC.
- wmf 5y ago
  A lot of customers are asking for private cloud.
- zozbot234 5y ago
  > - dedicated to virtualization, done their way > - rather inflexible in hardware specs > - vendor-locked at the rack - if you have hardware from someone else, it can't live in the same cabinet This describes legacy IBM platforms quite well. If they can leverage hyperscaling tech to be better and cheaper than what IBM is currently offering, that's enough to make it worthwhile.
  - yesitdoes 5y ago
    It kind of does. One main difference with IBM is the three letters I, B and M.
- rapsey 5y ago
  Why is it important what kind of virtualization? It works and since it is built for this hardware it will likely be more reliable then anything you're putting together yourself. The specs are damn good. When it is all top-of-the-line, inflexibility is kind of a mute point. Where else are you going to go? > But I'm not sure how many people asked for Apple-style hardware in the DC. Well integrated, performant and reliable hardware that runs VMs where you can put anything on it is pretty much all everyone running their own hardware is looking for. Honestly I am surprised how many here completely misunderstand what their value proposition is.
  - 5y ago
MangoCoffee 5y ago
>AMD MILAN Intel is losing on the client and server. everyone is jumping ship to either ARM or AMD for client/server. hopefully Intel new engineer CEO can turn it around like AMD engineer CEO (Lisa Su)
- dsr_ 5y ago
  Few companies buy on pure performance. Right now, AMD has the performance kings and the price/performance kings. Intel could win price/performance, but they would need to cannibalize their own low-end and mid-range market. If they could make a good bet that they would have high yield in one more cycle, that would make sense. If they don't think that will happen, there's nothing much that will save them, and they're extracting the money that they can right now.
  - StreamBright 5y ago
    I thought M1 had the best price/performance numbers, too bad Apple does not sell CPUs.
    - dsr_ 5y ago
      Or, more relevant to this discussion, servers.
      - zozbot234 5y ago
        They sell rack-mountable hardware (Mac Pros) running a certified Unix OS. It's only a matter of time until those are based on Apple Silicon too.
        floatboth 5y ago
        Rack Mac Pros are targeted at racks that are mostly filled with audio/video equipment. Apple really doesn't seem to have any interest in selling server products again.
throwkeep 5y ago
This looks great. Really well thought out, beautifully designed and presented.
Sanguinaire 5y ago
Looks fantastic, and the hardware specs appeal to me greatly - but I'm not sure there is an actual market outside the "cult of personality" bubble. A few SV wannabes will buy into this to trade off a Twitter relationship with the Oxide founders - but does anyone really see the IT teams at Daimler, Proctor & Gamble, Morgan Stanley... et al - actually going for this over HPE/Dell and AWS/Azure? We are a long way away from "Nobody ever got fired for buying from Oxide".
- tyingq 5y ago
  You wouldn't have to pitch it initially as a replacement for your on-prem HPE/Dell. It could be pitched as a replacement for the hosted private cloud you have from IBM, Oracle, etc, that you're unhappy with.
- nickik 5y ago
  Like everything else, start small, and grow. If you have a good product that actually works.
ebeip90 5y ago
I wonder if they used Monodraw[1] to create their diagrams? Looks like they did! [1]: https://monodraw.helftone.com https://monodraw.helftone.com
- thundergolfer 5y ago
  Can Monodraw produce the animation as well, as seen on the webpage?
- jessfraz 5y ago
  YES we did :)
agentdrtran 5y ago
I would love a mini version of this for homelabs.
- wmf 5y ago
  If the software stack ends up being open source someone could make a name for themself by porting it to run on Linux + random hardware.
  - yjftsjthsd-h 5y ago
    If someone manages to port bhyve to Linux, they will definitely make a name for themselves. But honestly, the equivalent is just libvirt on commodity hardware with openzfs storage; the value here is high end hardware with custom firmware and well-integrated software, not really something you can port usefully.
    - wmf 5y ago
      I'm more interested in the control plane but maybe I'm in the minority.
jmartrican 5y ago
Why the elevation constraint? "The elevation of the room where the rack is installed must be below 10,005 feet (3,050 meters)."
- ptomato 5y ago
  probably air density cooling solution is qualified for.
- mikey_p 5y ago
  Perhaps a constraint on cooling effectiveness due to air density?
- dralley 5y ago
  At 10,000 the air pressure is 1/3 less than at sea level, less density means less capacity for carrying heat, so the cooling might not be sufficient.
  - notacoward 5y ago
    Bingo. I've personally had to deal with this in other high-density systems. Less cooling not only has the obvious effects, but also reduces PS efficiency which can cause other problems. Cosmic-ray-induced memory errors can also be a problem at those altitudes (or even half that). That's a bit easier to deal with in principle, but the rate of ECC scrubbing required can start to impact performance. Stack that on top of thermal CPU throttling, and you'll have a system that's just slower than it should be. Just as importantly, the slowdown will be uneven across components, so it's effectively a different kind of system when you're debugging low-level issues. I think it's a good sign that they're aware of the additional support issues associated with higher altitude. Shows that they've really thought things through.
todd8 5y ago
This looks great. From a business perspective, I would be concerned that it would be hard to prevent companies like Dell from entering this space as a competitor quite rapidly.
- wmf 5y ago
  Enterprise vendors are culturally incapable of building opinionated hardware or software. That's a moat, but which side are the customers on?
  - AlphaSite 5y ago
    Isn’t this basically: https://www.vmware.com/products/vmc-on-dell-emc.html https://www.vmware.com/products/vmc-on-dell-emc.html Or https://www.hpe.com/us/en/greenlake.html https://www.hpe.com/us/en/greenlake.html Disclaimer vmware employee, but I dont work in this area.
    - mlindner 5y ago
      Sure, but you don't own those. You're renting the hardware from HP.
ksec 5y ago
May be instead of asking target market or audience, who are their competitors? (Edit: Previous Discussions https://news.ycombinator.com/item?id=21682360 https://news.ycombinator.com/item?id=21682360 ) Also thinking if the Website is not finished? All the "Read More" actually hide very little information, if so why hide it? And doesn't seems to explain the company very well. Seems like we need to listen to their PodCast to find out what is going on. ( Edit: Found a Youtube Video about it https://www.youtube.com/watch?v=vvZA9n3e5pc https://www.youtube.com/watch?v=vvZA9n3e5pc ) >Get the most efficient power rating and network speeds of 100GBps without the pain of cable 100GBps would be impressive, 100Gbps would be ... not much? A interesting thing is all the terminal like graphics are actually HTML/CSS and not images.
- hujun 5y ago
  >Get the most efficient power rating and network speeds of 100GBps without the pain of cable 100GBps would be impressive, 100Gbps would be ... not much? Even 100GBps(800Gbps) is not much for 2048 cores, depends on application, in certain applications, 32 cores could drive 100Gbps...
- daddylongstroke 5y ago
  > ...who are their competitors? Literally every single server vendor, and almost all (if not all?) storage vendors on the planet are pushing HCI because that is what mid and large size companies want. This is the fastest growing market segment in hardware (because they now realize that hybrid-cloud is the preferred customer model, and most of their customers now are deploying or already have deployed their own internal cloud). Oxide appears to me to be HCI done correctly. I currently work for one of their competitors, and for one, am keeping at eye on their careers page! Also, 100Gb meets requirements for 99.999% of the customers out there.
tw04 5y ago
The storage is always the difficult part in these architectures. Are you distributing across all nodes? It appears that each sled is an individual compute unit with 10 drives. Are the drives on a proverbial island and only accessible to that local node, or is there some distributed storage going on that you can talk about? On paper with RDMA and NVMe-OF you could access any drive from any compute unit... but that's easier said than done :)
gjvc 5y ago
0xide Computer Company deserves to do well. This is a solid private-cloud play aimed at those corporations (probably mainly financials, but other sectors too I'm sure...) who don't want to outsource to the likes of AWS / GCP.
- numbsafari 5y ago
  Healthcare is definitely another. Government, too (especially non-US).
  - tgoneaway 5y ago
    They are going to have to get their employees to stop burning down federal courthouses first. They'll never get a FCL with that history. Steve is a well-known outspoken anarchist that applauds violence for his politics and Bryan isn't much better with his woke agenda.
- twoodfin 5y ago
  Not just “don’t want to”, I’d hope: They should be able to win on the economics, too, assuming customers that care more about TCO for a fixed or steady-growing workload rather than elasticity.
ukd1 5y ago
Wow, I love the graphics on this website. Seems like it'd could be functional for actual interfaces - is that the plan? Congrats Bryan et al.
yabones 5y ago
Aesthetically, absolutely love it. In reality I would never want this type of hardware... It reminds me of the old boat anchor bladecenter rigs we used to use. They were great, up until you had to replace one of the blades after the support was up. It's not always practical to replace hardware every 3 years like we're supposed to, so this type of stuff sticks around and gets some barnacles. What would be fantastic would be if the entire industry committed to an open spec for large chassis like this with a standardized networking and storage overlay... But that would never happen because vendor lock-in is the big money maker in 'enterprise'. But wow, absolutely gorgeous machines.
- zozbot234 5y ago
  > What would be fantastic would be if the entire industry committed to an open spec for large chassis like this with a standardized networking and storage overlay Isn't the Open Compute Project supposed to be working on that kind of stuff?
benlivengood 5y ago
It's interesting that the RAM/CPU ratio is about double the default shapes from AWS/GCP. In practice I have generally seen those shapes run on the low side of CPU utilization for most workloads, so I think the choice makes sense. I'm curious if ARC will be running with primarycache=metadata to rely on low latency storage and in-VM cache, otherwise I could see ARC using a fair bit of that RAM overhead in the hosts.
deleted 5y ago
[deleted]
tom_mellior 5y ago
Something about this website takes my machine to 500% CPU load (Firefox/Linux). Good thing we will get 2048 cores soon...
- faitswulff 5y ago
  Ha! So I'm not the only one. My wife thought I had fired up a video game.
- Havoc 5y ago
  Same. FF/Win
trhway 5y ago
>The elevation of the room where the rack is installed must be below 10,005 feet (3,050 meters). seems to be excluding Bolivia and probably Mars too.
nickik 5y ago
So, does anybody think it would be overkill to put Kubernetes on this to host my blog?
nodesocket 5y ago
I have a few legacy HP Proliant (cheap eBay) rackmount servers in my office closet. Oxide looks awesome, but obviously not targeted for home / small business use. I was hoping they would offer single-u servers.
notacoward 5y ago
All NVMe seems like a good starting point, but I'd hope that some day there will be a more capacity-oriented variant for people who actually know what they're doing with exabyte-scale storage.
- doublepg23 5y ago
  Can you expand on what you're doing on an exabyte scale?
  - notacoward 5y ago
    Nothing any more, but I used to work on such systems at Facebook. The public name was Tectonic; there was a paper at FAST this year IIRC. As time goes by, this kind of scale is going to be more common. I still remember when having a single petabyte was something to brag about.
corysama 5y ago
If you haven’t been listening to their “On The Metal” podcast you are really missing out! https://oxide.computer/podcasts https://oxide.computer/podcasts It’s all fun stories from people doing amazing things with computer hardware and low level software. Like Ring-Sub-Zero and DRAM driver level software.
ThinkBeat 5y ago
Hmm. They basically reinvented mainframes. Seems it has a lot in common with Z series. Scalable locked in hardware, virtualization, reliability, engineered for hardware swaps, upgrades. A proprietary operating system (?) from what someone said. (Offshoot of Solaris +++ (???) By that I mean that most of it, or all of it might be open sourced forks, but it will be an OS only meant to run on their systems. (It would be fun to get it working at home, on a couple of PCs or a bunch of PIs) They lack specialized processors to offload some workloads to. Perhaps in modern terms shelfs of GPUs or a shelf fast FPGA , DSP processors. The possibilities are huge. I didn't find any mention of from what I read. They also lack the gigantic legacy effort to be compatible, which is a good thing.
- zozbot234 5y ago
  Their approach to reliability isn't quite on par with mainframes, AIUI. At least, not yet. And the programming model is also quite different - a mainframe can seamlessly scale from lots of tiny VM workloads (what Oxide seems to be going for) to large vertically-scaled shared-everything SSI, and anything in between.
  - wd45 5y ago
    Ignoring hardware reliability, thanks to the integration, their solution should be more reliable than whatever byzantine solutions are currently used in their target market. I've worked in a shop (a well-known name that I won't mention) that had a mix of "chat ops" and Perl scripts integrated with JIRA where you could request a Linux VM through a JIRA ticket and get it automatically provisioned, I assume from some big chassis running VMWare, and then use git+Puppet to configure it. It works, but it's a lot of software from different sources and there is always one thing or the other failing. And the security of all that stuff is probably patchy, regardless of audits. That being said, this solution is the mother of all lock ins... I could see it used for the non-critical part of a company's infrastructure. I would not run production stuff on it, but it could work for development systems, test boxes, etc. Basically give developers access and let them create and destroy as many VMs as they need, whenever they need.
bri3d 5y ago
It seems like a lot of Oxide information is currently hiding out in podcasts and other media - does anyone know how the AuthN, AuthZ, ACL system is going to work? One of the most powerful elements of the trust root system is in audit ability and access control for both service-to-service and human-to-system aspects and I'm really interested in seeing how this plays out. For example, a service mesh where hosts can be identified securely and authorized in a specific role unlocks a lot of low-friction service-to-service security. I'm curious what Oxide plan to provide in this space API and SDK wise. I see some Zanzibar related projects on their GitHub, so it can be assumed the ACL system will be based on the principles there - but that's more a framework than an implementation.
ohazi 5y ago
> Our firmware is open source. We will be transparent about bug fixes. No longer will you be gaslit by vendors about bugs being fixed but not see results or proof. There are lots of reasons to be enthusiastic about Oxide but for me, this one takes the cake. I hope they are successful, and I hope this attitude spreads far and wide.
slownews45 5y ago
How can they offer a secure boot solution under the GPLv3 - my understanding is the anti tivoization clauses means they need to release their keys or allow admins and hackers and others to escape the secure boot chain if they are physically in front of the machine or own the machine.
- mlindner 5y ago
  I don't know anything, but I remember hearing it's secure boot with only their releases, if you want to run your own software it's not secure boot anymore, but you're free to run whatever you want.
  - slownews45 5y ago
    One issue with this is normally this requires a bypass of secure boot option. So device is not set and forget secure, but you have to physically secure it (not that you wouldn't, but now it can really matter if someone has physical access to the machine). GPLv3 has been used more recently as a block to others taking code and developing things further given some of the clauses and incompatibilities it introduced.
keyle 5y ago
Side note, great web page, I couldn't stop scrolling like an addict on visual adrenaline.
fierro 5y ago
When can I run Kubernetes on this to host my personal blog?
calvinmorrison 5y ago
After the acquisition of Joyent by Samsung, who here is interested in buying an extremely locked down hardware that seems seems to be a next-gen SmartOS (provisioning etc), i.e. rethinking networking and nodes all in holistic sense. proposition is good, history is bad
totetsu 5y ago
What do you call the kind of tui graphics aesthetic that is being emulated on this webpage? and are there any frameworks to use it in the browser?
- possibleworlds 5y ago
  Expensive, since they engaged Pentagram for branding. More seriously, the animations have "ascii-animation" classes in the dom, tui is probably more fitting aesthetic label if you ask me. Not sure if a lib is involved or its custom. Either way it is done very nicely.
- yjftsjthsd-h 5y ago
  https://news.ycombinator.com/item?id=27295638 https://news.ycombinator.com/item?id=27295638
SMAAART 5y ago
If Joel Spolsky and Paul Graham had a baby....
soheil 5y ago
To me this is a webpage that promises the technology before the days of AWS and GCP with some cool ascii art animation. Sure, they have a solid team of engineers but what is the value proposition exactly? A blackbox server that we built to our taste (AMD Milan, what if I want Intel Xeon?) to provide you custom software to manage and monitor server health and notify you if you could upgrade/downgrade to a different size? Oh and no cables with lots of aesthetics to make your datacenter look pretty... And?
deeblering4 5y ago
Soo... we’re switching back to blade servers again? The problem with this model is its no longer commodity hardware. You are kinda locked into their exosystem of specialized network and server equipment. And it of course introduces some unique failure modes to mitigate too. Not to say its not a cool idea, it’s just interesting to see how hardware trends oscillate between commodity and highly specialized proprietary designs.
mapgrep 5y ago
Looks cool. Please add an RSS feed to your blog, oxide. So people can keep up. The rss logo is not a link (at least on mobile) and there is no auto discovery tag (that my reader can find). Also your site badly crashes Brave iOS browser fwiw.
tgtweak 5y ago
First thoughts: Nobody will give this company a chance in that top-dollar, high stakes enterprise space without a track record. Second thoughts: Oh it's former SUN/joyent guys, nvm. Will the console have fancy ncurses reporting and analytics like then product page?
mnemotronic 5y ago
It's an advertisement. So now ycombinator is doing spam distribution?
- yjftsjthsd-h 5y ago
  It's not spam if people here find it interesting.
hewrin10 5y ago
Sigh, yet another one of those wonderful companies that are way beyond my timezone. (and probably skill level too, to be frank)
locusm 5y ago
I hope this is one product of many and sometime in the future they cater to much smaller markets.
fredros 5y ago
I’m still reluctant on using appliances based on a niche OS like illuminos. It takes a huge amount of manpower to maintain a kernel, and this effort is not shared much. I guess that zfs is the selling point here, but still...
Carnageous 5y ago
Never been so turned on by hardware
herpderperator 5y ago
Reminded me of https://youtu.be/6KbRA2RjhgQ?t=115 https://youtu.be/6KbRA2RjhgQ?t=115 :)
edem 5y ago
This is my favorite part: > Some will say that we should be paying people differently based on different geographical locations. I know there are thoughtful people who pay folks differently based on their zip code, but (respectfully), we disagree with this approach. Companies spin this by explaining they are merely paying people based on their cost of living, but this is absurd: do we increase someone's salary when their spouse loses their job or when their kid goes to college? Do we slash it when they inherit money from their deceased parent or move in with someone? The answer to all of these is no, of course not: we pay people based on their work, not their costs. The truth is that companies pay people less in other geographies for a simple reason: because they can. We at Oxide just don't agree with this; we pay people the same regardless of where they pick up their mail.
throwaway2037 5y ago
FYI: The CTO of the Oxide Computer Company is Bryan Cantrill. (He is responding in this discussion as "bcantrill" -- I assume[!].) You can read about him on Wiki: https://en.wikipedia.org/wiki/Bryan_Cantrill https://en.wikipedia.org/wiki/Bryan_Cantrill He also has many interesting and thoughtful recorded talks on YouTube. I highly recommend them. I am posting this info because it seems their "team" page (https://oxide.computer/team/ https://oxide.computer/team/) is no longer working. I thought it was weird there was no way to see the senior leaders from the website. When I first opened the site, I vaguely remembered this brand name, but could not remember who was behind it. @bcantrill: I assume this is a mistake. Plus I cannot find a 'Team' link anywhere on the current website. // Edit I just found section "Meet the Team" on this page: https://oxide.computer/careers https://oxide.computer/careers
deleted 5y ago
[deleted]
lkcl 5y ago
"Attests the software version is the version that is valid and shipped by the Oxide Computer Company" this makes "Oxide Computer Company" the primary target and point of vulnerability in multiple ways. 1) rogue employees (state-sponsored, corporate espionage) could replace the software. customers could do nothing about it, and might not even be told. 2) sale of the company by the VCs or a Corporate take-over gives no guarantee that what is safe now will be safe in future, no matter what the VCs or the company says right now. 3) whatever expertise "Oxide Computer Company" thinks they have, they're the single-point-of-failure. the larger the number of customers, the less likely that a given vulnerability will be immediately fixed and distributed out. this is just some of the possibilities. sorry to say that there's so many things wrong with this idea it's really hard to hold back and not say anything. now, if the full source code right to the bedrock is available, and the CUSTOMER is given FULL CONTROL, THEN we do not have a problem. by "full control", that includes: * all DRM keys including TPM signing private keys * all peripheral initialisation source code (including DDR4 firmware, PCIe firmware and USB3 firmware) * BMC (Boot Management Console) source code * BIOS source code * Operating system source code * full source code for all tools and toolchains for the above to avoid vendor lock-in and the possibility of the toolchain itself introducing rogue code. this is one hell of a list and it's almost impossible to fulfil with today's "NDA'd proprietary firmware 3rd party licensing" mindset. the only company in this secure server space to my knowledge that's achieved this is Raptor Engineering with the TALOS-II, when running with the Kestrel BMC replacement, on the Lattice ECP5 FPGA.
nojvek 5y ago
I get an “ An unexpected error has occurred.” when I load the site.
ChrisArchitect 5y ago
Sleek AF Pentagram-designed website with a nice balance of style and nerdiness like the ASCII art animations. Can't miss the Halt and Catch Fire tv references including Haley Clark alongsize Woz and other tech legends and in the blog post about the launch a terminal window with character names like Gordon Clark, etc. Love to see it.
unixhero 5y ago
I cannot wait for this. However I also do not see when I will buy a full shelf of gear, even though I would love to. Will they also release a maxi version, micro version and a nano version? Ie. 2U server, PRO workstation and an small formfactor? I think the innovations they have done to these computers deserve to be even more places than just in a massive and awesome data processing rack.
- rugwirobaker 5y ago
  A small form factor would definitely make sense as test environment.
anarchyrucks 5y ago
Reminds me of "the box" from Silicon Valley comedy series.

876 points by ykl 5y ago

warrenm 5y ago

wish I was a good fit for the company!

zapita 5y ago

Who is the target customer for this?

caeril 5y ago
Cantrill has always said it's for people who want Facebook class on-premise infrastructure but don't have a $900B market cap and a hundred engineers designing and building custom boxes. Oh, and open firmware.
- jeffbee 5y ago
  Can definitely see it for a company size of Dropbox, big enough to already be working with ODMs, big enough to be sensitive to the kind of headaches you get from a heterogeneous fleet of ILOM processors designed by deranged engineers.
  - qbasic_forever 5y ago
    Dropbox is big enough that they could just acquire Oxide now before they even get to market. That might even be the plan all along. I can't imagine there are more than like a dozen companies that are their target market, i.e. big enough to need Facebook-level datacenters but not big enough (yet) to have that engineering team.
  - yjftsjthsd-h 5y ago
    > ILOM processors designed by deranged engineers. As opposed to what? But seriously, why do all ILOMs suck and are there any exceptions?
    - jeffbee 5y ago
      The people behind OpenBMC, RunBMC, LibreBMC are hoping to fix this.
    - daddylongstroke 5y ago
      They don't all suck. I always liked how Sun/Oracle and Fujitsu's did their LOMs.
lumost 5y ago
This level of integration surely won't come cheap, from what I recall of Server purchasing a target price of ~200-500k per rack would be expected on a TCO of roughly 2x the rack price over 3 years. (Assuming you are buying from Quanta/SuperMicro or other commodity integrator) It's possible the prices are different now, but you would need customers looking to drop > 1 million dollars in CapEx for the management capabilities they are providing. Possibly non-cloud Fortune-500?
- riking 5y ago
  The core customer profile appears to be aging startups that know how their software behaves and are currently way over 2k cores on AWS.
  - benlivengood 5y ago
    Startups with no significant use of anything from AWS other than EC2+EBS with an existing well-tested storage migration procedure.
- dcolkitt 5y ago
  That reminds me a lot of the Sun Microsystem mega servers from 20 years ago. Those were kind of the cure-all solutions to high scalability web services before Google et al. pioneered cloud-like services on commodity software.
  - nickik 5y ago
    Its kind of like if you took those servers and put a cloud like software layer on top and made many of them work together.
- kraig 5y ago
  They'd have to sell these at a significant loss to make up for the risk any company would have to take to build out a DC on first generation hardware from a startup.
  - 5y ago
atonse 5y ago
I can see this also being appealing for large orgs with sensitive data they don't want to put in a public cloud.
convolvatron 5y ago
good question. not someone too large to want to pay the per-node margins. not someone too small to want to pay .. not someone who is satisfied with using VMs on a cloud provider. not someone who is selling these as part of a turnkey solution for whatever segment is left. that said, I do feel persistently sad that we can't fix structural problems because the market is such a gradient descent world.
rapsey 5y ago
Plenty of companies live outside the cloud on their own hardware.
- daddylongstroke 5y ago
  The vast majority of companies live on their own hardware for business-critical workloads. It's not even close.
salmo 5y ago
I get the target of HyperConverged infrastructure. It's a pretty big market: potentially all private/colo datacenters. And there are only a few players left. Dell/EMC/VMWare, Nutanix, Cisco's schizophrenic offerings, a waning HP, cloud providers trying to make a 'hybrid' play, etc. And most don't buy one or two of these things. It's rows and rows. But most of those are so entrenched and wrapped up in their customers. I imagine the target here is actually acquisition, it would just be too hard to get a foothold as an up-and-comer. Also it usually means giving loaner gear to companies for an extended period for them to evaluate pre-purchase, showing your support, etc. That's a lot of up-front cost for someone without a warchest. I'm also kind of surprised by the site. It sells to geeks well, but isn't the normal "look at our customers in key industry segments!", "something something Gartner magic quadrant", "whitepapers!" thing. Selling to execs on these things is usually a matter of convincing them they're not making a "bad" decision. They're "cool", but enough industry people agree with them that it's not career limiting if it doesn't pan out. I like the idea of the product, and it would be nice to have another player. But it's like starting a new car company, and I feel like they're selling to mechanics.

fsociety 5y ago

I kind of want one of these in my home, I wonder what the price range will be.

gwbrooks 5y ago
I'm adding one of these to the if-I-win-the-lottery list. Nifty home media server AND it'll probably keep the house warm during the winter.
softfalcon 5y ago
“2048 x86 cores per rack” Probably a lot more than any sane person would pay for a home server. But I won’t stop you from trying! Wouldn’t it be cool to have that plugged into your local network?
- bayindirh 5y ago
  I'd get one. Would connect hot side to home HVAC and just run scalability tests of my code. That'd be hot, nice and expensive. And possibly noisy. Yes, noisy.
ptomato 5y ago
for what they're advertising there, on the order of $1mm, based on underlying hardware costs.
- kaliszad 5y ago
  Actually, standard Azure rack is on the order of $1.1 mm of hardware depending on SKU if I am not mistaken. So I would guess, it could be more like $2 mm. There is the aspect of management and other vendors Dell/EMC + VMware like you to pay way more than the hardware costs for e.g. VXRail/ vSphere licences. That is the real target.
  - salmo 5y ago
    And maintenance. I like the idea of having to give them a key to your house to replace a disk when the SMART alert phones home. :)

discardable_dan 5y ago

I don't know if I have a use for something like this, but the website aesthetics are just plain awesome.

tyingq 5y ago
Yeah, I noticed that too. The green wireframe looking stuff is actually text in spans/divs next to, or overlayed on pictures. The little "nodes" are this character, for example: ⎕. The effect is pretty unique.
- goodpoint 5y ago
  Looks like simple ascii art, typically used by CLI tools.
megous 5y ago
I like them too but just viewing the page on Firefox loads 5950X to 250%. A bit too much.
- mcdevilkiller 5y ago
  Wow. It loads almost instantly and doesn't slow down on my phone (LG G7). Must be something else?
  - megous 5y ago
    You have to scroll all the way through the page to activate all the gimmicks. Then it never stops, and loads 2.5 cores permanently to 100%, which makes CPU fan spin to the max.
    - kfajdsl 5y ago
      Oh my god, that's hilarious. I was wondering why my lap was warm all of a sudden. Htop said firefox was the culprit so I closed out all my tabs except this one. Then I read your comment, opened the page, and scrolled all the way to the bottom—my cpu temp just steadily rose till it throttled. All the animations are smooth, though. Note: I'm typing this from a 9 year old thin-and-light, so that's probably part of the problem.

atonse 5y ago

This looks interesting (although I'm not in the target market, too small)... But if I were looking at this, judging from the quality of people they've amassed in their engineering team, is there any chance they won't be acquired in 6 months? To anyone looking to take a bet on this, what is the answer to "what's your plan for when your stellar team gets acquired?" And what answer will satisfy that buyer? Update: Adding another question, does this "environment" (where any really great product with great talent in it can be acquired very quickly) have a chilling effect on purchases for products like this? Hopefully some Oxide people can answer :-)

bcantrill 5y ago
Hi! So, at every step -- from conception to funding to building the team and now building the product -- we have done so to build a big, successful public company. Not only do we (the founders) share that conviction, but it is shared by our investors and employees as well. For better or for ill, we are -- as we memorably declared to one investor -- ride or die. Also, if it's of any solace, I really don't think any of the existing players would be terribly interested in buying a company that has so thoroughly and unequivocally rejected so many of their accrued decisions! ;) I'm pretty sure their due diligence would reveal that we have taken a first principles approach here that is anathema to the iterative one they have taken for decades -- and indeed, these companies have shown time and time again that they don't want to risk their existing product lines to a fresh approach, no matter how badly customers want it.
- tw04 5y ago
  Isn't that what some of your old Sun compatriots thought with DSSD? :) Congrats on the announcement, here's hoping you're right! This looks too interesting to be swallowed by Oracle or HPe.
  - bcantrill 5y ago
    Fortunately, some of those same DSSD folks have joined us at Oxide -- and let's just say that they are of like mind with respect to Oxide's approach. ;)

newaccount2021 5y ago

Bravo! Better servers for people who want to own their infra. Too many people seek out cloud services just to get a modern control plane. Server "UI" has been long neglected. And finally, its nice to see people with brains building real things with nary a mention of "blockchain".

transfire 5y ago

Estimated price?

wmf 5y ago
Cheaper than AWS Outposts I guess. (Cheaper per core, not necessarily per rack.)
kondro 5y ago
Guessing at $400 - $1000 per core.

diwu1989 5y ago

Looks like a white labeled viking VDS2249R running custom software.

ChrisMarshallNY 5y ago
This is not my area of expertise, but it does look like that[0]. That "custom software," though, is where the magic often lies. As a software person that worked at hardware companies for most of my career, I know all too well, how disrespectful hardware people are of software. If they have a good software-respecting management chain, then it might be pretty awesome. [0] https://www.prnewswire.com/news-releases/viking-enterprise-solutions-introduces-new-nvme-over-fabrics--enterprise-storage-solution-for-high-performance-applications-301297571.html https://www.prnewswire.com/news-releases/viking-enterprise-s...
- selectodude 5y ago
  Well if Oxide's software stack is going to be open source, I guess we'll get a good look into their secret sauce.
- dralley 5y ago
  Let's be honest, software people are pretty disrespectful about hardware, too.
  - ChrisMarshallNY 5y ago
    Not me. I started as an EE. Wanna check out my first professional engineering project ever? https://littlegreenviper.com/TF30194/TF30194-Manual-1987.pdf https://littlegreenviper.com/TF30194/TF30194-Manual-1987.pdf (Downloads a PDF)

loudmax 5y ago

As I understand it, Oxide is going to have deep software integration into their hardware. So the expectation isn't that the servers in this rack will be running Windows or a generic Linux distribution. In case anyone from Oxide is here, is my understanding correct? And if so, will there be a way to run a smaller version of an Oxide system, say for testing or development, without purchasing an entire rack at a time? Anyway, glad to finally get a glimpse of what Oxide has to offer. Looking forward to seeing a lot more.

2trill2spill 5y ago
My understanding is you will use a API to provision virtual machines on top of the Oxide hypervisor/software stack, which is bhyve running on Illumos. So you can still just run your favorite Linux distro or windows or a BSD if you want[1]. [1]: https://soundcloud.com/user-760920229/why-your-servers-suck-and-how-oxide-computer-plans-to-make-this-better https://soundcloud.com/user-760920229/why-your-servers-suck-...
- SSLy 5y ago
  bhyve? what happened to the KVM port?
  - 2trill2spill 5y ago
    Apparently bhyve will be replacing KVM going forward, this article has a list of reasons[1]. [1]: https://omnios.org/info/bhyve https://omnios.org/info/bhyve
jhickok 5y ago
Agreed, I would love to hear more about the management plane. I'm glad it's API-driven, but I still have some questions about things like which hypervisor they are using. If it's a custom software stack, might be nice to get a miniature dev-kit!
- deleted 5y ago
  [deleted]

stefan_ 5y ago

Why are we hard coupling the hardware to the software? The whole secret of the success of M1 and ARM in servers is that lots of software has long ago stopped being hyper-aware of what hardware it is running on. What software are we talking about anyways? It's all incredibly vague, but it seems to reach all the way into the Kubernetes sphere. Why would I run this over something I can use on my next job?

monocasa 5y ago
It's probs cheaper than AWS if you already have on prem infra. AWS has pretty damn good margins. And the idea of "these racks are my kubernetes cluster and are supported by the OEM as such" has a lot of value to a lot of the medium sized IT departments I've run across. Can you expand on what you mean on "coupling the hardware to the software"?
alanbernstein 5y ago
Is Apple really a good counterexample for the success of integrating software and hardware?
tediousdemise 5y ago
> Why are we hard coupling the hardware to the software? The whole secret of the success of M1 and ARM in servers is that lots of software has long ago stopped being hyper-aware of what hardware it is running on. The software running on M1 is a bespoke fit for it. That's why the performance in macOS on M1 is phenomenal. It was custom made to execute optimally on it.
rcxdude 5y ago
It's basically a mini-ec2 in your server room. The software and hardware give you a platform to deploy VMs configured however you want.

boulos 5y ago

First, congrats! But second, I'd love to understand the compute vs storage tradeoff chosen here. Looking at the (pretty!) picture [1], I was shocked to see "Wow, it's mostly storage?". Is that from going all flash? Heading to https://oxide.computer/product https://oxide.computer/product for more details, lists: - 2048 cores - 30 TB of memory - 1024 TB of flash (1 PiB) Given how much of the rack is storage, I'm not sure which Milan was chosen (and so whether that's 2048 threads or 4196 [edit: real cores, 4196 threads]), but it seems like visually 4U is compute? [edit: nope] Is that a mistake on my part, because dual-socket Milan at 128 threads per socket is 256 threads per server, so you need at least 8 servers to hit 2048 "somethings", or do the storage nodes also have Milans [would make sense] and their compute is included [also fine!] -- and so similarly that's how you get a funky 30 TiB of memory? [Top-level edit from below: the green stuff are the nodes, including the compute. The 4U near the middle is the fiber] P.S.: the "NETWORK SPEED 100 GB/S" in all caps / CSS loses the presumably 100 Gbps (though the value in the HTML is 100 gb/s which is also unclear). [1] https://oxide.computer/_next/image?url=%2Fimages%2Frenders%2Foxide-rack-front.png&w=2048&q=75 https://oxide.computer/_next/image?url=%2Fimages%2Frenders%2...

ptomato 5y ago
It looks like they're doing 2u half width nodes, so I'd strongly suspect each node is 1TB of ram, one epyc 7713p, and 10 3.2TB u.2/u.3 drives. eta: also suspect 30TB total just means they're leaving 64GB ram for the hypervisor OS on each node.
- kaliszad 5y ago
  Leaving that RAM for ZFS L2 ARC perhaps? I don't think they would use Illumos as the hypervisor OS without also using OpenZFS with it. They also need some for management, the control UI, a DB for metrics and more. Btw. if I count correctly, they have 20 SSD slots per node (if a node is full width) and 16 nodes. They would need 2 TB to reach 1 PB of "raw" capacity with the obvious redundancy overhead of ~ 20%. It is also quite possible, they don't use ZFS at all and use e.g. Ceph or something like it but I don't think that is the case, because that wouldn't be cantrillian. :-) E.g. using Minio, they can provide something S3 like on top of a cluster of ZFS storage nodes too but they most likely get better latency with local ZFS and not a distributed filesystem. Financial institutions especially seem to be part of the target here and there latency can be king.

hlandau 5y ago

"Attests the software version is the version that is valid and shipped by the Oxide Computer Company" So in other words these servers will implement restrictive code signing practices and will be vendor-controlled, not owner-controlled? This is not my idea of "secure", and really in the wake of things like the Solarwinds or RSA hacks it shouldn't be anyone's idea of secure. Vendor-holds-the-keys is not an acceptable security model. A comment below mentions open firmware, open firmware is useless without the right to deploy modified versions of it. Happy to take clarification on this.

gruez 5y ago
"Attest" refers to https://en.wikipedia.org/wiki/Trusted_Computing#REMOTE-ATTESTATION https://en.wikipedia.org/wiki/Trusted_Computing#REMOTE-ATTES..., not forced code signing
- hlandau 5y ago
  I'm familiar with the concept. Does this mean that attestation to a different root of trust than Oxide will also be feasible, and that this is just a default?
  - hobofan 5y ago
    It's an "enterprise" product, so you can be certain that some amount of extra money will buy you that capability (if it's not already included).
  - gruez 5y ago
    Oxide makes the hardware, so it makes sense to use them as the root of trust since you already have to trust them to not make backdoored hardware. Why bother adding more parties? Also, for remote attestation to make sense, it needs to be done in the hardware itself (ie. keys burned into the silicon). I'm not sure how that's supposed to work if you add your own keys, or whether that would even make sense.
    - 5y ago

deleted 5y ago

[deleted]

renewiltord 5y ago

I'm an idiot. I thought this was like the SGI UV300 where you'd view the whole thing as a single computer and everything would be NUMA'd away. It looks like it's not like that, though.

rbanffy 5y ago

Looks interesting. I wonder if the high integration makes it behave as a single-image machine with that many cores.

wmf 5y ago
No, single system image is really expensive (see Superdome Flex) and most workloads can't justify that cost.
- zozbot234 5y ago
  Could one reimplement SSI at the OS layer, similar to existing distributed OS's? Distributed shared memory is usually dismissed because of the overhead involved in typical scenarios, but this kind of hardware platform might make it feasible.
  - wmf 5y ago
    There was Mosix at the OS layer in the 1990s and Virtual Iron at the hypervisor layer in the aughts. I think the cost and performance of software SSI just doesn't intersect with demand anywhere.
    - e12e 5y ago
      Plan9 on an oxide rack might get you close to an SSI work-a-like - but I think it's unlikely it'd be a practical use of the hw?
      - zozbot234 5y ago
        AIUI, Plan9 is not quite fully SSI. It is a distributed OS, and gets quite close, but it's missing support for distributed memory (i.e. software-implemented shared memory exposing a single global address space); it also does not do process checkpointing and auto-migration, without which you don't really have a "global" system image.

psanford 5y ago

Congrats Oxide team! More competition in this space is always a good thing. I'm curious about management. Can the rack operate completely standalone? I assume when you have multiple there will be some management abstraction above the rack layer? The closest direct equivalent that I can think of to this is AWS outposts. Are there any others that I'm forgetting?

wmf 5y ago
There are a bunch of enterprisey "private cloud" aka "converged infrastructure" racks like VxRack, Nutanix, etc.
_delirium 5y ago
The density they're getting here is significantly higher than AWS Outposts, which is interesting. The top-end (~$600k) AWS Outposts seem to max out at around 1k CPUs and 4.5 TB RAM in a rack (e.g. 12x m5.24xlarge = 12x 384 GB), while this rack can house 2k CPUs and 30 TB (!) RAM.
- psanford 5y ago
  Outposts seem like a solution to the problem, "for regulatory or compliance reasons we are required for data to reside and be processed within a physical space we control." For that problem, an organization that is otherwise on AWS might find Outposts to be appealing. I can imagine an engineering team's response to such a requirement as "Oh yeah? Fine, its but its going to cost you $600k per year per rack!". I believe Oxide is attempting to capture a much broader market than that.
- ptomato 5y ago
  it's better than that, actually, as an AWS vCPU is a hyperthread, not a full core, and these would have 4096 hyperthreads.
- kaliszad 5y ago
  Yes, well it isn't that dense either. As I have written, it's 32 CPUs (16x 2 CPUs). 1 TB of RAM per CPU is not that huge a deal, it's perhaps 16x 64 GB (Milan uses 8 channels, 2 DIMMs per channel is reasonable), if you consider that is 16 GB of RAM per core. In HPC, you would probably shrink it to 1/4 of the volume (half width, 1 U dual socket server). Oxide probably focuses on optimal thermal efficiency since their limit isn't the space so much as the power density/ max. power per rack in existing DCs, which they are already pushing hard. (Of course they have lower power options too but they probably will not use 2048 cores.)

alberth 5y ago

Who is the target buyer? Is this to compete against Nutanix / VCE?

fuzzylightbulb 5y ago
I think it would more play in the space of a NetApp/Cisco FlexPod or VCE's Vblock, but what those customers are really purchasing is the certified validation of core enterprise apps on a particular hardware/software stack, as well as the massive engineering and support organizations that those companies can bring to bear to validate firmware upgrade and to swoop in in the event of an issue. You also seem to get a LOT more flexibility. I am not a hater in the least but I really am failing to understand what is unique about this offering. It seems like you have no options regarding the internals, and so scaling compute separately from storage doesn't seem possible. I also am very suspect about offerings like this that have not yet released a second version of their key parts. Everyone says that they are going to be backwards compatible, but then the reality of managing heterogenous generations of gear in a homogenous fashion strikes and you get weird behavior all around. Long story short, I would love to know what a customer of this scale of physical infrastructure is getting with Oxide that they would not be better served by going to one of the major vendors.
- Tuna-Fish 5y ago
  Firmware that does not suck? Because that's something the current "major vendors" really are irredeemably terrible at.
etcet 5y ago
We use Nutanix where I work and this has made everyone very excited. Though they would need something similar to Nutanix CE to make us switch entirely (i.e. the ability to run non-production unsupported on commodity hardware).

kaliszad 5y ago

"Only" 2048 CPU cores per rack is actually not that much by nowadays standards - its 16 U of 2x 64 core CPUs. Perhaps is could be more U if they used the lower core counts but e.g. higher frequency per core SKUs but I don't think they do. (And the picture kind of confirms it). They use 2U servers though so they are able to use lower speed but bigger fans and have more expansion cards and 2,5" form factor drives perhaps. The of course have to fit storage, which needs lots of CPU PCIe lanes for all the NVMe storage and networking (probably 2 or 4 U) and power conversion to power the bus bar and more somewhere. They probably use the 42 U+ standard 19" racks to fit in standard customers DCs. They also don't have such a high power budget as custom DCs for cloud providers do. 1 PB of flash is quite a bit but you could get perhaps 5x as much with HDDs probably (even with a relatively low density of 40x 12 x 12 TB). The problem really is I think, they wouldn't be able to write the HDD firmware in Rust in time (or at all, because no HDD manufacturer would sell an HDD to them without making sure their proprietary firmware is used). SSDs don't necessarily have this property as they are much more like the other components of a modern server.

mindfulplay 5y ago

Congrats to Oxide computer! Excited to see tech startups do actual tech instead of chasing VC funded growth hacking. I wonder what sort of enterprise customers this targets.. (definitely not for individual devs)

rcarmo 5y ago

Well, congrats then! I've been waiting for news (and listening to the "On The Metal" podcast) for a long while now, and this seems like a great way to push the envelope on server hardware. (plus I suspect there will be more to come...)

kaliszad 5y ago

As for the website, some of the animations could use a spring like movement profile to feel more physical. The website also isn't reachable over IPv6, so I would be very careful with the promised IPv6 capabilities of the server too ;-)

_jal 5y ago

It sure looks pretty, but appears to be - - dedicated to virtualization, done their way - rather inflexible in hardware specs - vendor-locked at the rack - if you have hardware from someone else, it can't live in the same cabinet I guess if you just want a pretty data center in a box and look like what they consider a 'normal' enterprise to be, it might appeal. But I'm not sure how many people asked for Apple-style hardware in the DC.

wmf 5y ago
A lot of customers are asking for private cloud.
zozbot234 5y ago
> - dedicated to virtualization, done their way > - rather inflexible in hardware specs > - vendor-locked at the rack - if you have hardware from someone else, it can't live in the same cabinet This describes legacy IBM platforms quite well. If they can leverage hyperscaling tech to be better and cheaper than what IBM is currently offering, that's enough to make it worthwhile.
- yesitdoes 5y ago
  It kind of does. One main difference with IBM is the three letters I, B and M.
rapsey 5y ago
Why is it important what kind of virtualization? It works and since it is built for this hardware it will likely be more reliable then anything you're putting together yourself. The specs are damn good. When it is all top-of-the-line, inflexibility is kind of a mute point. Where else are you going to go? > But I'm not sure how many people asked for Apple-style hardware in the DC. Well integrated, performant and reliable hardware that runs VMs where you can put anything on it is pretty much all everyone running their own hardware is looking for. Honestly I am surprised how many here completely misunderstand what their value proposition is.
- 5y ago

MangoCoffee 5y ago

>AMD MILAN Intel is losing on the client and server. everyone is jumping ship to either ARM or AMD for client/server. hopefully Intel new engineer CEO can turn it around like AMD engineer CEO (Lisa Su)

dsr_ 5y ago
Few companies buy on pure performance. Right now, AMD has the performance kings and the price/performance kings. Intel could win price/performance, but they would need to cannibalize their own low-end and mid-range market. If they could make a good bet that they would have high yield in one more cycle, that would make sense. If they don't think that will happen, there's nothing much that will save them, and they're extracting the money that they can right now.
- StreamBright 5y ago
  I thought M1 had the best price/performance numbers, too bad Apple does not sell CPUs.
  - dsr_ 5y ago
    Or, more relevant to this discussion, servers.
    - zozbot234 5y ago
      They sell rack-mountable hardware (Mac Pros) running a certified Unix OS. It's only a matter of time until those are based on Apple Silicon too.
      - floatboth 5y ago
        Rack Mac Pros are targeted at racks that are mostly filled with audio/video equipment. Apple really doesn't seem to have any interest in selling server products again.

throwkeep 5y ago

This looks great. Really well thought out, beautifully designed and presented.

Sanguinaire 5y ago

Looks fantastic, and the hardware specs appeal to me greatly - but I'm not sure there is an actual market outside the "cult of personality" bubble. A few SV wannabes will buy into this to trade off a Twitter relationship with the Oxide founders - but does anyone really see the IT teams at Daimler, Proctor & Gamble, Morgan Stanley... et al - actually going for this over HPE/Dell and AWS/Azure? We are a long way away from "Nobody ever got fired for buying from Oxide".

tyingq 5y ago
You wouldn't have to pitch it initially as a replacement for your on-prem HPE/Dell. It could be pitched as a replacement for the hosted private cloud you have from IBM, Oracle, etc, that you're unhappy with.
nickik 5y ago
Like everything else, start small, and grow. If you have a good product that actually works.

ebeip90 5y ago

I wonder if they used Monodraw[1] to create their diagrams? Looks like they did! [1]: https://monodraw.helftone.com https://monodraw.helftone.com

thundergolfer 5y ago
Can Monodraw produce the animation as well, as seen on the webpage?
jessfraz 5y ago
YES we did :)

agentdrtran 5y ago

I would love a mini version of this for homelabs.

wmf 5y ago
If the software stack ends up being open source someone could make a name for themself by porting it to run on Linux + random hardware.
- yjftsjthsd-h 5y ago
  If someone manages to port bhyve to Linux, they will definitely make a name for themselves. But honestly, the equivalent is just libvirt on commodity hardware with openzfs storage; the value here is high end hardware with custom firmware and well-integrated software, not really something you can port usefully.
  - wmf 5y ago
    I'm more interested in the control plane but maybe I'm in the minority.

jmartrican 5y ago

Why the elevation constraint? "The elevation of the room where the rack is installed must be below 10,005 feet (3,050 meters)."

ptomato 5y ago
probably air density cooling solution is qualified for.
mikey_p 5y ago
Perhaps a constraint on cooling effectiveness due to air density?
dralley 5y ago
At 10,000 the air pressure is 1/3 less than at sea level, less density means less capacity for carrying heat, so the cooling might not be sufficient.
- notacoward 5y ago
  Bingo. I've personally had to deal with this in other high-density systems. Less cooling not only has the obvious effects, but also reduces PS efficiency which can cause other problems. Cosmic-ray-induced memory errors can also be a problem at those altitudes (or even half that). That's a bit easier to deal with in principle, but the rate of ECC scrubbing required can start to impact performance. Stack that on top of thermal CPU throttling, and you'll have a system that's just slower than it should be. Just as importantly, the slowdown will be uneven across components, so it's effectively a different kind of system when you're debugging low-level issues. I think it's a good sign that they're aware of the additional support issues associated with higher altitude. Shows that they've really thought things through.

todd8 5y ago

This looks great. From a business perspective, I would be concerned that it would be hard to prevent companies like Dell from entering this space as a competitor quite rapidly.

wmf 5y ago
Enterprise vendors are culturally incapable of building opinionated hardware or software. That's a moat, but which side are the customers on?
- AlphaSite 5y ago
  Isn’t this basically: https://www.vmware.com/products/vmc-on-dell-emc.html https://www.vmware.com/products/vmc-on-dell-emc.html Or https://www.hpe.com/us/en/greenlake.html https://www.hpe.com/us/en/greenlake.html Disclaimer vmware employee, but I dont work in this area.
  - mlindner 5y ago
    Sure, but you don't own those. You're renting the hardware from HP.

ksec 5y ago

May be instead of asking target market or audience, who are their competitors? (Edit: Previous Discussions https://news.ycombinator.com/item?id=21682360 https://news.ycombinator.com/item?id=21682360 ) Also thinking if the Website is not finished? All the "Read More" actually hide very little information, if so why hide it? And doesn't seems to explain the company very well. Seems like we need to listen to their PodCast to find out what is going on. ( Edit: Found a Youtube Video about it https://www.youtube.com/watch?v=vvZA9n3e5pc https://www.youtube.com/watch?v=vvZA9n3e5pc ) >Get the most efficient power rating and network speeds of 100GBps without the pain of cable 100GBps would be impressive, 100Gbps would be ... not much? A interesting thing is all the terminal like graphics are actually HTML/CSS and not images.

hujun 5y ago
>Get the most efficient power rating and network speeds of 100GBps without the pain of cable 100GBps would be impressive, 100Gbps would be ... not much? Even 100GBps(800Gbps) is not much for 2048 cores, depends on application, in certain applications, 32 cores could drive 100Gbps...
daddylongstroke 5y ago
> ...who are their competitors? Literally every single server vendor, and almost all (if not all?) storage vendors on the planet are pushing HCI because that is what mid and large size companies want. This is the fastest growing market segment in hardware (because they now realize that hybrid-cloud is the preferred customer model, and most of their customers now are deploying or already have deployed their own internal cloud). Oxide appears to me to be HCI done correctly. I currently work for one of their competitors, and for one, am keeping at eye on their careers page! Also, 100Gb meets requirements for 99.999% of the customers out there.

tw04 5y ago

The storage is always the difficult part in these architectures. Are you distributing across all nodes? It appears that each sled is an individual compute unit with 10 drives. Are the drives on a proverbial island and only accessible to that local node, or is there some distributed storage going on that you can talk about? On paper with RDMA and NVMe-OF you could access any drive from any compute unit... but that's easier said than done :)

gjvc 5y ago

0xide Computer Company deserves to do well. This is a solid private-cloud play aimed at those corporations (probably mainly financials, but other sectors too I'm sure...) who don't want to outsource to the likes of AWS / GCP.

numbsafari 5y ago
Healthcare is definitely another. Government, too (especially non-US).
- tgoneaway 5y ago
  They are going to have to get their employees to stop burning down federal courthouses first. They'll never get a FCL with that history. Steve is a well-known outspoken anarchist that applauds violence for his politics and Bryan isn't much better with his woke agenda.
twoodfin 5y ago
Not just “don’t want to”, I’d hope: They should be able to win on the economics, too, assuming customers that care more about TCO for a fixed or steady-growing workload rather than elasticity.

ukd1 5y ago

Wow, I love the graphics on this website. Seems like it'd could be functional for actual interfaces - is that the plan? Congrats Bryan et al.

yabones 5y ago

Aesthetically, absolutely love it. In reality I would never want this type of hardware... It reminds me of the old boat anchor bladecenter rigs we used to use. They were great, up until you had to replace one of the blades after the support was up. It's not always practical to replace hardware every 3 years like we're supposed to, so this type of stuff sticks around and gets some barnacles. What would be fantastic would be if the entire industry committed to an open spec for large chassis like this with a standardized networking and storage overlay... But that would never happen because vendor lock-in is the big money maker in 'enterprise'. But wow, absolutely gorgeous machines.

zozbot234 5y ago
> What would be fantastic would be if the entire industry committed to an open spec for large chassis like this with a standardized networking and storage overlay Isn't the Open Compute Project supposed to be working on that kind of stuff?

benlivengood 5y ago

It's interesting that the RAM/CPU ratio is about double the default shapes from AWS/GCP. In practice I have generally seen those shapes run on the low side of CPU utilization for most workloads, so I think the choice makes sense. I'm curious if ARC will be running with primarycache=metadata to rely on low latency storage and in-VM cache, otherwise I could see ARC using a fair bit of that RAM overhead in the hosts.

deleted 5y ago

[deleted]

tom_mellior 5y ago

Something about this website takes my machine to 500% CPU load (Firefox/Linux). Good thing we will get 2048 cores soon...

faitswulff 5y ago
Ha! So I'm not the only one. My wife thought I had fired up a video game.
Havoc 5y ago
Same. FF/Win

trhway 5y ago

>The elevation of the room where the rack is installed must be below 10,005 feet (3,050 meters). seems to be excluding Bolivia and probably Mars too.

nickik 5y ago

So, does anybody think it would be overkill to put Kubernetes on this to host my blog?

nodesocket 5y ago

I have a few legacy HP Proliant (cheap eBay) rackmount servers in my office closet. Oxide looks awesome, but obviously not targeted for home / small business use. I was hoping they would offer single-u servers.

notacoward 5y ago

All NVMe seems like a good starting point, but I'd hope that some day there will be a more capacity-oriented variant for people who actually know what they're doing with exabyte-scale storage.

doublepg23 5y ago
Can you expand on what you're doing on an exabyte scale?
- notacoward 5y ago
  Nothing any more, but I used to work on such systems at Facebook. The public name was Tectonic; there was a paper at FAST this year IIRC. As time goes by, this kind of scale is going to be more common. I still remember when having a single petabyte was something to brag about.

corysama 5y ago

If you haven’t been listening to their “On The Metal” podcast you are really missing out! https://oxide.computer/podcasts https://oxide.computer/podcasts It’s all fun stories from people doing amazing things with computer hardware and low level software. Like Ring-Sub-Zero and DRAM driver level software.

ThinkBeat 5y ago

Hmm. They basically reinvented mainframes. Seems it has a lot in common with Z series. Scalable locked in hardware, virtualization, reliability, engineered for hardware swaps, upgrades. A proprietary operating system (?) from what someone said. (Offshoot of Solaris +++ (???) By that I mean that most of it, or all of it might be open sourced forks, but it will be an OS only meant to run on their systems. (It would be fun to get it working at home, on a couple of PCs or a bunch of PIs) They lack specialized processors to offload some workloads to. Perhaps in modern terms shelfs of GPUs or a shelf fast FPGA , DSP processors. The possibilities are huge. I didn't find any mention of from what I read. They also lack the gigantic legacy effort to be compatible, which is a good thing.

zozbot234 5y ago
Their approach to reliability isn't quite on par with mainframes, AIUI. At least, not yet. And the programming model is also quite different - a mainframe can seamlessly scale from lots of tiny VM workloads (what Oxide seems to be going for) to large vertically-scaled shared-everything SSI, and anything in between.
- wd45 5y ago
  Ignoring hardware reliability, thanks to the integration, their solution should be more reliable than whatever byzantine solutions are currently used in their target market. I've worked in a shop (a well-known name that I won't mention) that had a mix of "chat ops" and Perl scripts integrated with JIRA where you could request a Linux VM through a JIRA ticket and get it automatically provisioned, I assume from some big chassis running VMWare, and then use git+Puppet to configure it. It works, but it's a lot of software from different sources and there is always one thing or the other failing. And the security of all that stuff is probably patchy, regardless of audits. That being said, this solution is the mother of all lock ins... I could see it used for the non-critical part of a company's infrastructure. I would not run production stuff on it, but it could work for development systems, test boxes, etc. Basically give developers access and let them create and destroy as many VMs as they need, whenever they need.

bri3d 5y ago

It seems like a lot of Oxide information is currently hiding out in podcasts and other media - does anyone know how the AuthN, AuthZ, ACL system is going to work? One of the most powerful elements of the trust root system is in audit ability and access control for both service-to-service and human-to-system aspects and I'm really interested in seeing how this plays out. For example, a service mesh where hosts can be identified securely and authorized in a specific role unlocks a lot of low-friction service-to-service security. I'm curious what Oxide plan to provide in this space API and SDK wise. I see some Zanzibar related projects on their GitHub, so it can be assumed the ACL system will be based on the principles there - but that's more a framework than an implementation.

ohazi 5y ago

> Our firmware is open source. We will be transparent about bug fixes. No longer will you be gaslit by vendors about bugs being fixed but not see results or proof. There are lots of reasons to be enthusiastic about Oxide but for me, this one takes the cake. I hope they are successful, and I hope this attitude spreads far and wide.

slownews45 5y ago

How can they offer a secure boot solution under the GPLv3 - my understanding is the anti tivoization clauses means they need to release their keys or allow admins and hackers and others to escape the secure boot chain if they are physically in front of the machine or own the machine.

mlindner 5y ago
I don't know anything, but I remember hearing it's secure boot with only their releases, if you want to run your own software it's not secure boot anymore, but you're free to run whatever you want.
- slownews45 5y ago
  One issue with this is normally this requires a bypass of secure boot option. So device is not set and forget secure, but you have to physically secure it (not that you wouldn't, but now it can really matter if someone has physical access to the machine). GPLv3 has been used more recently as a block to others taking code and developing things further given some of the clauses and incompatibilities it introduced.

keyle 5y ago

Side note, great web page, I couldn't stop scrolling like an addict on visual adrenaline.

fierro 5y ago

When can I run Kubernetes on this to host my personal blog?

calvinmorrison 5y ago

After the acquisition of Joyent by Samsung, who here is interested in buying an extremely locked down hardware that seems seems to be a next-gen SmartOS (provisioning etc), i.e. rethinking networking and nodes all in holistic sense. proposition is good, history is bad

totetsu 5y ago

What do you call the kind of tui graphics aesthetic that is being emulated on this webpage? and are there any frameworks to use it in the browser?

possibleworlds 5y ago
Expensive, since they engaged Pentagram for branding. More seriously, the animations have "ascii-animation" classes in the dom, tui is probably more fitting aesthetic label if you ask me. Not sure if a lib is involved or its custom. Either way it is done very nicely.
yjftsjthsd-h 5y ago
https://news.ycombinator.com/item?id=27295638 https://news.ycombinator.com/item?id=27295638

SMAAART 5y ago

If Joel Spolsky and Paul Graham had a baby....

soheil 5y ago

To me this is a webpage that promises the technology before the days of AWS and GCP with some cool ascii art animation. Sure, they have a solid team of engineers but what is the value proposition exactly? A blackbox server that we built to our taste (AMD Milan, what if I want Intel Xeon?) to provide you custom software to manage and monitor server health and notify you if you could upgrade/downgrade to a different size? Oh and no cables with lots of aesthetics to make your datacenter look pretty... And?

deeblering4 5y ago

Soo... we’re switching back to blade servers again? The problem with this model is its no longer commodity hardware. You are kinda locked into their exosystem of specialized network and server equipment. And it of course introduces some unique failure modes to mitigate too. Not to say its not a cool idea, it’s just interesting to see how hardware trends oscillate between commodity and highly specialized proprietary designs.

mapgrep 5y ago

Looks cool. Please add an RSS feed to your blog, oxide. So people can keep up. The rss logo is not a link (at least on mobile) and there is no auto discovery tag (that my reader can find). Also your site badly crashes Brave iOS browser fwiw.

tgtweak 5y ago

First thoughts: Nobody will give this company a chance in that top-dollar, high stakes enterprise space without a track record. Second thoughts: Oh it's former SUN/joyent guys, nvm. Will the console have fancy ncurses reporting and analytics like then product page?

mnemotronic 5y ago

It's an advertisement. So now ycombinator is doing spam distribution?

yjftsjthsd-h 5y ago
It's not spam if people here find it interesting.

hewrin10 5y ago

Sigh, yet another one of those wonderful companies that are way beyond my timezone. (and probably skill level too, to be frank)

locusm 5y ago

I hope this is one product of many and sometime in the future they cater to much smaller markets.

fredros 5y ago

I’m still reluctant on using appliances based on a niche OS like illuminos. It takes a huge amount of manpower to maintain a kernel, and this effort is not shared much. I guess that zfs is the selling point here, but still...

Carnageous 5y ago

Never been so turned on by hardware

herpderperator 5y ago

Reminded me of https://youtu.be/6KbRA2RjhgQ?t=115 https://youtu.be/6KbRA2RjhgQ?t=115 :)

edem 5y ago

This is my favorite part: > Some will say that we should be paying people differently based on different geographical locations. I know there are thoughtful people who pay folks differently based on their zip code, but (respectfully), we disagree with this approach. Companies spin this by explaining they are merely paying people based on their cost of living, but this is absurd: do we increase someone's salary when their spouse loses their job or when their kid goes to college? Do we slash it when they inherit money from their deceased parent or move in with someone? The answer to all of these is no, of course not: we pay people based on their work, not their costs. The truth is that companies pay people less in other geographies for a simple reason: because they can. We at Oxide just don't agree with this; we pay people the same regardless of where they pick up their mail.

throwaway2037 5y ago

FYI: The CTO of the Oxide Computer Company is Bryan Cantrill. (He is responding in this discussion as "bcantrill" -- I assume[!].) You can read about him on Wiki: https://en.wikipedia.org/wiki/Bryan_Cantrill https://en.wikipedia.org/wiki/Bryan_Cantrill He also has many interesting and thoughtful recorded talks on YouTube. I highly recommend them. I am posting this info because it seems their "team" page (https://oxide.computer/team/ https://oxide.computer/team/) is no longer working. I thought it was weird there was no way to see the senior leaders from the website. When I first opened the site, I vaguely remembered this brand name, but could not remember who was behind it. @bcantrill: I assume this is a mistake. Plus I cannot find a 'Team' link anywhere on the current website. // Edit I just found section "Meet the Team" on this page: https://oxide.computer/careers https://oxide.computer/careers

deleted 5y ago

[deleted]

lkcl 5y ago

"Attests the software version is the version that is valid and shipped by the Oxide Computer Company" this makes "Oxide Computer Company" the primary target and point of vulnerability in multiple ways. 1) rogue employees (state-sponsored, corporate espionage) could replace the software. customers could do nothing about it, and might not even be told. 2) sale of the company by the VCs or a Corporate take-over gives no guarantee that what is safe now will be safe in future, no matter what the VCs or the company says right now. 3) whatever expertise "Oxide Computer Company" thinks they have, they're the single-point-of-failure. the larger the number of customers, the less likely that a given vulnerability will be immediately fixed and distributed out. this is just some of the possibilities. sorry to say that there's so many things wrong with this idea it's really hard to hold back and not say anything. now, if the full source code right to the bedrock is available, and the CUSTOMER is given FULL CONTROL, THEN we do not have a problem. by "full control", that includes: * all DRM keys including TPM signing private keys * all peripheral initialisation source code (including DDR4 firmware, PCIe firmware and USB3 firmware) * BMC (Boot Management Console) source code * BIOS source code * Operating system source code * full source code for all tools and toolchains for the above to avoid vendor lock-in and the possibility of the toolchain itself introducing rogue code. this is one hell of a list and it's almost impossible to fulfil with today's "NDA'd proprietary firmware 3rd party licensing" mindset. the only company in this secure server space to my knowledge that's achieved this is Raptor Engineering with the TALOS-II, when running with the Kestrel BMC replacement, on the Lattice ECP5 FPGA.

nojvek 5y ago

I get an “ An unexpected error has occurred.” when I load the site.

ChrisArchitect 5y ago

Sleek AF Pentagram-designed website with a nice balance of style and nerdiness like the ASCII art animations. Can't miss the Halt and Catch Fire tv references including Haley Clark alongsize Woz and other tech legends and in the blog post about the launch a terminal window with character names like Gordon Clark, etc. Love to see it.

unixhero 5y ago

I cannot wait for this. However I also do not see when I will buy a full shelf of gear, even though I would love to. Will they also release a maxi version, micro version and a nano version? Ie. 2U server, PRO workstation and an small formfactor? I think the innovations they have done to these computers deserve to be even more places than just in a massive and awesome data processing rack.

rugwirobaker 5y ago
A small form factor would definitely make sense as test environment.

anarchyrucks 5y ago

Reminds me of "the box" from Silicon Valley comedy series.