4 ms·

I think this is a neat development, but doesn't this post overstate the case for reduction in complexity? I hope a Racker will correct me if I'm wrong, but: -

I think this is a neat development, but doesn't this post overstate the case for reduction in complexity? I hope a Racker will correct me if I'm wrong, but: - With the exception of the boot device, you're dependent on non-local storage over the network. - You're not going to get dedicated networking gear - OnMetal is still dependent on the virtual networking layer (OVS?) I can appreciate having more bang for the buck, but there are enough shared components here that I would be very hesitant to assume that that MTBF crosses the imaginary line where you can eschew fault-tolerant architecture patterns.

lnanek2 12y ago
What makes you think it is non-local? Two of the options are RAM only (32GB and 512GB). That's like memory chips plugged into the motherboard (what modern normal user computers often only have 2 to 4GB of). You would run a RAM disk if your software needed something more than the boot drive to run using that. That's why the numbers are so large. The other option is PCIe Flash, that's a solid state "disk" plugged into the motherboard expansion slot, basically. Again since that flat out state PCIe, they are saying it is local.
- lobster_johnson 12y ago
  Are you saying it's appropriate to run a database on a RAM disk? This product is offered without a disk, so it has to be coupled with a SAN or similar remote storage system.
  - russell_h 12y ago
    Our IO flavor has 2x 1.6TB PCIe flash cards, this is how we see most users running databases or otherwise persisting "hot" data. I've gotten to play with these over the last few months, and its really incredible how fast they are - they totally crush even high end SSDs. People have traditionally used these in a tiered setup to accelerate spinning disks, but the costs have come down to where it hardly makes sense to use anything else for low latency persistent storage. We're working to hook this up to our existing block storage offering as well, which will offer more flexibility to attach SSD or spinning disk backed block devices to any flavor.
    - lobster_johnson 12y ago
      Can you recommend a specific brand of PCIe flash card? How reliable are they?
      - russell_h 12y ago
        We are running these in OnMetal: http://www.lsi.com/products/flash-accelerators/pages/nytro-warpdrive-blp4-1600.aspx http://www.lsi.com/products/flash-accelerators/pages/nytro-w... Reliability is similar to an SSD. The underlying storage tech is basically the same: some flash chips, with a controller that handles wear leveling and so on. Just that instead of running operations through a SATA controller it uses the PCIe bus, which offers higher bandwidth and lower latency. Interestingly, Apple is shipping PCIe flash in all (I think) of their latest laptops.
        lobster_johnson 12y ago
        Oh, I see, just plain SSDs with PCIe instead of SATA. Yes, that seems logical.
jayofdoom 12y ago
For the network, you're getting 2x 10Gbit connections, configured in an HA-Bond, and plugged into a real switch. There's no current dependency on a virtual networking layer.
- lambdaops 12y ago
  How do IPs get assigned? Is that still over nova network?
- oasisbob 12y ago
  That's good to know. How and where are ACLs handled?
  - russell_h 12y ago
    Engineering manager on OnMetal here. Our first iteration, you'll get two networks: "ServiceNet" is our intra-DC network, which is unmetered (ie, free bandwidth) and "PublicNet" which is the internet and will be billed just like we bill for other cloud server bandwidth. Collectively you get 10Gb/s across these, which you can break down however you want. To you these just show up as VLANs which will be configured on build. We're working on hooking these boxes up to our "Cloud Networks", which are per-tenant software defined networks. If this sounds like a fun project, hit me up, contact info is in my profile.

lnanek2 12y ago

What makes you think it is non-local? Two of the options are RAM only (32GB and 512GB). That's like memory chips plugged into the motherboard (what modern normal user computers often only have 2 to 4GB of). You would run a RAM disk if your software needed something more than the boot drive to run using that. That's why the numbers are so large. The other option is PCIe Flash, that's a solid state "disk" plugged into the motherboard expansion slot, basically. Again since that flat out state PCIe, they are saying it is local.

lobster_johnson 12y ago
Are you saying it's appropriate to run a database on a RAM disk? This product is offered without a disk, so it has to be coupled with a SAN or similar remote storage system.
- russell_h 12y ago
  Our IO flavor has 2x 1.6TB PCIe flash cards, this is how we see most users running databases or otherwise persisting "hot" data. I've gotten to play with these over the last few months, and its really incredible how fast they are - they totally crush even high end SSDs. People have traditionally used these in a tiered setup to accelerate spinning disks, but the costs have come down to where it hardly makes sense to use anything else for low latency persistent storage. We're working to hook this up to our existing block storage offering as well, which will offer more flexibility to attach SSD or spinning disk backed block devices to any flavor.
  - lobster_johnson 12y ago
    Can you recommend a specific brand of PCIe flash card? How reliable are they?
    - russell_h 12y ago
      We are running these in OnMetal: http://www.lsi.com/products/flash-accelerators/pages/nytro-warpdrive-blp4-1600.aspx http://www.lsi.com/products/flash-accelerators/pages/nytro-w... Reliability is similar to an SSD. The underlying storage tech is basically the same: some flash chips, with a controller that handles wear leveling and so on. Just that instead of running operations through a SATA controller it uses the PCIe bus, which offers higher bandwidth and lower latency. Interestingly, Apple is shipping PCIe flash in all (I think) of their latest laptops.
      - lobster_johnson 12y ago
        Oh, I see, just plain SSDs with PCIe instead of SATA. Yes, that seems logical.

jayofdoom 12y ago

For the network, you're getting 2x 10Gbit connections, configured in an HA-Bond, and plugged into a real switch. There's no current dependency on a virtual networking layer.

lambdaops 12y ago
How do IPs get assigned? Is that still over nova network?
oasisbob 12y ago
That's good to know. How and where are ACLs handled?
- russell_h 12y ago
  Engineering manager on OnMetal here. Our first iteration, you'll get two networks: "ServiceNet" is our intra-DC network, which is unmetered (ie, free bandwidth) and "PublicNet" which is the internet and will be billed just like we bill for other cloud server bandwidth. Collectively you get 10Gb/s across these, which you can break down however you want. To you these just show up as VLANs which will be configured on build. We're working on hooking these boxes up to our "Cloud Networks", which are per-tenant software defined networks. If this sounds like a fun project, hit me up, contact info is in my profile.