6 ms·

It seems they're using xnbd to provide the storage for the box. On Debian, the box is running a xnbd-client, which provides the rootfs: xnbd-client --blocksi

It seems they're using xnbd to provide the storage for the box. On Debian, the box is running a xnbd-client, which provides the rootfs: xnbd-client --blocksize 4096 --retry=900 10.1.32.197 4096 /dev/nbd0 root@scw-6d2eff:~# df -h Filesystem Size Used Avail Use% Mounted on /dev/nbd0 46G 507M 44G 2% / I did not check, but I hope there's some filtering/authentication for that process. Like EC2 and others, the box has an internal ip allocated and an external ip via nat (or something else). I was able to download a file from OVH with ~500 Mbit/s throughput when writing to the filesystem. Writing to /dev/null it increases to ~650 Mbit/s. Running an untuned nginx, it can do ~300 req/s with ~20% cpu util. Seeing as the storage is still virtual, I'm not really sure why this would be so much better than a virtual machine. I like the idea though.

alexchamberlain 12y ago
Is it as simple as storing the backing file on an SSD?
- mrsirduke 12y ago
  The documentation has the following to say on the subject: > Each server has access to a pool of local drives. These drives are exported to servers via the NBD protocol which effectively makes them network drives. However, network between these drives and C1 nodes are dedicated PCB tracks which ensures minimal latency and avoids network congestion. > There is no redundancy on these volumes, you need to handle redundancy on your side! They are archieved to a permanent storage when you start and stop your server. > Local volumes are 100% SSD drives which are able to deliver a lot of IOPS and are perfect for random read/write patterns. The maximum size of LSSD volumes is 150GB. Found on https://www.scaleway.com/faq/server#local_volumes https://www.scaleway.com/faq/server#local_volumes
  - ponytech 12y ago
    Do you think that would be acceptable for a database under heavy load, compared to a true dedicated server? Does anyone has tested this product and can share some benchmark?
    - fweespeech 12y ago
      As long as you build it from source and realize the limited power of ARM, yes. I wouldn't trust the packages for ARM because there are variations. Even GoLang doesn't work on all ARM processors, only some! True dedicated servers would be much more powerful, and frankly, this is a service form Online.NET. http://www.online.net/en/dedicated-server/dedibox-classic http://www.online.net/en/dedicated-server/dedibox-classic For 3x the price, you can get a real Xeon processor [that will likely perform equivalent or better to 3x scaleway nodes] and 8x the RAM. At this price point, I don't see the appeal outside of the object storage.
      - 4ad 12y ago
        Go works on all ARM processors that use ARMv5 or newer. Today it would be pretty hard to find an ARMv4 CPU.
regularfry 12y ago
xnbd is an interesting choice, if they're using it for the server. In theory it can do live-migration, but the live-migration scheme it uses has (or had, it's been a while since I looked) a bit of a flaw in that it could lose writes. It'll be doing better than a typical virtual machine because there's no contention on the host side, kernel or IO, and at this point, you've probably got the storage device to yourself so there's no bus contention. The trade-off is shutdown time: they've got to slurp your disc image off to their block store before they can release your node back to the pool.
seunosewa 12y ago
So their "no noisy neighbours" claim is essentially incorrect. (Poor IO due to shared storage is the main performance issue with virtual servers.)
- pquerna 12y ago
  When you use a remote blockstore like they are as your storage, you also generally are always booting via PXE, and so far have a provider controlled kernel. I used to work on Rackspace OnMetal: http://www.rackspace.com/cloud/servers/onmetal http://www.rackspace.com/cloud/servers/onmetal https://journal.paul.querna.org/articles/2014/07/02/putting-teeth-in-our-public-cloud/ https://journal.paul.querna.org/articles/2014/07/02/putting-... In OnMetal this not the case -- you get local storage, you have your local kernel that you can re-install or do whatever you want to -- and if we have a control plane outage, you can reboot -- there is no dependency once your server is created on a provider hosted PXE server. OnMetal isn't cheapo-ARM servers, but it is possible to build a "bare metal" cloud, which is truly bare metal, like you would normally get, and no remote block stores for your root FS.
  - mwcampbell 12y ago
    Do you think it would be feasible to provide a bare-metal cloud made of cheap ARM or Atom servers while giving each server its own independent SSD?
    - pquerna 12y ago
      Yes, most definately. OpenStack Ironic[1] can do this today, given you had right hardware. Making that hardware "mutli-tenant" is not a trivial project, mostly because the firmware stacks are still very closed, but it can be done. Top of Rack switches are also another area of complexity/pain, but also feasible to overcome with a little engineering. [1]: OpenStack has its complexities, but the OnMetal project has upstreamed everything feasible we can.
      - andrewsomething 12y ago

alexchamberlain 12y ago

Is it as simple as storing the backing file on an SSD?

mrsirduke 12y ago
The documentation has the following to say on the subject: > Each server has access to a pool of local drives. These drives are exported to servers via the NBD protocol which effectively makes them network drives. However, network between these drives and C1 nodes are dedicated PCB tracks which ensures minimal latency and avoids network congestion. > There is no redundancy on these volumes, you need to handle redundancy on your side! They are archieved to a permanent storage when you start and stop your server. > Local volumes are 100% SSD drives which are able to deliver a lot of IOPS and are perfect for random read/write patterns. The maximum size of LSSD volumes is 150GB. Found on https://www.scaleway.com/faq/server#local_volumes https://www.scaleway.com/faq/server#local_volumes
- ponytech 12y ago
  Do you think that would be acceptable for a database under heavy load, compared to a true dedicated server? Does anyone has tested this product and can share some benchmark?
  - fweespeech 12y ago
    As long as you build it from source and realize the limited power of ARM, yes. I wouldn't trust the packages for ARM because there are variations. Even GoLang doesn't work on all ARM processors, only some! True dedicated servers would be much more powerful, and frankly, this is a service form Online.NET. http://www.online.net/en/dedicated-server/dedibox-classic http://www.online.net/en/dedicated-server/dedibox-classic For 3x the price, you can get a real Xeon processor [that will likely perform equivalent or better to 3x scaleway nodes] and 8x the RAM. At this price point, I don't see the appeal outside of the object storage.
    - 4ad 12y ago
      Go works on all ARM processors that use ARMv5 or newer. Today it would be pretty hard to find an ARMv4 CPU.

regularfry 12y ago

xnbd is an interesting choice, if they're using it for the server. In theory it can do live-migration, but the live-migration scheme it uses has (or had, it's been a while since I looked) a bit of a flaw in that it could lose writes. It'll be doing better than a typical virtual machine because there's no contention on the host side, kernel or IO, and at this point, you've probably got the storage device to yourself so there's no bus contention. The trade-off is shutdown time: they've got to slurp your disc image off to their block store before they can release your node back to the pool.

seunosewa 12y ago

So their "no noisy neighbours" claim is essentially incorrect. (Poor IO due to shared storage is the main performance issue with virtual servers.)

pquerna 12y ago
When you use a remote blockstore like they are as your storage, you also generally are always booting via PXE, and so far have a provider controlled kernel. I used to work on Rackspace OnMetal: http://www.rackspace.com/cloud/servers/onmetal http://www.rackspace.com/cloud/servers/onmetal https://journal.paul.querna.org/articles/2014/07/02/putting-teeth-in-our-public-cloud/ https://journal.paul.querna.org/articles/2014/07/02/putting-... In OnMetal this not the case -- you get local storage, you have your local kernel that you can re-install or do whatever you want to -- and if we have a control plane outage, you can reboot -- there is no dependency once your server is created on a provider hosted PXE server. OnMetal isn't cheapo-ARM servers, but it is possible to build a "bare metal" cloud, which is truly bare metal, like you would normally get, and no remote block stores for your root FS.
- mwcampbell 12y ago
  Do you think it would be feasible to provide a bare-metal cloud made of cheap ARM or Atom servers while giving each server its own independent SSD?
  - pquerna 12y ago
    Yes, most definately. OpenStack Ironic[1] can do this today, given you had right hardware. Making that hardware "mutli-tenant" is not a trivial project, mostly because the firmware stacks are still very closed, but it can be done. Top of Rack switches are also another area of complexity/pain, but also feasible to overcome with a little engineering. [1]: OpenStack has its complexities, but the OnMetal project has upstreamed everything feasible we can.
    - andrewsomething 12y ago