3 ms·

Properly tuned NFS should be extremely fast... I've managed NFS clusters for HPC that exceeded 800Mbit/s and 20k IOPS

• sytse 11y ago
  The problem is that we should distribute the load over multiple servers https://gitlab.com/gitlab-com/operations/issues/1 https://gitlab.com/gitlab-com/operations/issues/1 Right now the lonely repository server has over 200k context switches per second.
  • matheweis 11y ago
    Your engineers probably know this, but CephFS is not well known for handing high IOPS loads; make sure to architect accordingly :)
    • sytse 11y ago
      Thanks for the advice, do you maybe have a link with more details?
      • matheweis 11y ago
        Mirantis gave a talk last year at OpenStack with real-world numbers... They had to scale to 500 storage nodes to achieve 50k IOPS aggregate. https://www.openstack.org/summit/vancouver-2015/summit-videos/presentation/building-your-first-ceph-cluster-for-openstack-fighting-for-performance-solving-tradeoffs https://www.openstack.org/summit/vancouver-2015/summit-video... https://www.slideshare.net/mobile/mirantis/ceph-talk-vancouver-20 https://www.slideshare.net/mobile/mirantis/ceph-talk-vancouv...
        • sytse 11y ago
          Wow, thanks for those numbers and the links, I've added them to https://gitlab.com/gitlab-com/operations/issues/1#note_4483625 https://gitlab.com/gitlab-com/operations/issues/1#note_44836...