5 ms·

Indeed. I'd also like to know if there are any alternatives that offer similar performance in an OSS wrapper.

by chton 12y ago

rasur 12y ago
Not sure about OSS offerings, but the larger Arista switches come with FPGAs, which I hear is used by some HFT firms to achieve blinding speeds.
- chton 12y ago
  This is usually the problem, almost every single option I've seen is based on dedicated hardware. This is fine for many, but not for those of us running in cloud hosts and other general-purpose locations.
  - pjc50 12y ago
    You can't possibly have a de-layered system on virtualised cloud hardware! Besides, dedicated hosting for a 1U or 2U rack machine is not expensive. If you have enough traffic that it's worth building your own TCP solution, you already have enough servers that you're spending a considerable amount of money on.
    - chton 12y ago
      I'm aware of that, but if we could reduce dedicated hosting to a single machine through simpler code, we can feed the information from there into a cloud system. It helps a lot if the machine can be a general purpose one, since that reduces hosting costs, maintenance overhead and risk.
      - jamiesonbecker 12y ago
        Definitely -- +1 to general purpose machine and doing this on normal hardware without special NIC's. However, as soon as you dump the OS, the machine would no longer be suitable as a shared tenancy/cloud host. It might be useful for some sort of dedicated service offering (that would be a cool AWS feature and allow things like Vyatta that're tied right into the kernel and SDN), but not for general purpose cloud hosting. The hypervisor/container/OS are still needed to enforce roles, manage resources, etc.
        chton 12y ago
        I agree, though as bcoates indicated, cloud hosts are adapting to this demand. While bare-metal access is indeed very unlikely to happen in a shared tenancy environment, there will be at least some efforts towards lower-level access.
    - bcoates 12y ago
      Sure you can: as a random example, AWS supports SR-IOV today: you can directly communicate with a NIC from a userland linux process if you set everything up right. The general idea is to have hardware with direct virtualization support (which is increasingly available on commodity hardware), then have a 'control plane' of layered, virtualized syscall APIs that configure a 'data plane' of virtualization-aware hardware. Permitted I/O operations occur just as if they were on bare metal, with asymptotically zero performance overhead, because you can process an arbitrary amount of data without invoking any code at the OS/hypervisor/cloud provider layer. For example, my rented, virtualization-aware CPU allows me to run any non-privileged code that stays within a certain block of address space; my rented, virtualization-aware NIC allows me to send and receive any ethernet frames that match certain header bits; and my rented, virtualization-aware disk allows me to read and write to a certain range of LBAs. The nth-layer OS or cloud host or whatever can come in and alter these permissions at will but it need not examine every single syscall to see if it conforms to policy.
  - justincormack 12y ago
    Snabb switch has been working on zero copy drivers for kvm, for openstack. But you are unlikely to see that on generic cloud vms soon. If you want performance you need control of hardware.
- pjc50 12y ago
  I was involved in one of these; interesting project. http://www.argondesign.com/case-studies/2013/sep/18/high-performance-trading/ http://www.argondesign.com/case-studies/2013/sep/18/high-per... The 10G interface is presented as 64 bits of data every clock cycle, so if you can express your algorithm entirely as a byte-by-byte state machine you can parse messages with minimal delay.
  - chton 12y ago
    that is a very interesting project. How does the 170ns latency compare to others? This isn't really my area of expertise :p
    - pjc50 12y ago
      Extremely well; 170ns is the length of the incoming market data packet, and the reply starts immediately on end of incoming packet. It takes more than that just to DMA a packet into main memory from PC gbit interface.
      - chton 12y ago
        That is very cool. Where would you go from there? It seems if you're already so fast that you can start replying as soon as all the data is received, there isn't much more you can do. Or are there techniques for going beyond that?
        pjc50 12y ago
        You can push the start of the reply earlier than the end of the input with some tricks. Beyond that you're limited by causality.
        minimax 12y ago
        The bit of the video where he talks about writing the order to the wire and then flubbing the checksum if he doesn't actually want to send the order was nuts. I work in the industry and I had heard of people using FPGAs on switches but not that particular technique. Thanks for posting that.
mattgodbolt 12y ago
http://www.openonload.org/ http://www.openonload.org/ is an open source kernel bypass stack, although out of the box it only works with (somewhat) expensive Solarflare NICs
- deadgrey19 12y ago
  This stack may be open source, but it is handcrafted to work only with the Solarflare NICs. Having looked at it seriously, we decided that it wasn't worth trying to port this to our own devices and therefore wrote yet another one. This time, without the pretence of "open".
ramLlama 12y ago
Thing is, userspace networking is a lot like the GPU business. Often times, the software is free or even OSS, but the hardware is proprietary. I know Intel has software for their NIC's to enable userpace networking, but I don't think any HFT uses it. It may be all right for hobbyist experimentation. I left the industry about a year ago, so my information is a bit out-of-date, but my former employer used either Solarflare cards with the OpenOnload stack (very good cards and awesome software) or the Mellanox CX series with VMA stack (amazing hardware with mediocre software. VMA is now open-source, so perhaps that situation has improved). Note that these cards are a few thousand dollars each, so out-of-reach for hobbyists.
- lrizzo 12y ago
  I can't comment on HFT because I have no experience on that, and their focus is latency rather than throughput. For the latter (which matters in routers, for instance), netmap runs on everything (either natively or with some emulation). The intel 10g cards (which DPDK of course supports) are around 3-400$ i think, the 1G cards start around a few 10's of dollars (not that you need userspace networking at 1G).
deadgrey19 12y ago
There are one or two. The most prominent is Netmap (https://code.google.com/p/netmap/ https://code.google.com/p/netmap/) another is the Intel Dataplane Development Kit (DPDK - http://www.dpdk.org/ http://www.dpdk.org/. Both of these allow you to handle millions of real packets, per second from standard software. No lua coding required.

Indeed. I'd also like to know if there are any alternatives that offer similar performance in an OSS wrapper.

by chton 12y ago

Indeed. I'd also like to know if there are any alternatives that offer similar performance in an OSS wrapper.

rasur 12y ago

Not sure about OSS offerings, but the larger Arista switches come with FPGAs, which I hear is used by some HFT firms to achieve blinding speeds.

chton 12y ago
This is usually the problem, almost every single option I've seen is based on dedicated hardware. This is fine for many, but not for those of us running in cloud hosts and other general-purpose locations.
- pjc50 12y ago
  You can't possibly have a de-layered system on virtualised cloud hardware! Besides, dedicated hosting for a 1U or 2U rack machine is not expensive. If you have enough traffic that it's worth building your own TCP solution, you already have enough servers that you're spending a considerable amount of money on.
  - chton 12y ago
    I'm aware of that, but if we could reduce dedicated hosting to a single machine through simpler code, we can feed the information from there into a cloud system. It helps a lot if the machine can be a general purpose one, since that reduces hosting costs, maintenance overhead and risk.
    - jamiesonbecker 12y ago
      Definitely -- +1 to general purpose machine and doing this on normal hardware without special NIC's. However, as soon as you dump the OS, the machine would no longer be suitable as a shared tenancy/cloud host. It might be useful for some sort of dedicated service offering (that would be a cool AWS feature and allow things like Vyatta that're tied right into the kernel and SDN), but not for general purpose cloud hosting. The hypervisor/container/OS are still needed to enforce roles, manage resources, etc.
      - chton 12y ago
        I agree, though as bcoates indicated, cloud hosts are adapting to this demand. While bare-metal access is indeed very unlikely to happen in a shared tenancy environment, there will be at least some efforts towards lower-level access.
  - bcoates 12y ago
    Sure you can: as a random example, AWS supports SR-IOV today: you can directly communicate with a NIC from a userland linux process if you set everything up right. The general idea is to have hardware with direct virtualization support (which is increasingly available on commodity hardware), then have a 'control plane' of layered, virtualized syscall APIs that configure a 'data plane' of virtualization-aware hardware. Permitted I/O operations occur just as if they were on bare metal, with asymptotically zero performance overhead, because you can process an arbitrary amount of data without invoking any code at the OS/hypervisor/cloud provider layer. For example, my rented, virtualization-aware CPU allows me to run any non-privileged code that stays within a certain block of address space; my rented, virtualization-aware NIC allows me to send and receive any ethernet frames that match certain header bits; and my rented, virtualization-aware disk allows me to read and write to a certain range of LBAs. The nth-layer OS or cloud host or whatever can come in and alter these permissions at will but it need not examine every single syscall to see if it conforms to policy.
- justincormack 12y ago
  Snabb switch has been working on zero copy drivers for kvm, for openstack. But you are unlikely to see that on generic cloud vms soon. If you want performance you need control of hardware.
pjc50 12y ago
I was involved in one of these; interesting project. http://www.argondesign.com/case-studies/2013/sep/18/high-performance-trading/ http://www.argondesign.com/case-studies/2013/sep/18/high-per... The 10G interface is presented as 64 bits of data every clock cycle, so if you can express your algorithm entirely as a byte-by-byte state machine you can parse messages with minimal delay.
- chton 12y ago
  that is a very interesting project. How does the 170ns latency compare to others? This isn't really my area of expertise :p
  - pjc50 12y ago
    Extremely well; 170ns is the length of the incoming market data packet, and the reply starts immediately on end of incoming packet. It takes more than that just to DMA a packet into main memory from PC gbit interface.
    - chton 12y ago
      That is very cool. Where would you go from there? It seems if you're already so fast that you can start replying as soon as all the data is received, there isn't much more you can do. Or are there techniques for going beyond that?
      - pjc50 12y ago
        You can push the start of the reply earlier than the end of the input with some tricks. Beyond that you're limited by causality.
        minimax 12y ago
        The bit of the video where he talks about writing the order to the wire and then flubbing the checksum if he doesn't actually want to send the order was nuts. I work in the industry and I had heard of people using FPGAs on switches but not that particular technique. Thanks for posting that.

mattgodbolt 12y ago

http://www.openonload.org/ http://www.openonload.org/ is an open source kernel bypass stack, although out of the box it only works with (somewhat) expensive Solarflare NICs

deadgrey19 12y ago
This stack may be open source, but it is handcrafted to work only with the Solarflare NICs. Having looked at it seriously, we decided that it wasn't worth trying to port this to our own devices and therefore wrote yet another one. This time, without the pretence of "open".

ramLlama 12y ago

Thing is, userspace networking is a lot like the GPU business. Often times, the software is free or even OSS, but the hardware is proprietary. I know Intel has software for their NIC's to enable userpace networking, but I don't think any HFT uses it. It may be all right for hobbyist experimentation. I left the industry about a year ago, so my information is a bit out-of-date, but my former employer used either Solarflare cards with the OpenOnload stack (very good cards and awesome software) or the Mellanox CX series with VMA stack (amazing hardware with mediocre software. VMA is now open-source, so perhaps that situation has improved). Note that these cards are a few thousand dollars each, so out-of-reach for hobbyists.

lrizzo 12y ago
I can't comment on HFT because I have no experience on that, and their focus is latency rather than throughput. For the latter (which matters in routers, for instance), netmap runs on everything (either natively or with some emulation). The intel 10g cards (which DPDK of course supports) are around 3-400$ i think, the 1G cards start around a few 10's of dollars (not that you need userspace networking at 1G).

deadgrey19 12y ago

There are one or two. The most prominent is Netmap (https://code.google.com/p/netmap/ https://code.google.com/p/netmap/) another is the Intel Dataplane Development Kit (DPDK - http://www.dpdk.org/ http://www.dpdk.org/. Both of these allow you to handle millions of real packets, per second from standard software. No lua coding required.