4 ms·

Not sure about OSS offerings, but the larger Arista switches come with FPGAs, which I hear is used by some HFT firms to achieve blinding speeds.

by rasur 12y ago

chton 12y ago
This is usually the problem, almost every single option I've seen is based on dedicated hardware. This is fine for many, but not for those of us running in cloud hosts and other general-purpose locations.
- pjc50 12y ago
  You can't possibly have a de-layered system on virtualised cloud hardware! Besides, dedicated hosting for a 1U or 2U rack machine is not expensive. If you have enough traffic that it's worth building your own TCP solution, you already have enough servers that you're spending a considerable amount of money on.
  - chton 12y ago
    I'm aware of that, but if we could reduce dedicated hosting to a single machine through simpler code, we can feed the information from there into a cloud system. It helps a lot if the machine can be a general purpose one, since that reduces hosting costs, maintenance overhead and risk.
    - jamiesonbecker 12y ago
      Definitely -- +1 to general purpose machine and doing this on normal hardware without special NIC's. However, as soon as you dump the OS, the machine would no longer be suitable as a shared tenancy/cloud host. It might be useful for some sort of dedicated service offering (that would be a cool AWS feature and allow things like Vyatta that're tied right into the kernel and SDN), but not for general purpose cloud hosting. The hypervisor/container/OS are still needed to enforce roles, manage resources, etc.
      - chton 12y ago
        I agree, though as bcoates indicated, cloud hosts are adapting to this demand. While bare-metal access is indeed very unlikely to happen in a shared tenancy environment, there will be at least some efforts towards lower-level access.
  - bcoates 12y ago
    Sure you can: as a random example, AWS supports SR-IOV today: you can directly communicate with a NIC from a userland linux process if you set everything up right. The general idea is to have hardware with direct virtualization support (which is increasingly available on commodity hardware), then have a 'control plane' of layered, virtualized syscall APIs that configure a 'data plane' of virtualization-aware hardware. Permitted I/O operations occur just as if they were on bare metal, with asymptotically zero performance overhead, because you can process an arbitrary amount of data without invoking any code at the OS/hypervisor/cloud provider layer. For example, my rented, virtualization-aware CPU allows me to run any non-privileged code that stays within a certain block of address space; my rented, virtualization-aware NIC allows me to send and receive any ethernet frames that match certain header bits; and my rented, virtualization-aware disk allows me to read and write to a certain range of LBAs. The nth-layer OS or cloud host or whatever can come in and alter these permissions at will but it need not examine every single syscall to see if it conforms to policy.
- justincormack 12y ago
  Snabb switch has been working on zero copy drivers for kvm, for openstack. But you are unlikely to see that on generic cloud vms soon. If you want performance you need control of hardware.
pjc50 12y ago
I was involved in one of these; interesting project. http://www.argondesign.com/case-studies/2013/sep/18/high-performance-trading/ http://www.argondesign.com/case-studies/2013/sep/18/high-per... The 10G interface is presented as 64 bits of data every clock cycle, so if you can express your algorithm entirely as a byte-by-byte state machine you can parse messages with minimal delay.
- chton 12y ago
  that is a very interesting project. How does the 170ns latency compare to others? This isn't really my area of expertise :p
  - pjc50 12y ago
    Extremely well; 170ns is the length of the incoming market data packet, and the reply starts immediately on end of incoming packet. It takes more than that just to DMA a packet into main memory from PC gbit interface.
    - chton 12y ago
      That is very cool. Where would you go from there? It seems if you're already so fast that you can start replying as soon as all the data is received, there isn't much more you can do. Or are there techniques for going beyond that?
      - pjc50 12y ago
        You can push the start of the reply earlier than the end of the input with some tricks. Beyond that you're limited by causality.
        minimax 12y ago
        The bit of the video where he talks about writing the order to the wire and then flubbing the checksum if he doesn't actually want to send the order was nuts. I work in the industry and I had heard of people using FPGAs on switches but not that particular technique. Thanks for posting that.

Not sure about OSS offerings, but the larger Arista switches come with FPGAs, which I hear is used by some HFT firms to achieve blinding speeds.

by rasur 12y ago

Not sure about OSS offerings, but the larger Arista switches come with FPGAs, which I hear is used by some HFT firms to achieve blinding speeds.

chton 12y ago

This is usually the problem, almost every single option I've seen is based on dedicated hardware. This is fine for many, but not for those of us running in cloud hosts and other general-purpose locations.

pjc50 12y ago
You can't possibly have a de-layered system on virtualised cloud hardware! Besides, dedicated hosting for a 1U or 2U rack machine is not expensive. If you have enough traffic that it's worth building your own TCP solution, you already have enough servers that you're spending a considerable amount of money on.
- chton 12y ago
  I'm aware of that, but if we could reduce dedicated hosting to a single machine through simpler code, we can feed the information from there into a cloud system. It helps a lot if the machine can be a general purpose one, since that reduces hosting costs, maintenance overhead and risk.
  - jamiesonbecker 12y ago
    Definitely -- +1 to general purpose machine and doing this on normal hardware without special NIC's. However, as soon as you dump the OS, the machine would no longer be suitable as a shared tenancy/cloud host. It might be useful for some sort of dedicated service offering (that would be a cool AWS feature and allow things like Vyatta that're tied right into the kernel and SDN), but not for general purpose cloud hosting. The hypervisor/container/OS are still needed to enforce roles, manage resources, etc.
    - chton 12y ago
      I agree, though as bcoates indicated, cloud hosts are adapting to this demand. While bare-metal access is indeed very unlikely to happen in a shared tenancy environment, there will be at least some efforts towards lower-level access.
- bcoates 12y ago
  Sure you can: as a random example, AWS supports SR-IOV today: you can directly communicate with a NIC from a userland linux process if you set everything up right. The general idea is to have hardware with direct virtualization support (which is increasingly available on commodity hardware), then have a 'control plane' of layered, virtualized syscall APIs that configure a 'data plane' of virtualization-aware hardware. Permitted I/O operations occur just as if they were on bare metal, with asymptotically zero performance overhead, because you can process an arbitrary amount of data without invoking any code at the OS/hypervisor/cloud provider layer. For example, my rented, virtualization-aware CPU allows me to run any non-privileged code that stays within a certain block of address space; my rented, virtualization-aware NIC allows me to send and receive any ethernet frames that match certain header bits; and my rented, virtualization-aware disk allows me to read and write to a certain range of LBAs. The nth-layer OS or cloud host or whatever can come in and alter these permissions at will but it need not examine every single syscall to see if it conforms to policy.
justincormack 12y ago
Snabb switch has been working on zero copy drivers for kvm, for openstack. But you are unlikely to see that on generic cloud vms soon. If you want performance you need control of hardware.

pjc50 12y ago

I was involved in one of these; interesting project. http://www.argondesign.com/case-studies/2013/sep/18/high-performance-trading/ http://www.argondesign.com/case-studies/2013/sep/18/high-per... The 10G interface is presented as 64 bits of data every clock cycle, so if you can express your algorithm entirely as a byte-by-byte state machine you can parse messages with minimal delay.

chton 12y ago
that is a very interesting project. How does the 170ns latency compare to others? This isn't really my area of expertise :p
- pjc50 12y ago
  Extremely well; 170ns is the length of the incoming market data packet, and the reply starts immediately on end of incoming packet. It takes more than that just to DMA a packet into main memory from PC gbit interface.
  - chton 12y ago
    That is very cool. Where would you go from there? It seems if you're already so fast that you can start replying as soon as all the data is received, there isn't much more you can do. Or are there techniques for going beyond that?
    - pjc50 12y ago
      You can push the start of the reply earlier than the end of the input with some tricks. Beyond that you're limited by causality.
      - minimax 12y ago
        The bit of the video where he talks about writing the order to the wire and then flubbing the checksum if he doesn't actually want to send the order was nuts. I work in the industry and I had heard of people using FPGAs on switches but not that particular technique. Thanks for posting that.