3 ms·

If you don’t mind a digression for someone who needs help… What is the architecture pattern of a trading platform? I am looking to build a system that is able

by adampk 5y ago

If you don’t mind a digression for someone who needs help… What is the architecture pattern of a trading platform? I am looking to build a system that is able to: - receive 1000s incoming streams of data - save the data - make data available to live subscribers The closest analogous system I can think of is bond/stock/commodity/etc price subscriptions for traders. I feel like this many in - many out data stream architecture should be solved by now and I rather not start from scratch in architecture and technology choices. I can’t find the right phrase to even google to get started.

tikkabhuna 5y ago
I assume your live subscribers are external people over the internet? The trading platforms I've worked on tend to order things into a single stream that you can act upon. This helps with testing, race conditions, auditability. You will want to be able to replay an exact series of events to recreate conditions. LMAX Disruptor[1] and Aeron[2] are two open source examples of something widely used, either using the libraries themselves or as concepts. Some things that trading systems (generally?) need which you may not, which might simplify your architecture: - Every message must be delivered and in order. UDP is usually used over TCP, as the platform will likely want more control over handling missed messages. - It will be common to have read/write applications that need to add a message as a response. Writing is difficult as you need to be quick, otherwise a subsequent message from another application might invalidate your message. Do subscribers need older data? How do dropped packets impact the system? Can they just be forgotten? What latency requirements do you have? Do subscribers also write to the same stream? - [1] https://lmax-exchange.github.io/disruptor/ https://lmax-exchange.github.io/disruptor/ - [2] https://github.com/real-logic/aeron https://github.com/real-logic/aeron
- adampk 5y ago
  This is great thank you much appreciated!

by adampk 5y ago

tikkabhuna 5y ago

I assume your live subscribers are external people over the internet? The trading platforms I've worked on tend to order things into a single stream that you can act upon. This helps with testing, race conditions, auditability. You will want to be able to replay an exact series of events to recreate conditions. LMAX Disruptor[1] and Aeron[2] are two open source examples of something widely used, either using the libraries themselves or as concepts. Some things that trading systems (generally?) need which you may not, which might simplify your architecture: - Every message must be delivered and in order. UDP is usually used over TCP, as the platform will likely want more control over handling missed messages. - It will be common to have read/write applications that need to add a message as a response. Writing is difficult as you need to be quick, otherwise a subsequent message from another application might invalidate your message. Do subscribers need older data? How do dropped packets impact the system? Can they just be forgotten? What latency requirements do you have? Do subscribers also write to the same stream? - [1] https://lmax-exchange.github.io/disruptor/ https://lmax-exchange.github.io/disruptor/ - [2] https://github.com/real-logic/aeron https://github.com/real-logic/aeron

adampk 5y ago
This is great thank you much appreciated!