6 ms·
At all of the exchanges and trading firms I’ve worked with (granted none in crypto) one of the “must haves” has been a reconciliation system out of band of the
by HolyLampshade 2y ago
At all of the exchanges and trading firms I’ve worked with (granted none in crypto) one of the “must haves” has been a reconciliation system out of band of the trading platforms. In practice one of these almost always belongs to the risk group (this is usually dependent on drop copy), but the other is entirely based on pcaps at the point of contact with every counterparty and positions/trades reconstructed from there.
If any discrepancies are found that persist over some time horizon it can be cause to stop all activity.
- ajb 2y agoWait, pcap as in wireshark packet capture?
- Loic 2y agoI suppose Pre-Calculated Aggregated Positions, but I am not an expert in the field.
- w23j 2y agoI would also really like to know that! It generally seems to be a thing in trading: https://databento.com/pcaps https://databento.com/pcaps There is also this (though this page does not specify what pcap means): https://www.lseg.com/en/data-analytics/market-data/data-feeds/tick-history/tick-history-pcap https://www.lseg.com/en/data-analytics/market-data/data-feed...
- hnbear 2y agoLook up Corvil devices by Pico. Commonly used in finance. https://www.pico.net/corvil-analytics/ https://www.pico.net/corvil-analytics/
- tnlnbn 2y agoI'm not the commenter, but yes, often trading firms record all order gateway traffic to from brokers or exchanges at the TCP/IP packet level, in what are referred to as "pcap files". Awkwardly low-level to work with, but it means you know for sure what you sent, not what your software thought it was sending!
- pclmulqdq 2y agoThe ultimate source of truth about what orders you sent to the exchange is the exact set of bits sent to the exchange. This is very important because your software can have bugs (and so can theirs), so using the packet captures from that wire directly is the only real way to know what really happened.
- generic92034 2y agoBut then the software capturing, storing and displaying the packets can also have bugs.
- bostik 2y agoAmong all the software installed in a reputable Linux system, tcpdump and libpcap are some of the most battle tested pieces one can find. Wireshark has bugs, yes. Mostly in the dissectors and in the UI. But the packet capture itself is through libpcap. Also, to point out the obvious: pcap viewers in turn are auditable if and when necessary.
- wjholden 2y agoCisco switches can mirror ports with a feature called Switch Port Analyzer (SPAN). For a monitored port, one can specify the direction (frames in, out, or both), and the destination port or VLAN. SPAN ports are great for network troubleshooting. They're also nice for security monitors, such as an intrusion detection system. The IDS logically sees traffic "on-line," but completely transparent to users. If the IDS fails, traffic fails open (which wouldn't be acceptable in some circumstances, but it all depends on your priorities).
- generic92034 2y agoWhen I think "Cisco" I think error-free. /s No, really, I get where you and your parent are coming from. It is a low probability. But occasionally there is also thoroughly verified application code out there. That is when you are asking yourself where the error really is. It could be any layer.
- hnbear 2y agoTypically not a literal pcap. Not just wireshsrk running persistently everywhere. There are systems you can buy (eg by Pico) that you mirror all traffic to and they store it, index it, and have pre-configured parsers for a lot of protocols to make querying easier. Think Splunk/ELK for network traffic by packet.
- hn_version_0023 2y agoThe Corvil devices used by Pico have IME largely been replaced by Arista 7130 Metamux platforms at the capture “edge”
- HolyLampshade 2y agoWhich is great for the companies that have made the switch because those corvils were truly terrible.
- cjalmeida 2y agoExcept it is literal “pcap” as they capture all packets at layer 3. I don’t know the exact specifications of Pico appliances, but it would not surprise me they’re running Linux + libpcap + some sort of timeseries DB
- hnbear 2y agoWell, probably, but I meant more like it's not typically someone running tcpdump everywhere and someone analyzing with Wireshark, rather than a systems configured to do this at scale across the desktop.
- kelnos 2y agoI don't think that's what anyone was assuming. A "pcap" is a file format for serialized network packets, not a particular application that generates them.
- HolyLampshade 2y agoLooks like tnlnbn already answered, but the other benefit to having a raw network capture is often this is performed on devices (pico and exablaze just to name two) that provide very precise timestamping on a packet by packet basis, typically as some additional bytes prepended to the header. Most modern trading systems performing competitive high frequency or event trades have performance thesholds in the tens of nanos, and the only place to land at that sort of precision is running analysis on a stable hardware clock.
- SnorkelTan 2y agoLooking at the order messages sent to and received from another trading system was not uncommon when I worked in that neck of the woods
- chairmansteve 2y agoThe crypto firms are moving fast and breaking things. No need for that kind of safety shit, right? Would slow things down. Reminds me of Boeing.
- nobodyandproud 2y agoSo is this capture used to reconstruct FIX messages?
- HolyLampshade 2y agoYeah, FIX or whatever proprietary binary fixed-length protocols (OUCH or BOE for example) the venue uses for order instructions. Some firms will also capture market data (ITCH, PITCH, Pillar Integrated) at the edge of the network at a few different cross connects to help evaluate performance of the exchange’s edge switches or core network.