7 ms·
Designing a Low Latency 10G Ethernet Core (2023)
- Neywiny 1y agoIt seems fun to be a high frequency FPGA trader designer. All my FPGA is much lower power consumption so I don't get to play with stuff like gigs of external SRAM or the QDR stuff or whatnot
- typpilol 1y agoI feel like it's also got to be super stressful Imagine getting a call you system is down and is costing them millions per hour
- toast0 1y agoOn the other hand, no nights or weekends.
- hamonrye 1y agoPoE is the way to use low voltage in electronics.
- noitpmeder 1y agoThis is becoming less and less true every day. 24/7 markets are coming whether you want them or not, and if you're not trading outside of US Equity hours you're leaving significant money on the table.
- dfex 1y agoGenuinely curious - would 24x7 low-latency trading matter then? Wouldn't after-hours trading (for your local timezone) be happening on an exchange in another location? Or do you mean dark pools/private exchanges that may run 24x7?
- Galanwe 1y ago> This is becoming less and less true every day. 24/7 markets are coming whether you want them or not This has still to exist for real. Every couple of years there is a resurgence of "we should have 24/7 equity markets", followed by milestone announcements of Nasdaq. The reality is more... nuanced. First, there has to be a wide enough window for corporate actions and news dissemination to happen. Not only is that a regulatory requirement, but anyway no sane investor would like to trade in such a window even if possible. Second, there are already early sessions, pre opening sessions, and late sessions on most American markets. The liquidity there is almost inexistant, so I wouldn't say there is a huge demand for extended trading hours by non-retail participants. And last, half the liquidity of US markets is not on lit books, and half the remaining lit liquidity is close to the auction. I don't think it would even be a net profit for lit exchanges to extend their hours in these conditions. As for dark pool, I think only 2 or 3 of the smaller ones are 24/7, like the one from IB. > if you're not trading outside of US Equity hours you're leaving significant money on the table HFTs need a lot of liquidity and tight spreads, that's pretty much only doable at scale in the US and Japan. Europe and EM are either not liquid enough, or only really able to sustain low capital ad-hoc strategies.
- ta1243 1y agoI'd want to know why they had such a system as a single point of failure
- kakacik 1y agoNo business is designed ideally. Now do you want the job or we move to another one who will accept this?
- ta1243 1y agoFortunately in important industries that attitude doesn't fly. Imagine a single person making a single mistake causing a building to collapse? Or a nuclear plant to meltdown.
- hshdhdhehd 1y agoFailure modes can be a bug making a bad trade while doing so at 100% uptime.
- zorked 1y agoThe way to think is the opposite: your system is making millions per hour.
- hsbauauvhabzb 1y agoNo the way to think is ‘it’s down less than our closest competitors system’
- deleted 1y ago[deleted]
- ktm5j 1y agoWhat about when something goes wrong because you overlooked some detail?
- jacquesm 1y agoShit happens. I wrote the firmware for a multi-link HDLC card which earned vast amounts of money for the company that commissioned it. I was quite aware of my responsibility and so was the hardware team. We stress tested the crap out of it before releasing to production and fortunately it never locked up when it was in actual use but we had some pretty wild and very rare (so hard to trigger and reproduce) bugs which delayed deployment considerably. But none of this gave me the kind of feeling that working on fuel estimation software for aircraft gave. That's when there are in the most literal sense lives on the line, and that's a completely different kind of pressure. You simply can not fuck up. It also really rammed home the value of code review and having a good specification so that you can ensure that within the envelope of input parameters your software does what it is supposed to do.
- drmpeg 1y agoMulti-link HDLC. That brings me back to 1985 and the Intel 8274, Zilog Z8530 and the Western Digital WD2511 (that implemented the layer 2 protocol in silicon).
- gosub100 1y agoYou can code a failsafe, or what they call in other industries, an interlock. If an order exceeds some sane limit, or a loss is incurred that exceeds a limit, or the limit-checker fails to proactively ping the interlock, then an independent mechanism shuts it down. This is how, say, a modern x-ray machine prevents itself from failing in a way that radiation gets stuck on.
- mungoman2 1y agoI agree! Seems like very interesting work.
- amelius 1y ago> It seems fun to be a high frequency FPGA trader designer. Yes, if you think that doing pointless work is fun. What do you tell your grandchildren?
- le-mark 1y agoHigh frequency trading is far from the only pointless activity software and hardware developers are employed to do. At least in that realm they are pushing the state of the art, unlike presenting ads or yet another crud app.
- amelius 1y agoPushing the state of the art? That's only meaningful if the results are published. But I suspect that everything is kept secret. Come on, it is not difficult to find better things to do with your brains. Like work in medicine, etc.
- OneDeuxTriSeiGo 1y agoYou do realise that Jane Street (one of the big HFT market makers) publishes basically all their technical work and contributes to massive amounts of open source software? They are to my knowledge one of the main contributors to the OCaml ecosystem as well.
- jbstack 1y agoIt's a bit unfair to call it pointless. Empirical studies have shown that HFT increase liquidity in markets, leading to narrowed spreads, market depth, faster execution speeds, cross-market informational efficiency, etc. All of these are generally beneficial to other non-HFT market participants. Now you might counter that it also brings some negatives (e.g. fleeting liquidity, market manipulation risk, etc.) and there's certainty some validity to that. But none of this supports an argument that it's pointless. More generally, activities aren't pointless if you are earning money from it which pays your rent, puts food on your table, etc. A large portion of the world works mainly for this goal. It's perfectly valid to tell your grandchildren "I spent my life making sure we'd have enough money for you and your parents to live comfortably", and we shouldn't look down on someone who does that.
- userbinator 1y agoless than 60ns loopback latency There are some Ethernet switches with 4ns latency, and those do more than just sending and receiving, so there's clearly still an order of magnitude of improvement still available. 4ns is basically ~40 cycles of the bit clock for 10G Ethernet.
- mrlongroots 1y agoThe other funny bit is that one-way PCIe latency is 250ns-ish (don't quote me on the exact numbers), which imposes a hard 1us constraint on latency between two hosts.
- mgrosvenor 1y agoYou can go quicker with CXL, but not by much.
- foobar10000 1y agoThere are always DPUs like NextSilicon, NVidia's Blue*, etc - at under 100 ns of SoF to fast compute though.
- rasz 1y agoCut Through switching needs what, 14 bytes? Thats ~12ns on 10G? How would you do it in 40 cycles?
- Perz1val 1y agoNot my are of expertise, but afaik the neat part is that you don't do it in cycles. The "program" lives as the transistor layout, not as instructions
- wtallis 1y agoThe cycles in question are not those of any processor, but those of the signal coming in the wire. No matter how it's implemented, a switch cannot react to bits that are still in the wire and haven't made it to the switch yet.
- throwaway2037 1y agoLinkedIn tells me: https://uk.linkedin.com/in/ttchisholm https://uk.linkedin.com/in/ttchisholm FPGA engineer with a focus on ultra-low latency networking at Jane Street. Yikes.
- anthonj 1y agoNot sure i get the implication, something wrong with the company?
- lloydatkinson 1y agoSome people just love dropping “yikes” type comments and then never explaining themselves.
- lloydatkinson 1y agoA few of those same cringe “yikes” users appear to have downvoted my comment too. Truly a strange bunch.
- exogenousdata 1y agoI’m not the parent, but I’d assume that this was an impressed ‘yikes.’ While Jane is not the most well known firm for ultra low latency trading, an FPGA engineer from that firm could be considered an expert practitioner in the field.
- dev_hugepages 1y agoIs this really unexpected? A person who knows ultra-low-latency networking with FPGAs writes an article about making low-latency networking hardware for a company that needs high-speed, low-latency communications (high-frequency trading)
- up2isomorphism 1y agoI can see the guy is bored to death at Jane Street.
- ksec 1y ago> the design has less than 60ns loopback latency What is the current state of the art figures?
- tripdout 1y agoThat's so cool.