9 ms·
How to Build an Exchange (2017)
- samwillis 4y agoThe context here is that Sam Bankman-Fried of FTX and a number of the other executives (Caroline Ellison and Brett Harrison) were previously traders at Jane Street. This video is from 2017, a couple of years prior to the FTX launch. It makes you wander if any of them were in attendance.
- ackbar03 4y agoSbf was playing LoL at the same time
- NkVczPkybiXICG 4y agoFTX didn’t make any of the architectural decisions presented in this talk.
- dcolkitt 4y agoYeah, as an exchange FTX was pretty terrible. The matching engine was notorious for being high latency, unreliable, and having all sorts of undocumented issues. Every algo trader there will tell you horror stories about trying to cancel an open order, getting a cancel confirmation, then getting notified that the order was executed. Sometimes seconds later. What really gave FTX a competitive advantage was its cross-margining system. It was super easy to post collateral in any supported asset, and all open positions were automatically margined against each other. So if you were doing something like pairs trades between spot and futures, you didn't need to constantly monitor and manually move capital between the silos. In hindsight though it's not clear if this was possible because of better engineering, or just because Alameda was internalizing all the risk.
- CoolGuySteve 4y agoThe main advantage of FTX's exchange was that it was far better than Binance's but that's barely saying anything. I'm not sure why crypto exchanges have such lousy technology. Lost acks, weird parser errors, missing order id fields, sessions giving no indication that the matching engine is down, etc. It seems like any conceivable way an exchange can break will be encountered after only a couple months of trading. It's a complete shit show compared to traditional exchanges
- w1nst0nsm1th 4y agoBecause it was all a scam and didn't need to polish details ?
- elmomle 4y agoOne of the stranger things coming out of the collapse of FTX is that it wasn't _all_ a scam; if SBF and co they had simply let FTX function without giving special privileges to Alameda, it would have been a pretty non-scammy crypto exchange (overinflated in value, sure, but not fundamentally scamming its customers) and they all could have made great deals of money on that alone. But of course, that isn't what they did.
- dcolkitt 4y agoThere have been a few valiant attempts to build crypto exchanges with modern technology and matching engines. Generally the problem these run into is that, while the algo traders and market makers love them, they have no competitive advantage in attracting retail order flow. Trading firms have no interest in just trading against each other, so without retail flow nothing happens.
- hot_gril 4y agoHaving worked on several side projects that involve money, I'll bet most money-related computer systems are full of race conditions and other wacky behavior. There are so many gotchas. For example, just writing a simple "bank account" system/database, it's tricky to ensure in a performant way that concurrent transactions don't bring a user's balance below 0. If you're sending out money via ACH or something, try handling all the ways that can fail and need to be retried. I trade regular stocks in my Chase account. A few times, it was down for maintenance after-hours, meaning I couldn't enqueue trades to execute the next day. Not a big deal, but doesn't inspire confidence.
- wruza 4y agohorror stories about trying to cancel an open order, getting a cancel confirmation, then getting notified that the order was executed Sounds like tuesday to me. One thing you learn about exchanges (I mean all of them as a software category) is that you can’t trust: documentation, reasonable expectations, the experience you’ve got last week, the idea that all symbols behave similarly, reproducibility on test servers and vice versa, to name a few. Don’t rely on these and you’ll be relatively safe. But really, expecting a deterministic confirmation is a rookie mistake. Exchanges feel very asynchronous after a short while and the intuition should tell you that that a confirmation is just “ok, I hear ya”, not “effective immediately we’ll suspend our queues and reschedule trades to fulfill your urgent request”. The temporal uncertainty of order status is a quite common phenomenon, ime.
- vgatherps 4y agoFTX wouldn’t just ack your cancel request and then give you a fill - you would get cancel successful messages, order status updates that cancelled your order with no filled quantity, and then ten seconds later get a fill for that order. Occasionally sending out zero balances was another FTX special.
- deleted 4y ago[deleted]
- jasonzemos 4y agoWhile I wouldn't be surprised to see some proactive PR from the company in the wake of recent events, perhaps including the post on the zkSNARK competition last month; which was in my humble opinion one of the few submissions that has truly moved me since the late 2000's on this site: I feel obliged to attest that the people I knew working at Jane Street at that time were some of the straightest shooters one could know. My anecdote is that SBF's toxicity didn't originate in their culture. Perhaps it was a counteraction to it, if anything.
- sdenott 4y agoCan you link zkSnark convo mentioned here?
- yellowstuff 4y agohttps://blog.janestreet.com/zero-knowledge-fpgas-hardcaml/ https://blog.janestreet.com/zero-knowledge-fpgas-hardcaml/
- bcjordan 4y ago@dang could we get a (2017) added to this? Changes the context quite a bit given FTX folks worked at Jane Street previously and a modern version of this talk would have a much different context.
- 1vuio0pswjnm7 4y agoFTX's Gary Wang worked at Google. https://www.forbes.com/profile/gary-wang-1/ https://www.forbes.com/profile/gary-wang-1/ It's intriguing how Forbes 400 list includes criminals. If someone tricks people into "investing" with them and they accumulate enough billions of US dollars, and it takes a few years for the authorities to act, then do they get a spot on the Forbes 400. What are the rules.
- ctvo 4y agoI don't think the context changes at all. This talk is about high performing systems (millions of requests per second) written in OCalm. There is nothing here related to the criminality of FTX or what its non-technical founder decided to do with customer funds as a business decision. It matters very little that one or more of them used to work as an analyst at Jane Street.
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- gjvc 4y ago> a modern version of this talk would have a much different context How?
- gjvc 4y ago> a modern version of this talk would have a much different context Everything would be in a different context, yes. But how would the content be different?
- Zaheer 4y agoFun fact: Jane Street, Citadel and other HFT firms pay exorbitantly. You can see the base salary ($200-300k) posted public here: https://www.janestreet.com/join-jane-street/position/4274288002/ https://www.janestreet.com/join-jane-street/position/4274288... Even interns make $120/hr: https://www.levels.fyi/internships/ https://www.levels.fyi/internships/
- kennend3 4y agoHave family working there. You can get past 300K fairly easily once you factor in signing bonus, annual bonus, and "extras". They pay VERY VERY well indeed.
- Zaheer 4y agoYup, that figure I posted was just base but we often see folks getting multiples of that through bonuses.
- paxys 4y agoThey do pay extremely well, but are also very selective. The kind of talent they are looking for can earn similar salaries at large tech companies like Google ($400K-500K TC is pretty common at staff+ levels).
- kennend3 4y agoOne difference is Google, etc are all doing massive layoffs and i doubt JS, Citadel, etc will do this.
- ttobbaybbob 4y ago2023 version would have more than one mention of {compliance, regulat*, accounting}
- deleted 4y ago[deleted]
- ketralnis 4y agoWould it? 2017 is well post 2008 crisis, SOX, all of the big changes in this area
- matheist 4y agoAnyone know of any open-source exchanges at the level of sophistication described here?
- deleted 4y ago[deleted]
- shakezula 4y agoI wrote an open source one in Go that I'm still working on. It's never meant for production. The code is at github.com/dylanlott/orderbook and there's an accompanying write up on it at dylanlott.com/orderbook if you're interested.
- eftychis 4y agoThank you for the writeup.
- dang 4y agoRelated: How to Build an Exchange (2017) [video] - https://news.ycombinator.com/item?id=27879230 https://news.ycombinator.com/item?id=27879230 - July 2021 (12 comments)
- paulpauper 4y agoIn 2017 we extended our trading experience, infrastructure, and technology to digital assets, and we’re now trading crypto 24/7 around the world. We price all the most actively traded spot tokens such as Bitcoin and Ethereum in addition to derivatives like futures and ETPs, and we trade on almost all major global market centers, including crypto exchanges and traditional venues. We’re also a major OTC crypto liquidity provider, and our JCX single-dealer platform provides reliable, integratable liquidity in digital assets. Our institutional-grade crypto offering is a natural extension of our deep cross-asset expertise and our experience trading complex, difficult-to-price securities. Interesting. Crypto has much less liquidly overnight compared to other markets, no? How would this work? Coinbase?
- mianos 4y agoMost other markets actually have periods when they don't have a matching engine running because the instrument or settlement is country specific. Crypto has 24 hour liquidity. There are periods that are quieter but it is 24/7 unlike, for example, a typical equities exchange. (How I know, 25 years writing exchanges).
- hardwaregeek 4y agoI've always been curious whether someone will succeed in using the same development strategy as Jane Street, namely using a non-C++ language as their primary language. I suspect OCaml may have been a very good choice back in the early 2000's simply because C++ was a much different language and there weren't many options for other languages. Building up an ecosystem was not a bad plan because most languages did not have much of an ecosystem anyways. Nowadays ecosystems are a lot more developed. But on the flip side, that could mean you could stand to use a language like Rust that has an ecosystem and interoperability with C++. I know Tsuru Capital uses Rust but I'm not familiar with any other firms. It also could be that Jane Street succeeded regardless or even despite OCaml. At the end of the day OCaml isn't going to ensure that your trading strategies work. It's not going to suddenly create alpha. But I do suspect they've gotten serious hiring and marketing returns by using OCaml. And benefited the OCaml ecosystem as a byproduct, which is a pretty solid side-effect, even if side effects are not very functional :D
- codetrotter 4y ago> whether someone will succeed in using the same development strategy as Jane Street, namely using a non-C++ language as their primary language It’s gonna happen and it’s gonna be Rust > you could stand to use a language like Rust that has an ecosystem and interoperability with C++. Indeed :)
- LAC-Tech 4y agoMaybe. Rust as a core language is less productive than ocaml, by a lot. But Rusts standard library is just so much better than Ocamls (including Jane Streets version) - which tends to even it out.
- woodruffw 4y agoI think there’s probably a hidden coefficient here: I would readily believe that an experienced OCaml developer is more productive than an equivalently experienced Rust developer, but that a junior Rust developer is probably several times more productive than a similarly junior OCaml programmer. (This is not unique to this pairing of languages.)
- jjoe 4y agoAre there any technical books or reading material about matching engines out there?
- bob1029 4y agoRelated: https://lmax-exchange.github.io/disruptor/ https://lmax-exchange.github.io/disruptor/ I've been using the .NET port of Disruptor to good success. Once you understand the underlying pattern, you can apply it everywhere without pulling in a dependency. You would be astonished at what will actually fit on 1 x86 thread in 2023. Instruction-level parallelism can give you unbelievable throughput, assuming your batches are reasonably-sized and everything fits neatly into the various caches. Other related links: https://martinfowler.com/articles/lmax.html https://martinfowler.com/articles/lmax.html https://www.youtube.com/watch?v=qDhTjE0XmkE https://www.youtube.com/watch?v=qDhTjE0XmkE [Evolution of Financial Exchange Architectures]
- nullhash 4y agoHow exactly would would this design keep track of how many funds a client has in order to fulfill an order? How does NASDAQ keep track of client's total funds?
- iav 4y agoThey don't. Remember that NASDAQ's clients are Jane Street, Goldman Sachs, i.e. not you or me. They don't onboard a client with doing due diligence. Then there is a clearing and settlement system called DTCC that requires every firm to post collateral equal to their trading volume. This system's primary job is to make sure everyone has enough funds to pay for their trades across all exchanges, not just NASDAQ.
- rusebor 4y agohave been wondering for a while why HFTs exist when exchanges have holidays and trading hours [1]? doesn't the sub-milisecond speed contradict to the several hours pause in trading? wouldn't be possible to impose some kind of a delay (say an hour) on all participants? [1] https://www.nyse.com/markets/hours-calendars https://www.nyse.com/markets/hours-calendars