3 ms·
How would any cloud offering deal with something like, say, the full options data feed, which is close to 40 Gb/s of binary packed goodies? You need both a very
by TacticalCoder 2y ago
How would any cloud offering deal with something like, say, the full options data feed, which is close to 40 Gb/s of binary packed goodies? You need both a very fat pipe and ultra-low latency: does the cloud, any cloud, offer that?
Also: how often have you guys seen the stock market being down? What's the "x nines" availability of, say, the US stock market and US options feed?
Now: do we wanna talk about the various cloud outages that made the news? Sometimes lasting hours?
Also what I've seen with the cloud is websites are now displaying spinners everywhere, for the myriad of not-low-latency-at-all microservices often taking seconds to respond. And that'd be on an ultra low latency fiber to the home setup, with 2 Gb/s down (and the ISP really supporting that).
Why the heck do I have to wait seconds for oh-so-many things to display in my browser, on a last gen Ryzen ultra-speedy machine, with a super fat and low-latency Internet pipe? The worst offenders being all those banking websites showing a balance of 0 instead of "-" or "n/a" while fetching my info: nearly gives me heart attack every single time.
I take it it has to do with micro-services all contacting shitloads of other micro-services, all living in the not-low-latency cloud. The problem being compounded by an army, a generation, of programmers who have never learned anything about optimization or latency and who solve every problem they have with the only hammer they have: the cloud. All these programmers know are JSON (or, worse, XML)...
I mean: JSON vs 40 Gbit/s of interrupted bit-packed binary feeds? How could these two world ever reconcile?
Now I don't do HFT but I do trade options and I do it through a desktop app and that app also offers an API through which I can fetch prices, send orders, etc. It's a good old Java app. And it's more advanced than any website I've ever used.
Can we please not enshittify everything with countless micro-services and JSON files in the cloud?
- __alexs 2y agoOnly someone that both misunderstands microservices and HFT architectures could have this opinion. 100 Gb/s is possible on AWS via Direct Connect.
- smabie 2y agoWhat does cloud have to do with micro services and json? Crypto exchanges already run on the cloud and it works.. mostly fine?
- toast0 2y ago> Also: how often have you guys seen the stock market being down? What's the "x nines" availability of, say, the US stock market and US options feed? Well, it goes down every afternoon :p and it's down on the weekends. More like seven sevens than nine nines. It's been a while since I noticed a story about a significant stock exchange disruption, but there's a lot of things going on there. Tickers are largely independent, so it's easy to shard, and exchanges do shard them; (operational) trading outages often affect only a single stock, or rarely a set of stocks. There are procedures for administrative trading halts on individual stocks or the whole market and procedures to resume trading during the market day. There are also procedures for resuming trading after an operational error halted trading; I'm sure most traders don't like brief outages, and exchanges do their best to avoid them, but they can happen and be resolved with out a lot of confusion because the procedures are known and manageable. There's also redundancy. If one exchange is having difficulty, there are many others that likely still work. Whole system events are usually not operations issues, but trading issues --- one or several participants placed weird orders, the exchanges processed them, and weird things resulted. 'Circuit breakers' have been designed in to pause trading when this happens. This is an intentional outage, and it's ok because it's intentional and the parameters are known.