24 ms·

Goldman Sachs is spending $100M to shave milliseconds off stock trades

195 points by gscott 7y ago

wikibob 7y ago
Anyone here working at Goldman? I’ve heard very mixed things about working there. Would love some first hand stories! Have things changed from some years ago where the technology didn’t really get any respect and everyone was second class to the front office traders? How’s the work environment? Is there still a dress code? Is the system kept running by heroics and sleepless nights, or is there a mature understanding of human factors?
sabujp 7y ago
their stock isn't doing so great, it seems to track the rest of the financial markets (bac, jpm, c, etc)
vannevar 7y ago
It seems like we could save a lot of pointless expenditure on an ultimately meaningless arms race in flash trading if we imposed reasonable limits on the time required to hold an equity in order for a trade to be legally recognized.
- sabujp 7y ago
  we should also not allow the president to engage in trade wars, but i possibly digress
  - parsnips 7y ago
    please digress.
- Guvante 7y ago
  Allowing trading of fractional pennies has been theorized to be another way to break up flash trading as you need to accurately guess the real price not just get your exact price up before anyone else.
- nickles 7y ago
  > pointless expenditure on an ultimately meaningless arms race What about price discovery is pointless? Would you prefer that prices update only once a day? Once a week? Once a month? Realtime pricing of securities and derivatives is critical for an efficiently functioning economy. > if we imposed reasonable limits on the time required to hold an equity in order for a trade to be legally recognized This would damage the ability of market makers to function, ultimately driving up the cost of offering pensions, 401k plans, retail investing, low fee ETFs, etc. If these companies are willing to spend their money competing for the ability to offer you a better, faster price, why is this upsetting? If you're opposed to the emphasis on latency in equity markets, focus on rule 612 of Reg NMS, which prohibits showing more competitive prices.
  - whatshisface 7y ago
    Nanosecond pricing updates might be a little more than anybody needs. Maybe if the exchange cleared once every second, that would serve anyone's purposes. Remember the market is ultimately about allocating capital between businesses and governments, and for economic purposes it doesn't need to run any faster than they can.
    - nickles 7y ago
      > Nanosecond pricing updates might be a little more than anybody needs That may be the case, but the 'problem' is caused because market makers aren't legally allowed to offer better prices than they might otherwise desire. It doesn't seem prudent to layer flawed regulation on top of flawed regulation when there's a simpler solution. > Remember the market is ultimately about allocating capital between businesses and governments Equity markets have many purposes. Companies indeed raise money by issuing equity (IPOs and secondaries), and there are cases where governments participate for monetary purposes (BoJ & SNB's equity purchases). But markets also allow: * employees to sell equity compensation they've received whenever they want * retail investors to diversify their asset allocations * sovereign wealth funds, pension funds, endowments, and other real money sources return on their portfolios * for hedging financial risks * companies to return money to shareholders in the form of buy backs > for economic purposes it doesn't need to run any faster than they can All the previously mentioned functions happen all the time, and the marks (prices) generated from this activity helps inform many other economic functions. Everything from central bank policy to insurance pricing depends on well functioning markets. In light of this, why should we make things less efficient by slowing things down and driving the cost up?
- golemiprague 7y ago
  How is it different to any other competition between companies, what makes this one specifically pointless? Your idea will just put more money in the pocket of the bank traders rather than spreading the wealth to all the IT engineers and equipment manufacturers which supply this slight advantage, what's wrong with that?
- friendlybus 7y ago
  Even if the business functions were pointless, the research and execution going into these projects is valuable knowledge that likely has broad application.
- jwilliams 7y ago
  I wouldn't say it's meaningless. These types of technologies are usually arbitraged out; ending up with a few large players that are squeezing out a return. It means high frequency volatility is taken out of the equation for the rest of us -- Which generally is a benefit for other players in the market. FX has (effectively) been that way for a long time.
- simula67 7y ago
  A simpler solution might be to just increase the transaction tax
- dna_polymerase 7y ago
  But why? Who is the victim here?
  - Jolter 7y ago
    Who pays the $100M? Yes, the customer of the bank, of course. Which usually means you and me are paying in the end.
    - dna_polymerase 7y ago
      You are a client of Goldman Sachs? I think you won't worry about a share of $100M then.
JudgeWapner 7y ago
Tangentially related, but I wonder how long before HFT harnesses neutrino transmission to send market data directly through the earth? The MINOS [1] project is underway to send neutrinos from Fermilab in Illinois to detectors in northern Minnesota. Straight through the earth's crust. Detecting neutrinos is hard, and modulating data on them will probably be even more difficult, but if it shaves a few micro/milliseconds off trading time, I'm sure someone will make it happen eventually. HFT have already added dedicated undersea fiber-optic links [can't find source, NYC->LDN is obvious, I think there was talk or they began one from CHI/NYC->TOK but might be wrong, it involved the melting icecaps to easily sink the fiber cable], shortwave radio broadcasts [3], and microwave links [4]. I think HFT is frivolous, personally, but if it helps networking I think it would be cool to someday talk "on neutrinos" through the center of the earth! [1] https://en.wikipedia.org/wiki/MINOS https://en.wikipedia.org/wiki/MINOS [3] https://sniperinmahwah.wordpress.com/2018/05/07/shortwave-trading-part-i-the-west-chicago-tower-mystery/ https://sniperinmahwah.wordpress.com/2018/05/07/shortwave-tr... [4] https://arstechnica.com/information-technology/2016/11/private-microwave-networks-financial-hft/ https://arstechnica.com/information-technology/2016/11/priva...
- 4ntonius8lock 7y ago
  Very interesting. Physical wars brought pointless spending that prompted innovation. I wonder if the mini-financial wars being waged will result in some cool tech coming out of all the craziness. I had never though of this when thinking about the rent seeking nature of many parts of the financial industry.
  - rmah 7y ago
    Already has. The first notable example that springs to mind is "message queues" which were pioneered for use in finance. Companies like TIBCO grew fat on such software and now they are commonplace.
- smallnamespace 7y ago
  So some quick Googling shows that the neutrino beam fires every 1.3 seconds[1] and the 'near' detector on average captures 20 neutrinos, the 'far' detector sees <1 detection [2]. Note that the latter paper is from 2005 so this may have been substantially upgraded. Each 'spill' takes 8.67 milliseconds, so if you simply vary the firing time to send a signal using coordinated clocks, naively you could send up to ln(1000 / 8.67) = 6.8 bits per second, but at the cost of the very latency that you're trying to minimize. I'm sure there are smarter schemes, like also varying the neutrino beam density, but at 20 neutrinos per detection there's not much channel capacity in the amplitude either. It looks like the main bottleneck is in the neutrino generation though, you can use more accelerator beams without needing to build more detectors right away. Note also that the 'far' detector is only 500 miles away, so depending on how collimated the neutrino beam is you may be losing a lot of signal to get all the way through the Earth. [1] http://nusoft.fnal.gov/nova/public/neutrinos.html http://nusoft.fnal.gov/nova/public/neutrinos.html [2] http://neutron.physics.ucsb.edu/docs/MINOS_info.pdf http://neutron.physics.ucsb.edu/docs/MINOS_info.pdf
dboreham 7y ago
Vamipre Squid...
ralph84 7y ago
Most exchanges have an auction process that sets opening and closing prices. They let everyone get their orders in and then run an algorithm to find the price that will execute the most volume. They could do the same process every 5 minutes and only allow stocks to trade in the auction. Then all of the resources used on pointless HFT could be used on something economically productive.
- compsciphd 7y ago
  even every 1s. If we can do auctions for web ads, we can do auctions for the stock exchange.
  - nickles 7y ago
    > even every 1s. If we can do auctions for web ads, we can do auctions for the stock exchange. There was ~$107 billion in online advertising in 2018. There was ~$145 billion traded in Nasdaq listed equities yesterday. Global financial markets and online advertising have different requirements. [0] https://www.marketingcharts.com/advertising-trends/spending-and-spenders-108416 https://www.marketingcharts.com/advertising-trends/spending-... [1] http://www.nasdaqtrader.com/Trader.aspx?id=DailyMarketSummary http://www.nasdaqtrader.com/Trader.aspx?id=DailyMarketSummar...
    - dlhavema 7y ago
      How is the 145 billion calculated? If I buy and sell with a 100$ 100 times does that count as $10,000 or $100?
      - dmoy 7y ago
        The number quoted is total volume, so it would be $10k
    - buzzdenver 7y ago
      > There was ~$107 billion in online advertising in 2018. There was ~$145 billion traded in Nasdaq listed equities yesterday. That is a pointless comparison. That $107b was all revenue for somebody (Google, Facebook, etc), while the $145b was just the nominal value of shares traded. (edited typo)
codingslave 7y ago
People love to rail on HFT, but at this point, its really not that profitable. It's just a reality of trading in the markets. There was a blip of time between 2008 and 2014 when HFT was extremely profitable. Those inefficiencies have been gone from the market for years. People were whooped into anger about how much money was being made, at this point its a complete non issue and needs to be removed from the highlight reels aimed at generating anger in the public. Lets move on from discussing the boogey man that is HFT, its really nothing.
- H8crilA 7y ago
  As always, some profit is to be made from some trading practice - the market removes it within a few years of "mass discovery". All working as intended.
- nickles 7y ago
  Adding to this, HFT is a product of rule 612 of Reg NMS (the sub-penny rule). Markets are not allowed to show quotes in increments of less than $0.01 for most names. Since traders cannot compete on price, they have been forced to compete exclusively on speed. The impact of such regulation was tested by the SEC recently with the 'tick size' program. Instead of reducing the minimum increment, some names saw it increased to $0.05. The hope was to increase liquidity while decreasing volatility in these names. In fact, those names experienced decreased liquidity with no decrease in volatility. HFT is a result of regulation. [0] https://www.sec.gov/divisions/marketreg/subpenny612faq.htm https://www.sec.gov/divisions/marketreg/subpenny612faq.htm [1] https://www.benzinga.com/general/education/18/04/11517027/the-secs-tick-size-program-likely-to-end-heres-why-thats-a-good-thi https://www.benzinga.com/general/education/18/04/11517027/th...
  - twic 7y ago
    On the other hand, the Intercontinental Exchange reduced the tick size for sterling interest rate futures towards the end of 2018, and the result was ... decreased liquidity! And resulting increased volatility. I'm not sure anyone knows for sure why this happened, but the best theory i've heard is that the reduction in tick size reduced the expected profits of market makers, because they are collecting less spread on every contract they turn around, while not affecting their potential losses, because external factors which cause the market to jump three basis points will still cause it to jump three basis points. Halved regular profits divided by constant occasional losses equals no longer worth bothering with.
abalone 7y ago
I was pretty impressed by Rust's latest showing in the techempower benchmarks. The Actix framwork blew the ceiling off of previous benchmarks. SIMD code, lock free, static dispatch, pipelined postgres driver, and other performance enhancements.[1] As fast or faster than the C/C++ frameworks. [1] https://github.com/TechEmpower/FrameworkBenchmarks/issues/4834 https://github.com/TechEmpower/FrameworkBenchmarks/issues/48...
- MuffinFlavored 7y ago
  Does Goldman Sachs use Rust/actix for quick trading?
  - H8crilA 7y ago
    People that are very serious about it use ASICs/FPGAs. I'm sure Goldman does too. For slightly slower stuff it's the garbage collector and OS scheduling that's the problem (as both can unpredictably inject many milliseconds pauses). No GC and good usage of low level kernel primitives is the game.
jedberg 7y ago
Can someone explain to me what is gained by processing the trades in real time vs. batching the processing into say 1 second increments? What does GS gain by being able to get their trade there a few milliseconds before the competition and what do I as a consumer gain from this?
- harryh 7y ago
  Batching won't solve any problems you might think it would solve. The real world is continuous. Because of this, everyone is incentivized to wait until the last possible millisecond (or nanosecond) before the auction to submit bids so the incentive to build fast acting systems remains. There are other problems as well, like how to deal with bid/ask imbalances at auction time.
- wallace_f 7y ago
  You don't gain anything. GS is in an arms race with other fintech firms to be first in line to act on new information.
  - denormalfloat 7y ago
    That's not right, you (we) get accurately priced securities. We also get tighter bid-ask spreads.
    - wallace_f 7y ago
      Accurately priced securities 100ms faster (which have had any profit opportunity already arbitraged). Grea.t.
    - 9HZZRfNlpR 7y ago
      The market is closed so much of the time available for us means we don't need these milliseconds for accurate prices/price-establishing?
- maximente 7y ago
  suppose "real time" means order of magnitude milliseconds for example's sake. the price could change a bunch in that interval of 1000ms. you may think "well only slightly" - fractions of cents - but if GS can make fractions of pennies on those events, scaled up to all seconds that the market is open, you can see why that's potentially attractive. for whatever it's worth, GS in 2009 claimed that HFT generated <1% of their profits. whether you believe that is another thing.
vijaybritto 7y ago
"Using so-called microservices to break complicated problems into easy-to-solve ones" I would love to see microservices which actually solve problems and reduce complexity! :(
- winrid 7y ago
  I am curious how that will help latency here. It must be pretty bad if refactoring and adding more network connections is an improvement.
  - vijaybritto 7y ago
    Yeah I know people preferring microservices for its horizontal scalability advantage but never for reducing latency. Maybe something was lost in the translation to the author.
  - koffiezet 7y ago
    Only thing that I can think of is that it would make it easier to detect the bottlenecks. Other than that, certainly in high-frequency trading - I only see it adding latency.
  - DrBazza 7y ago
    They were likely running old Tibco/RV systems (network) distributed across the network (common for 1990s to early 2000 trading systems), and replaced the system (and hardware) with multi-core boxes, and use shared memory for message passing. Reduces internal latency from milliseconds to sub-microsecond.
    - winrid 7y ago
      Makes sense in this case. I guess you can just deploy the microservices as containers on the same machine or what have you.
- koffiezet 7y ago
  Complexity is never reduced, but made more manageable. Microservices make a good architecture a lot harder, certainly if you have to start with a blank slate and don't really have an idea what you're up against. However, things that are virtually impossible to enforce or even very hard to implement in a monolith, are made manageable if you have the proper setup. That includes central standardised logging, tracing, metrics and monitoring. If you have these in place and can enforce them, you're off to a good start. These things rarely happen in a v1 though - which usually is a POC that ends up in production, and if you have a full-blown microservice architecture from the start, this will probably grow into something a lot worse than huge PHP monolith. With microservices you need to design a 'platform', and ad-hoc POC development never results in a good design, but just something that functionally works.
puskavi 7y ago
yay more control to bankers!
ta4534543fdgd 7y ago
They've taken greed into stellar territory! Well done...
- justanegg 7y ago
  yawn
xvilka 7y ago
Would be nice to read more about technical sides of this transition. Also getting rid of Java code, that is so popular among financial institutions, will increase the speed. Something like Rust is a perfect fit[1] for such industries. [1] https://internals.rust-lang.org/t/proposal-business-applications-working-group/10400 https://internals.rust-lang.org/t/proposal-business-applicat...
Rafuino 7y ago
While not related to high frequency trading, per se, the Google research on the gap between nanosecond and millisecond latencies (AKA "attack of the killer microseconds") springs to mind with this article. Typically the hardware bottleneck in HFT, I assume, is network latency, which leads to colocating trading servers near the exchanges themselves...so how do you improve on that? Smart NICs/FPGAs? Better non-volatile storage (there's hardware on the market today that's getting us to read latencies in the single digits on PCIe bus at least)...? ASICs for HFT specifically? This is obviously a marketing piece put out to attract quants to Goldman Sachs given the lack of detail, but I naturally wonder where that $100M is going to be invested. https://cacm.acm.org/magazines/2017/4/215032-attack-of-the-killer-microseconds/fulltext https://cacm.acm.org/magazines/2017/4/215032-attack-of-the-k...
- pjc50 7y ago
  It's certainly possible to do it in FPGA, I was involved in this project a few years ago: http://www.argondesign.com/news/2014/jun/23/ultra-low-latency-hft-options-trading-system/ http://www.argondesign.com/news/2014/jun/23/ultra-low-latenc... Although it limits the complexity of the model and makes it harder to iterate on the development of the model.
stakhanov 7y ago
...actually it doesn't sound to me like this article is about HFT-based prop trading at all (prop trading would mean Goldman Sachs taking positions onto their own books), but about the business unit called GSAT (Goldman Sachs algorithmic trading) who execute trades on behalf of clients, so never taking any positions onto their own books. The traditional market model used to be that at every point in time, a market maker would offer a price to buy/sell at. The bid would, of course, be lower than the ask, the difference being called the spread. When a pension fund wanted to execute a trade, they would have had to cross the spread, and, statistically speaking, half the spread would immediately accrue to the market maker as profit. ..so this is money that YOU, the holder of a pension, are losing, and that THEY, the rich folks acting as market makers that the public likes to get mad at, are taking away from you. The business model of GSAT is that a pension fund can ask Goldman Sachs to use algorithms to do things on their behalf that are less naive than what I just described above and end up giving less money to the market maker. For example, GSAT can become a market maker on your behalf, but make a market on only one side, i.e. only offer a price to buy at or only offer a price to sell at, with the resulting trades being executed on your behalf, so it is now you who makes money off those trades, in the same way as it would traditionally be a market maker's privilege to do. The public loves to get mad at Wall Street. But they should please get their facts straight about who the good guys are and who the bad guys are.
- stillworks 7y ago
  Have to agree. Catering to HFT shops was already in 100 microsecond latency a few years ago using commodity hardware and software (cannot speak of Goldman Sachs but another one of similar ilk, can't imagine GS were much far away if not even better)
  - mieseratte 7y ago
    Worked in an HFT shop and we were down to optimizing nanoseconds. Lots of folks had war stories about spending months to shave off tens of nanoseconds off pieces used millions of times per day.
    - 7y ago
1e-9 7y ago
Markets around the world are determining prices on a massive variety of instruments that derive value from the current and future value of products such as currencies, interest rates, equities, grains, livestock, metals, oil, gasoline, natural gas, and electricity. These prices allow us to prioritize resources, make fair transactions, and manage risk (i.e. buy insurance on the value of critical products so that we can plan and invest more effectively). The value of all these instruments are related to one another, as well as to realtime world events, in complex ways that no one entity has a perfect view of. Consequently, markets work by polling the expertise of many different parties who all understand a piece of how things should be valued. This results in millions, if not billions, of interconnected price-discovery feedback loops. The markets are kind of like a massive, distributed, realtime, ensemble, recursive predictor that performs much better than any one of its individual component algorithms could. The reason why shaving a few milliseconds (or even microseconds) can be beneficial is because the price discovery feedback loops get faster, which allows the system to determine a giant pricing vector that is more self-consistent, stable, and beneficial to the economy. It's similar to how increasing the sample rate of a feedback control system improves performance and stability. Providers of such benefits to the markets get rewarded through profit.
- NohatCoder 7y ago
  Meh, the short term price fluctuations are almost pure noise. Faster trading just incase the frequency of the noise. Increasing frequency only works if the feedback loops are stable.
  - 1e-9 7y ago
    Over very short time scales, the price fluctuations vary from almost pure noise to almost pure signal. The almost pure noise situation occurs far more often, but even then, with good estimation techniques, you can extract a signal component that can improve pricing a small amount. Improving many interrelated prices by small amounts can lead to a significant overall improvement to the markets. The almost pure signal situation occurs far less often, but it is almost always extremely important to overall stability. Handling it well can minimize market overreactions and extreme events such as flash crashes and market panics.
sfthrower89 7y ago
We thought we were going to have healthcare, robots, and flying cars. Instead we have wasted two generations' worth of intellect on .. advertisements and fast stock trading. I'm ready for the giant asteroid.
tosh 7y ago
A great read on high frequency trading: Barbarians at the Gate https://queue.acm.org/detail.cfm?id=2536492 https://queue.acm.org/detail.cfm?id=2536492
- harryh 7y ago
  Title is actually Barbarians at the Gateways which is, of course, as play on words on the actual Barbarians at the Gate: https://en.wikipedia.org/wiki/Barbarians_at_the_Gate:_The_Fall_of_RJR_Nabisco https://en.wikipedia.org/wiki/Barbarians_at_the_Gate:_The_Fa... Another great tail of finance that operated at much slower speeds.
dp141 7y ago
Amazing to me how much impassioned defence there is of HFT. I assume a lot of the HN readership works in HFT.
gekkeboom 7y ago
This is called rent seeking. Just siphoning money from society without any benefit.
decoyworker 7y ago
Profiting in this way seems to be a bastardization of the purpose of the market and provides no value.
seanhunter 7y ago
For what it's worth, I used to work for GS and was in the algorithmic brokerage business for 2009-2010. One thing a lot of commenters on this thread are missing here is this isn't the same as an HFT or hedge fund, this is the brokerage business - executing orders on behalf of GSs clients, who are largely institutional investors, hedge and pension funds etc. The real-world impact of increased speed of execution by brokers is less money left on the table for HFTs to snap up and less slippage to the actual economic beneficiary (ie the actual retail investor or pension fund investing people's money gets a better trade).
- jfengel 7y ago
  Does GS do HFT itself? Is there a conflict of interest?
  - seanhunter 7y ago
    I'm not a current employee, but at the time that I was, they did have a small unit who did pseudo-HFT on a prop basis and various electronic market-making businesses, but they were all firewalled off from the people who did brokerage business. We didn't share code or information with them and for obvious reasons they had no ability to view client order flow, unexecuted orders etc. Likewise people in the high-touch (ie voice) execution business couldn't see anything in the GSAT business even though they could use our algos to execute orders if they want to. My understanding (could be incorrect) is that their quant trading business (what they called their 'HFT' shop, although it wasn't really high-frequency compared to real HFTs like Winton, Knight, Jump, Citadel or whatever) was probably going to get shuttered as they moved out of proprietary risk-taking generally, but I don't have any information either way. In GSAT at the time, compared to others on the street our tech was pretty sophisticated intellectually but not fast (eg we didn't have ultrafast marketdata, our exchange latency was quite high and our execution algo speed was pretty slow) and so we had to do a lot of smart coding to prevent ourselves being ripped off by actual HFTs given they could move so much faster than we could. There's a lot more to HFT than reg NMS by the way, I was working in London, so we did all the GSAT trading on European exchanges none of which has anything crazy like reg NMS and there was still HFT shenanigans of various kinds that people would try (eg timing arbitrages if they could see that you had different execution speeds on different venues etc).
charlixcx 7y ago
Well it sounds like a lot of money, but probably isn't even THAT much for GS. Like, flops for that amount of money happen in hollywood movies quite often.
ausbah 7y ago
To me this is incredibly ridiculous, millions more dumped into the brain draining maw of finance.
- taspeotis 7y ago
  Forgive the AMP link but Bloomberg has a paywall of sorts now. https://www.google.com.au/amp/s/www.bloomberg.com/amp/opinion/articles/2019-03-11/you-need-a-big-tower-to-trade-stocks https://www.google.com.au/amp/s/www.bloomberg.com/amp/opinio... > And while Scientel clearly has high-frequency trading in mind—“the name that keeps coming up among industry sources is Citadel”—that’s not all it has in mind. It’s a tower that can send signals for lots of stuff. “Scientel has said it will equip its Aurora tower with 28 antennas—24 for public safety and municipal use and four for ‘private’ purposes.” People often complain that high-frequency trading encourages a socially wasteful arms race, and you certainly see some of that here, as lots of trading firms compete to put their antennas as close as possible to the CME servers. But another way to interpret this story is that high-frequency trading is subsidizing high-tech communications infrastructure for everyone else, building towers to send high-speed signals for public safety and municipal use just to justify a couple of high-frequency trading antennas.
  - ausbah 7y ago
    Reminds me of this comic https://smbc-comics.com/index.php?id=3890 https://smbc-comics.com/index.php?id=3890
heelix 7y ago
I know a couple of the folks doing this sort of coding and while I'll tweak my APIs to be in the 10s of milliseconds they are tuning for nanosecond optimization. Crazy to see code where the speed of light is becoming a bottleneck.
tjpaudio 7y ago
I really wish we would curb this kind of behavior. Even if this speed is related to the brokerage business and not HFT, it is unfair to retail investors that large businesses have better market access. My solution: exchanges are required by law to process contracts in pulses with strict execution ordering rules. E.g., every 15 seconds process all buy and sells, processing from highest bid and lowest ask first for each security. Basically a turn-based stock market.
m0rphling 7y ago
Tom Scott did a nice video [1] on IEX's setup with 38 miles of fiber to add enough latency as a mechanism of fairness. Purportedly, this benefits all exchanges after it was successfully deployed. It seems like the incentives are not there for larger, more established exchanges to implement such blanket latency. So, firms like GS see a benefit to this kind of investment. [1] https://www.youtube.com/watch?v=d8BcCLLX4N4 https://www.youtube.com/watch?v=d8BcCLLX4N4
sytelus 7y ago
I would highly recommend the move Hummingbird Project which does a pretty good job of what it takes to build this kind of networks: https://www.youtube.com/watch?v=U8DweHM0Gnc https://www.youtube.com/watch?v=U8DweHM0Gnc

195 points by gscott 7y ago

Anyone here working at Goldman? I’ve heard very mixed things about working there. Would love some first hand stories! Have things changed from some years ago where the technology didn’t really get any respect and everyone was second class to the front office traders? How’s the work environment? Is there still a dress code? Is the system kept running by heroics and sleepless nights, or is there a mature understanding of human factors?

sabujp 7y ago

their stock isn't doing so great, it seems to track the rest of the financial markets (bac, jpm, c, etc)

vannevar 7y ago

It seems like we could save a lot of pointless expenditure on an ultimately meaningless arms race in flash trading if we imposed reasonable limits on the time required to hold an equity in order for a trade to be legally recognized.

sabujp 7y ago
we should also not allow the president to engage in trade wars, but i possibly digress
- parsnips 7y ago
  please digress.
Guvante 7y ago
Allowing trading of fractional pennies has been theorized to be another way to break up flash trading as you need to accurately guess the real price not just get your exact price up before anyone else.
nickles 7y ago
> pointless expenditure on an ultimately meaningless arms race What about price discovery is pointless? Would you prefer that prices update only once a day? Once a week? Once a month? Realtime pricing of securities and derivatives is critical for an efficiently functioning economy. > if we imposed reasonable limits on the time required to hold an equity in order for a trade to be legally recognized This would damage the ability of market makers to function, ultimately driving up the cost of offering pensions, 401k plans, retail investing, low fee ETFs, etc. If these companies are willing to spend their money competing for the ability to offer you a better, faster price, why is this upsetting? If you're opposed to the emphasis on latency in equity markets, focus on rule 612 of Reg NMS, which prohibits showing more competitive prices.
- whatshisface 7y ago
  Nanosecond pricing updates might be a little more than anybody needs. Maybe if the exchange cleared once every second, that would serve anyone's purposes. Remember the market is ultimately about allocating capital between businesses and governments, and for economic purposes it doesn't need to run any faster than they can.
  - nickles 7y ago
    > Nanosecond pricing updates might be a little more than anybody needs That may be the case, but the 'problem' is caused because market makers aren't legally allowed to offer better prices than they might otherwise desire. It doesn't seem prudent to layer flawed regulation on top of flawed regulation when there's a simpler solution. > Remember the market is ultimately about allocating capital between businesses and governments Equity markets have many purposes. Companies indeed raise money by issuing equity (IPOs and secondaries), and there are cases where governments participate for monetary purposes (BoJ & SNB's equity purchases). But markets also allow: * employees to sell equity compensation they've received whenever they want * retail investors to diversify their asset allocations * sovereign wealth funds, pension funds, endowments, and other real money sources return on their portfolios * for hedging financial risks * companies to return money to shareholders in the form of buy backs > for economic purposes it doesn't need to run any faster than they can All the previously mentioned functions happen all the time, and the marks (prices) generated from this activity helps inform many other economic functions. Everything from central bank policy to insurance pricing depends on well functioning markets. In light of this, why should we make things less efficient by slowing things down and driving the cost up?
golemiprague 7y ago
How is it different to any other competition between companies, what makes this one specifically pointless? Your idea will just put more money in the pocket of the bank traders rather than spreading the wealth to all the IT engineers and equipment manufacturers which supply this slight advantage, what's wrong with that?
friendlybus 7y ago
Even if the business functions were pointless, the research and execution going into these projects is valuable knowledge that likely has broad application.
jwilliams 7y ago
I wouldn't say it's meaningless. These types of technologies are usually arbitraged out; ending up with a few large players that are squeezing out a return. It means high frequency volatility is taken out of the equation for the rest of us -- Which generally is a benefit for other players in the market. FX has (effectively) been that way for a long time.
simula67 7y ago
A simpler solution might be to just increase the transaction tax
dna_polymerase 7y ago
But why? Who is the victim here?
- Jolter 7y ago
  Who pays the $100M? Yes, the customer of the bank, of course. Which usually means you and me are paying in the end.
  - dna_polymerase 7y ago
    You are a client of Goldman Sachs? I think you won't worry about a share of $100M then.

JudgeWapner 7y ago

Tangentially related, but I wonder how long before HFT harnesses neutrino transmission to send market data directly through the earth? The MINOS [1] project is underway to send neutrinos from Fermilab in Illinois to detectors in northern Minnesota. Straight through the earth's crust. Detecting neutrinos is hard, and modulating data on them will probably be even more difficult, but if it shaves a few micro/milliseconds off trading time, I'm sure someone will make it happen eventually. HFT have already added dedicated undersea fiber-optic links [can't find source, NYC->LDN is obvious, I think there was talk or they began one from CHI/NYC->TOK but might be wrong, it involved the melting icecaps to easily sink the fiber cable], shortwave radio broadcasts [3], and microwave links [4]. I think HFT is frivolous, personally, but if it helps networking I think it would be cool to someday talk "on neutrinos" through the center of the earth! [1] https://en.wikipedia.org/wiki/MINOS https://en.wikipedia.org/wiki/MINOS [3] https://sniperinmahwah.wordpress.com/2018/05/07/shortwave-trading-part-i-the-west-chicago-tower-mystery/ https://sniperinmahwah.wordpress.com/2018/05/07/shortwave-tr... [4] https://arstechnica.com/information-technology/2016/11/private-microwave-networks-financial-hft/ https://arstechnica.com/information-technology/2016/11/priva...

4ntonius8lock 7y ago
Very interesting. Physical wars brought pointless spending that prompted innovation. I wonder if the mini-financial wars being waged will result in some cool tech coming out of all the craziness. I had never though of this when thinking about the rent seeking nature of many parts of the financial industry.
- rmah 7y ago
  Already has. The first notable example that springs to mind is "message queues" which were pioneered for use in finance. Companies like TIBCO grew fat on such software and now they are commonplace.
smallnamespace 7y ago
So some quick Googling shows that the neutrino beam fires every 1.3 seconds[1] and the 'near' detector on average captures 20 neutrinos, the 'far' detector sees <1 detection [2]. Note that the latter paper is from 2005 so this may have been substantially upgraded. Each 'spill' takes 8.67 milliseconds, so if you simply vary the firing time to send a signal using coordinated clocks, naively you could send up to ln(1000 / 8.67) = 6.8 bits per second, but at the cost of the very latency that you're trying to minimize. I'm sure there are smarter schemes, like also varying the neutrino beam density, but at 20 neutrinos per detection there's not much channel capacity in the amplitude either. It looks like the main bottleneck is in the neutrino generation though, you can use more accelerator beams without needing to build more detectors right away. Note also that the 'far' detector is only 500 miles away, so depending on how collimated the neutrino beam is you may be losing a lot of signal to get all the way through the Earth. [1] http://nusoft.fnal.gov/nova/public/neutrinos.html http://nusoft.fnal.gov/nova/public/neutrinos.html [2] http://neutron.physics.ucsb.edu/docs/MINOS_info.pdf http://neutron.physics.ucsb.edu/docs/MINOS_info.pdf

dboreham 7y ago

Vamipre Squid...

ralph84 7y ago

Most exchanges have an auction process that sets opening and closing prices. They let everyone get their orders in and then run an algorithm to find the price that will execute the most volume. They could do the same process every 5 minutes and only allow stocks to trade in the auction. Then all of the resources used on pointless HFT could be used on something economically productive.

compsciphd 7y ago
even every 1s. If we can do auctions for web ads, we can do auctions for the stock exchange.
- nickles 7y ago
  > even every 1s. If we can do auctions for web ads, we can do auctions for the stock exchange. There was ~$107 billion in online advertising in 2018. There was ~$145 billion traded in Nasdaq listed equities yesterday. Global financial markets and online advertising have different requirements. [0] https://www.marketingcharts.com/advertising-trends/spending-and-spenders-108416 https://www.marketingcharts.com/advertising-trends/spending-... [1] http://www.nasdaqtrader.com/Trader.aspx?id=DailyMarketSummary http://www.nasdaqtrader.com/Trader.aspx?id=DailyMarketSummar...
  - dlhavema 7y ago
    How is the 145 billion calculated? If I buy and sell with a 100$ 100 times does that count as $10,000 or $100?
    - dmoy 7y ago
      The number quoted is total volume, so it would be $10k
  - buzzdenver 7y ago
    > There was ~$107 billion in online advertising in 2018. There was ~$145 billion traded in Nasdaq listed equities yesterday. That is a pointless comparison. That $107b was all revenue for somebody (Google, Facebook, etc), while the $145b was just the nominal value of shares traded. (edited typo)

codingslave 7y ago

People love to rail on HFT, but at this point, its really not that profitable. It's just a reality of trading in the markets. There was a blip of time between 2008 and 2014 when HFT was extremely profitable. Those inefficiencies have been gone from the market for years. People were whooped into anger about how much money was being made, at this point its a complete non issue and needs to be removed from the highlight reels aimed at generating anger in the public. Lets move on from discussing the boogey man that is HFT, its really nothing.

H8crilA 7y ago
As always, some profit is to be made from some trading practice - the market removes it within a few years of "mass discovery". All working as intended.
nickles 7y ago
Adding to this, HFT is a product of rule 612 of Reg NMS (the sub-penny rule). Markets are not allowed to show quotes in increments of less than $0.01 for most names. Since traders cannot compete on price, they have been forced to compete exclusively on speed. The impact of such regulation was tested by the SEC recently with the 'tick size' program. Instead of reducing the minimum increment, some names saw it increased to $0.05. The hope was to increase liquidity while decreasing volatility in these names. In fact, those names experienced decreased liquidity with no decrease in volatility. HFT is a result of regulation. [0] https://www.sec.gov/divisions/marketreg/subpenny612faq.htm https://www.sec.gov/divisions/marketreg/subpenny612faq.htm [1] https://www.benzinga.com/general/education/18/04/11517027/the-secs-tick-size-program-likely-to-end-heres-why-thats-a-good-thi https://www.benzinga.com/general/education/18/04/11517027/th...
- twic 7y ago
  On the other hand, the Intercontinental Exchange reduced the tick size for sterling interest rate futures towards the end of 2018, and the result was ... decreased liquidity! And resulting increased volatility. I'm not sure anyone knows for sure why this happened, but the best theory i've heard is that the reduction in tick size reduced the expected profits of market makers, because they are collecting less spread on every contract they turn around, while not affecting their potential losses, because external factors which cause the market to jump three basis points will still cause it to jump three basis points. Halved regular profits divided by constant occasional losses equals no longer worth bothering with.

abalone 7y ago

I was pretty impressed by Rust's latest showing in the techempower benchmarks. The Actix framwork blew the ceiling off of previous benchmarks. SIMD code, lock free, static dispatch, pipelined postgres driver, and other performance enhancements.[1] As fast or faster than the C/C++ frameworks. [1] https://github.com/TechEmpower/FrameworkBenchmarks/issues/4834 https://github.com/TechEmpower/FrameworkBenchmarks/issues/48...

MuffinFlavored 7y ago
Does Goldman Sachs use Rust/actix for quick trading?
- H8crilA 7y ago
  People that are very serious about it use ASICs/FPGAs. I'm sure Goldman does too. For slightly slower stuff it's the garbage collector and OS scheduling that's the problem (as both can unpredictably inject many milliseconds pauses). No GC and good usage of low level kernel primitives is the game.

jedberg 7y ago

Can someone explain to me what is gained by processing the trades in real time vs. batching the processing into say 1 second increments? What does GS gain by being able to get their trade there a few milliseconds before the competition and what do I as a consumer gain from this?

harryh 7y ago
Batching won't solve any problems you might think it would solve. The real world is continuous. Because of this, everyone is incentivized to wait until the last possible millisecond (or nanosecond) before the auction to submit bids so the incentive to build fast acting systems remains. There are other problems as well, like how to deal with bid/ask imbalances at auction time.
wallace_f 7y ago
You don't gain anything. GS is in an arms race with other fintech firms to be first in line to act on new information.
- denormalfloat 7y ago
  That's not right, you (we) get accurately priced securities. We also get tighter bid-ask spreads.
  - wallace_f 7y ago
    Accurately priced securities 100ms faster (which have had any profit opportunity already arbitraged). Grea.t.
  - 9HZZRfNlpR 7y ago
    The market is closed so much of the time available for us means we don't need these milliseconds for accurate prices/price-establishing?
maximente 7y ago
suppose "real time" means order of magnitude milliseconds for example's sake. the price could change a bunch in that interval of 1000ms. you may think "well only slightly" - fractions of cents - but if GS can make fractions of pennies on those events, scaled up to all seconds that the market is open, you can see why that's potentially attractive. for whatever it's worth, GS in 2009 claimed that HFT generated <1% of their profits. whether you believe that is another thing.

vijaybritto 7y ago

"Using so-called microservices to break complicated problems into easy-to-solve ones" I would love to see microservices which actually solve problems and reduce complexity! :(

winrid 7y ago
I am curious how that will help latency here. It must be pretty bad if refactoring and adding more network connections is an improvement.
- vijaybritto 7y ago
  Yeah I know people preferring microservices for its horizontal scalability advantage but never for reducing latency. Maybe something was lost in the translation to the author.
- koffiezet 7y ago
  Only thing that I can think of is that it would make it easier to detect the bottlenecks. Other than that, certainly in high-frequency trading - I only see it adding latency.
- DrBazza 7y ago
  They were likely running old Tibco/RV systems (network) distributed across the network (common for 1990s to early 2000 trading systems), and replaced the system (and hardware) with multi-core boxes, and use shared memory for message passing. Reduces internal latency from milliseconds to sub-microsecond.
  - winrid 7y ago
    Makes sense in this case. I guess you can just deploy the microservices as containers on the same machine or what have you.
koffiezet 7y ago
Complexity is never reduced, but made more manageable. Microservices make a good architecture a lot harder, certainly if you have to start with a blank slate and don't really have an idea what you're up against. However, things that are virtually impossible to enforce or even very hard to implement in a monolith, are made manageable if you have the proper setup. That includes central standardised logging, tracing, metrics and monitoring. If you have these in place and can enforce them, you're off to a good start. These things rarely happen in a v1 though - which usually is a POC that ends up in production, and if you have a full-blown microservice architecture from the start, this will probably grow into something a lot worse than huge PHP monolith. With microservices you need to design a 'platform', and ad-hoc POC development never results in a good design, but just something that functionally works.

puskavi 7y ago

yay more control to bankers!

ta4534543fdgd 7y ago

They've taken greed into stellar territory! Well done...

justanegg 7y ago
yawn

xvilka 7y ago

Would be nice to read more about technical sides of this transition. Also getting rid of Java code, that is so popular among financial institutions, will increase the speed. Something like Rust is a perfect fit[1] for such industries. [1] https://internals.rust-lang.org/t/proposal-business-applications-working-group/10400 https://internals.rust-lang.org/t/proposal-business-applicat...

Rafuino 7y ago

While not related to high frequency trading, per se, the Google research on the gap between nanosecond and millisecond latencies (AKA "attack of the killer microseconds") springs to mind with this article. Typically the hardware bottleneck in HFT, I assume, is network latency, which leads to colocating trading servers near the exchanges themselves...so how do you improve on that? Smart NICs/FPGAs? Better non-volatile storage (there's hardware on the market today that's getting us to read latencies in the single digits on PCIe bus at least)...? ASICs for HFT specifically? This is obviously a marketing piece put out to attract quants to Goldman Sachs given the lack of detail, but I naturally wonder where that $100M is going to be invested. https://cacm.acm.org/magazines/2017/4/215032-attack-of-the-killer-microseconds/fulltext https://cacm.acm.org/magazines/2017/4/215032-attack-of-the-k...

pjc50 7y ago
It's certainly possible to do it in FPGA, I was involved in this project a few years ago: http://www.argondesign.com/news/2014/jun/23/ultra-low-latency-hft-options-trading-system/ http://www.argondesign.com/news/2014/jun/23/ultra-low-latenc... Although it limits the complexity of the model and makes it harder to iterate on the development of the model.

stakhanov 7y ago

...actually it doesn't sound to me like this article is about HFT-based prop trading at all (prop trading would mean Goldman Sachs taking positions onto their own books), but about the business unit called GSAT (Goldman Sachs algorithmic trading) who execute trades on behalf of clients, so never taking any positions onto their own books. The traditional market model used to be that at every point in time, a market maker would offer a price to buy/sell at. The bid would, of course, be lower than the ask, the difference being called the spread. When a pension fund wanted to execute a trade, they would have had to cross the spread, and, statistically speaking, half the spread would immediately accrue to the market maker as profit. ..so this is money that YOU, the holder of a pension, are losing, and that THEY, the rich folks acting as market makers that the public likes to get mad at, are taking away from you. The business model of GSAT is that a pension fund can ask Goldman Sachs to use algorithms to do things on their behalf that are less naive than what I just described above and end up giving less money to the market maker. For example, GSAT can become a market maker on your behalf, but make a market on only one side, i.e. only offer a price to buy at or only offer a price to sell at, with the resulting trades being executed on your behalf, so it is now you who makes money off those trades, in the same way as it would traditionally be a market maker's privilege to do. The public loves to get mad at Wall Street. But they should please get their facts straight about who the good guys are and who the bad guys are.

stillworks 7y ago
Have to agree. Catering to HFT shops was already in 100 microsecond latency a few years ago using commodity hardware and software (cannot speak of Goldman Sachs but another one of similar ilk, can't imagine GS were much far away if not even better)
- mieseratte 7y ago
  Worked in an HFT shop and we were down to optimizing nanoseconds. Lots of folks had war stories about spending months to shave off tens of nanoseconds off pieces used millions of times per day.
  - 7y ago

1e-9 7y ago

Markets around the world are determining prices on a massive variety of instruments that derive value from the current and future value of products such as currencies, interest rates, equities, grains, livestock, metals, oil, gasoline, natural gas, and electricity. These prices allow us to prioritize resources, make fair transactions, and manage risk (i.e. buy insurance on the value of critical products so that we can plan and invest more effectively). The value of all these instruments are related to one another, as well as to realtime world events, in complex ways that no one entity has a perfect view of. Consequently, markets work by polling the expertise of many different parties who all understand a piece of how things should be valued. This results in millions, if not billions, of interconnected price-discovery feedback loops. The markets are kind of like a massive, distributed, realtime, ensemble, recursive predictor that performs much better than any one of its individual component algorithms could. The reason why shaving a few milliseconds (or even microseconds) can be beneficial is because the price discovery feedback loops get faster, which allows the system to determine a giant pricing vector that is more self-consistent, stable, and beneficial to the economy. It's similar to how increasing the sample rate of a feedback control system improves performance and stability. Providers of such benefits to the markets get rewarded through profit.

NohatCoder 7y ago
Meh, the short term price fluctuations are almost pure noise. Faster trading just incase the frequency of the noise. Increasing frequency only works if the feedback loops are stable.
- 1e-9 7y ago
  Over very short time scales, the price fluctuations vary from almost pure noise to almost pure signal. The almost pure noise situation occurs far more often, but even then, with good estimation techniques, you can extract a signal component that can improve pricing a small amount. Improving many interrelated prices by small amounts can lead to a significant overall improvement to the markets. The almost pure signal situation occurs far less often, but it is almost always extremely important to overall stability. Handling it well can minimize market overreactions and extreme events such as flash crashes and market panics.

sfthrower89 7y ago

We thought we were going to have healthcare, robots, and flying cars. Instead we have wasted two generations' worth of intellect on .. advertisements and fast stock trading. I'm ready for the giant asteroid.

tosh 7y ago

A great read on high frequency trading: Barbarians at the Gate https://queue.acm.org/detail.cfm?id=2536492 https://queue.acm.org/detail.cfm?id=2536492

harryh 7y ago
Title is actually Barbarians at the Gateways which is, of course, as play on words on the actual Barbarians at the Gate: https://en.wikipedia.org/wiki/Barbarians_at_the_Gate:_The_Fall_of_RJR_Nabisco https://en.wikipedia.org/wiki/Barbarians_at_the_Gate:_The_Fa... Another great tail of finance that operated at much slower speeds.

dp141 7y ago

Amazing to me how much impassioned defence there is of HFT. I assume a lot of the HN readership works in HFT.

gekkeboom 7y ago

This is called rent seeking. Just siphoning money from society without any benefit.

decoyworker 7y ago

Profiting in this way seems to be a bastardization of the purpose of the market and provides no value.

seanhunter 7y ago

For what it's worth, I used to work for GS and was in the algorithmic brokerage business for 2009-2010. One thing a lot of commenters on this thread are missing here is this isn't the same as an HFT or hedge fund, this is the brokerage business - executing orders on behalf of GSs clients, who are largely institutional investors, hedge and pension funds etc. The real-world impact of increased speed of execution by brokers is less money left on the table for HFTs to snap up and less slippage to the actual economic beneficiary (ie the actual retail investor or pension fund investing people's money gets a better trade).

jfengel 7y ago
Does GS do HFT itself? Is there a conflict of interest?
- seanhunter 7y ago
  I'm not a current employee, but at the time that I was, they did have a small unit who did pseudo-HFT on a prop basis and various electronic market-making businesses, but they were all firewalled off from the people who did brokerage business. We didn't share code or information with them and for obvious reasons they had no ability to view client order flow, unexecuted orders etc. Likewise people in the high-touch (ie voice) execution business couldn't see anything in the GSAT business even though they could use our algos to execute orders if they want to. My understanding (could be incorrect) is that their quant trading business (what they called their 'HFT' shop, although it wasn't really high-frequency compared to real HFTs like Winton, Knight, Jump, Citadel or whatever) was probably going to get shuttered as they moved out of proprietary risk-taking generally, but I don't have any information either way. In GSAT at the time, compared to others on the street our tech was pretty sophisticated intellectually but not fast (eg we didn't have ultrafast marketdata, our exchange latency was quite high and our execution algo speed was pretty slow) and so we had to do a lot of smart coding to prevent ourselves being ripped off by actual HFTs given they could move so much faster than we could. There's a lot more to HFT than reg NMS by the way, I was working in London, so we did all the GSAT trading on European exchanges none of which has anything crazy like reg NMS and there was still HFT shenanigans of various kinds that people would try (eg timing arbitrages if they could see that you had different execution speeds on different venues etc).

charlixcx 7y ago

Well it sounds like a lot of money, but probably isn't even THAT much for GS. Like, flops for that amount of money happen in hollywood movies quite often.

ausbah 7y ago

To me this is incredibly ridiculous, millions more dumped into the brain draining maw of finance.

taspeotis 7y ago
Forgive the AMP link but Bloomberg has a paywall of sorts now. https://www.google.com.au/amp/s/www.bloomberg.com/amp/opinion/articles/2019-03-11/you-need-a-big-tower-to-trade-stocks https://www.google.com.au/amp/s/www.bloomberg.com/amp/opinio... > And while Scientel clearly has high-frequency trading in mind—“the name that keeps coming up among industry sources is Citadel”—that’s not all it has in mind. It’s a tower that can send signals for lots of stuff. “Scientel has said it will equip its Aurora tower with 28 antennas—24 for public safety and municipal use and four for ‘private’ purposes.” People often complain that high-frequency trading encourages a socially wasteful arms race, and you certainly see some of that here, as lots of trading firms compete to put their antennas as close as possible to the CME servers. But another way to interpret this story is that high-frequency trading is subsidizing high-tech communications infrastructure for everyone else, building towers to send high-speed signals for public safety and municipal use just to justify a couple of high-frequency trading antennas.
- ausbah 7y ago
  Reminds me of this comic https://smbc-comics.com/index.php?id=3890 https://smbc-comics.com/index.php?id=3890

heelix 7y ago

I know a couple of the folks doing this sort of coding and while I'll tweak my APIs to be in the 10s of milliseconds they are tuning for nanosecond optimization. Crazy to see code where the speed of light is becoming a bottleneck.

tjpaudio 7y ago

I really wish we would curb this kind of behavior. Even if this speed is related to the brokerage business and not HFT, it is unfair to retail investors that large businesses have better market access. My solution: exchanges are required by law to process contracts in pulses with strict execution ordering rules. E.g., every 15 seconds process all buy and sells, processing from highest bid and lowest ask first for each security. Basically a turn-based stock market.

m0rphling 7y ago

Tom Scott did a nice video [1] on IEX's setup with 38 miles of fiber to add enough latency as a mechanism of fairness. Purportedly, this benefits all exchanges after it was successfully deployed. It seems like the incentives are not there for larger, more established exchanges to implement such blanket latency. So, firms like GS see a benefit to this kind of investment. [1] https://www.youtube.com/watch?v=d8BcCLLX4N4 https://www.youtube.com/watch?v=d8BcCLLX4N4

sytelus 7y ago

I would highly recommend the move Hummingbird Project which does a pretty good job of what it takes to build this kind of networks: https://www.youtube.com/watch?v=U8DweHM0Gnc https://www.youtube.com/watch?v=U8DweHM0Gnc