13 ms·

Markets around the world are determining prices on a massive variety of instruments that derive value from the current and future value of products such as curr

by 1e-9 7y ago

Markets around the world are determining prices on a massive variety of instruments that derive value from the current and future value of products such as currencies, interest rates, equities, grains, livestock, metals, oil, gasoline, natural gas, and electricity. These prices allow us to prioritize resources, make fair transactions, and manage risk (i.e. buy insurance on the value of critical products so that we can plan and invest more effectively). The value of all these instruments are related to one another, as well as to realtime world events, in complex ways that no one entity has a perfect view of. Consequently, markets work by polling the expertise of many different parties who all understand a piece of how things should be valued. This results in millions, if not billions, of interconnected price-discovery feedback loops. The markets are kind of like a massive, distributed, realtime, ensemble, recursive predictor that performs much better than any one of its individual component algorithms could. The reason why shaving a few milliseconds (or even microseconds) can be beneficial is because the price discovery feedback loops get faster, which allows the system to determine a giant pricing vector that is more self-consistent, stable, and beneficial to the economy. It's similar to how increasing the sample rate of a feedback control system improves performance and stability. Providers of such benefits to the markets get rewarded through profit.

NohatCoder 7y ago
Meh, the short term price fluctuations are almost pure noise. Faster trading just incase the frequency of the noise. Increasing frequency only works if the feedback loops are stable.
- 1e-9 7y ago
  Over very short time scales, the price fluctuations vary from almost pure noise to almost pure signal. The almost pure noise situation occurs far more often, but even then, with good estimation techniques, you can extract a signal component that can improve pricing a small amount. Improving many interrelated prices by small amounts can lead to a significant overall improvement to the markets. The almost pure signal situation occurs far less often, but it is almost always extremely important to overall stability. Handling it well can minimize market overreactions and extreme events such as flash crashes and market panics.
  - NohatCoder 7y ago
    The faster you can trade, the faster you can flash crash. The current state is that bots can crash the market faster than humans can react, how does that help stability?
    - 1e-9 7y ago
      The faster you trade, the faster you can detect and help limit or prevent a flash crash caused by others trading poorly.
WanderPanda 7y ago
This system is the only one unbiased estimator / decision maker humans ever found. Though it has quite high variance, in the long-term its results are just astounding
- jodrellblank 7y ago
  If it's unbiased, that's the same as saying it's random? If it leads towards efficiency, productivity, most beneficial allocation, as suggested, those are all biases.
  - spenczar5 7y ago
    An unbiased estimator is a technical term. It means that its errors in estimation are equally distributed above and below the true value. Edit to add: I am not sure GP was using the term accurately, either.
    - eyeundersand 7y ago
      I second this. An issue I have found with technical minutiae having,for lack of a better term, general names is that they are prone to being used wrong. And once a critical mass of persons start using it wrongly, there's no going back.
    - 1e-9 7y ago
      An unbiased estimator is one whose expected value equals the true value. Its sampling distribution can be asymmetrical with median not equal to the true value. You seem to be referring to a median-unbiased estimator.
      - spenczar5 7y ago
        You are correct, of course. It’s too late for me to correct my comment; I slipped up while trying to de-technicalize the definition. Thanks for the correction.
  - FabHK 7y ago
epiphanitus 7y ago
>> The markets are kind of like a massive, distributed, realtime, ensemble, recursive predictor that performs much better than any one of its individual component algorithms could. That's really interesting, I never thought of it that way. >> markets work by polling the expertise of many different parties who all understand a piece of how things should be valued. Does the whole picture ever become apparent to all of the interested parties after the fact? Or do market movements remain subject to a high degree of interpretation even after they happen?
- 1e-9 7y ago
  Nobody has the whole picture. I would say that the more extreme an individual event is, the easier it tends to be to understand in retrospect. This depends greatly upon one's analytical sophistication, level of market data, fundamental product understanding, and professional network (to know what happened in other firms).
  - raxxorrax 7y ago
    If nobody has the whole picture, how can we be sure if it's beneficial? edit: Or asked differently: What do we have now with high frequency trade established compared to the situation before?
    - growse 7y ago
      It's enormously cheaper to do business on markets nowadays. Back in the old days, spreads were sometimes multiple dollars on human-made markets with a lot of inventory. With HFT, people compete to offer the best market, and spreads are in the pennies.
      - deleted 7y ago
        [deleted]
- FabHK 7y ago
  That argument is basically Hayek's (and Mises's) answer to the economic calculation problem. Hayek argued (in "The Use of Knowledge in Society") that free markets are better than centralised planning, because of dispersed knowledge. https://en.wikipedia.org/wiki/Friedrich_Hayek#The_economic_calculation_problem https://en.wikipedia.org/wiki/Friedrich_Hayek#The_economic_c... https://en.wikipedia.org/wiki/The_Use_of_Knowledge_in_Society https://en.wikipedia.org/wiki/The_Use_of_Knowledge_in_Societ...
sumedh 7y ago
> The reason why shaving a few milliseconds (or even microseconds) can be beneficial is because the price discovery feedback loops get faster Berkshire Hathway has couple of trades every minute and difference in bid/ask prices is huge around $1000+, yet you dont see people complaining about that.
- 1e-9 7y ago
  The bid/ask spread is a separate issue from latency (Edit: although it should tend to be less with better price discovery). To appreciate the benefit of lower latency, I think you have to consider the bigger picture of many interrelated price discovery feedback loops involving many instruments. Many small speedups can result in a much more stable and beneficial system. In the case of Berkshire Hathaway, the value of their stock is dependent on the value of many other equities, interest rates, energies, raw materials, etc.
  - sumedh 7y ago
    > the value of their stock is dependent on the value of many other equities, interest rates, energies, raw materials, etc. Exactly, its the same with other stocks. > The bid/ask spread is a separate issue from latency. Yes but it also the second point made by HFT's that they reduce the spread.
    - ycombobreaker 7y ago
      Any market maker's bid-ask spread will be dictated by a combination of factors, including aspects like latency (relative to competitors), expectations on holding time, historical volatility and spread (that one is somewhat circular), availability of information from correlated markets, ability to hedge into correlated markets (and what are _those_ spreads, as the market maker will be crossing that spread to hedge), trading fees, etc. In markets where many of those aspects can be minimized, market makers will end up very tight and the dominant factor becomes latency. a fast market maker can offer a tighter spread to counterparties and will see a virtuous cycle of increased trading opportunities leading to revenue to stay fast. If the other factors remain significant risks, then the benefits of latency optimization fall off. So, spread and latency are related, but other market fundamentals do play a part.
antientropic 7y ago
Is there any empirical evidence that these perceived benefits to society actually ever materialize? It's clear that there is a benefit to a trader from knowing something milliseconds before the rest of the market (otherwise Goldman wouldn't be doing this), but it's not clear at all to me that it helps the rest of us.
- grenoire 7y ago
  We know, empirically, that a lack of liquidity increases trading costs, which in turn is directly channeled to the prices of goods and services that rely on this liquidity (more or less everything in the world, even more indirectly ones like education). It's difficult to say 'things would be X% more expensive' because of the interconnected complexity the GP was talking about, but there is definitely a very apparent benefit.
  - codetrotter 7y ago
    > a lack of liquidity increases trading costs Sure but, does a couple of milliseconds to or from affect that?
    - grenoire 7y ago
      You have to consider some form of multiplier effect that comes into play. Because the prices of assets are derivatives of others (incl. many, many other variables, of course) these millisecond savings end up turning into seconds, even minutes en masse.
    - dcolkitt 7y ago
      Yes, and I'll tell you exactly why. Markets consist of informed and uninformed traders. For example your typical Joe Sixpack rebalancing his 401k is uninformed. His trading does not tell you anything about the underlying value of the stocks. The typical informed trader is a hedge fund manager, who's investing tons of resources in gaining an informational edge. If he's buying a stock at a particular time that is in and of itself a signal that said stock is worth more than you thought it was otherwise. Liquidity providers love trading with uninformed traders. The problem with informed traders is that you don't want to be on the other side of their trades. Since liquidity providers are the immediate counterparties to most order flow, they're the ones that primarily eat this cost. To counter this, liquidity providers invest enormous resources in profiling order flow to try to identify when to what degree its informed. This allows them to provide lower costs and more liquidity to uninformed traders, like Joe Sixpack. It's analogous to how requiring a checkup allows life insurance companies to provide lower premiums, particularly to those who are healthy. One of the most important ways to profile order flow is to quickly adjust quotes as market conditions evolve. For example said hedge fund manager may be trading a thesis that the chipmakers are all undervalued. He may come in and buy Intel, AMD and Nvidia in one swoop. If an HFT sees a huge buy hit Intel, it can bump its quotes on AMD for a few milliseconds. If its a cigar-chomping hedge fund manager executing a basket algorithm, it's quite likely that he'll try to hit AMD and thus pay the higher price. But if its Joe Sixpack the probability that his trade just coincidentally lands in a 5 millisecond time window is vanishingly small.
ypcx 7y ago
The theory sounds great. But why then, our streets are lined with homeless, and our nations are stricken with poverty? Could it be that the only real aim and motivation of market traders is to earn money? One day, maybe.... when these are replaced with DAOs on the blockchain. But until then it's the Wolf of Wall Street.
- grenoire 7y ago
  The markets on which financial instruments are traded are effectively separate from the ones where policy decisions are being made on, such as labour, vocational education, and healthcare. It's not the derivatives trading that is problematic, it's the market inefficiencies not being resolved by governments.
- JDiculous 7y ago
  Because having homeless people lining our streets on our commutes to/from our jobs is a daily reminder that if we don't work hard enough to increase corporate profits, then our bosses might lay us off and we'll end up like them. That or moral apathy. At some point in the 80s we decided that markets driven by business profits should dictate every aspect of society. I imagine 100 years from now they'll look back at today in disgust.
  - assblaster 7y ago
    100 years after Communism burst on the scene, we currently look at that development with disgust. Right now most people have access to abundant food, cellphone in every pocket, access to a wealth of information, access to transportation, incredible medical advances. I can't imagine that the progress we've made would be scorned. Like other market driven forces, bad players will not be rewarded as information about them increases. However, if information is not increased because of something like a company buying a newspaper so investigative threats can be used against politicians to avoid information gathering, then we have problems. This is more crony capitalism than just capitalism.
    - annabellish 7y ago
dalbasal 7y ago
The upshot of this argument is that this is valuable activity. We need markets to price tradable assets and provide liquidity. The counterargument is that there are diminishing and/or negative returns to increased liquidity and velocity. Take just stocks. Liquidity is not a problem. You have liquidity whether trades take minutes or milliseconds. Pricing? I'd say we have pricing covered too, at least the pricing that more/faster algorithmic trading will contribute. Meanwhile, all this stuff costs money, people, resources that aren't available for actual productive work instead of overhead.
- lallysingh 7y ago
  > Meanwhile, all this stuff costs money, people, resources that aren't available for actual productive work instead of overhead. That's a very high standard. What's productive? What's productive enough, in your book, to be worth the effort used here?
  - dalbasal 7y ago
    Median productiveness is plenty, assuming this is not productive activity.
    - lallysingh 7y ago
      But what's "productive?". Farming? Manufacturing? What if the farmed food is unhealthy? If the manufactured good is wasteful?
- 1e-9 7y ago
  My view is that current liquidity and pricing are far from perfect, particularly when it matters most (e.g. a market panic) and particularly in the global context of the vast universe of interrelated instruments that need better relative pricing. Given that we benefit from realtime pricing, milliseconds matter when you must determine a large vector of prices with complex dependencies using the ensemble recursive system that is the global market.
  - 7y ago
justinjlynn 7y ago
Yes, that's the model. So, tell me. Would any profit be possible in an ideally functioning market? If so, how is any item worth more than the total cost of all inputs including externalities? If not, how is such a dynamic system attracted to an ideal state, given that those who would make the market ideally functional are best placed to gain from market inefficiency and dysfunction? Given an answer, do you have a sound game theory explanation for this? Additionally, what is the nyquist limit for such an ideally realised market, if it is indeed to be modelled as a recursive sampled approximator and how is this derived? Given an infinitely recursive network of arbitrarily connected market agents, is any such calculation convergent? If so, why? If not, how does the market ever converge to any appropriate price - a price which accurately reflects the market conditions excluding pricing operations and market costs which aren't directly related to the production of the instrument in question? Keep in mind that, if the market is functioning ideally no market participant will exceed the nyquist rate as all participants knowledge of market conditions converges to zero. How is any sampling rate, excluding zero, convergent? If not, how is any such market realisable? If so, what is the loss function between ideal model and realisable, perfectly imperfect real world implementation? What is the minimum profit, if not zero, and why? However, it does seem that arbitrage opportunities decrease when such high-speed trading is occurring and, does so even more quickly the faster trading speed and market sampling are increased. How can we account for this, if not by increased market efficiency? I conjecture that, by ever increasing the sampling rate and the speed at which transactions complete, markets are not being made more efficient. Instead, I hypothesise that, as markets directly effect the price of the instrument reflexively, the feedback latency produced creates relative local pockets of perceived value - which are only profitable trades in relation to local information asymmetry. As the vast majority of high-speed trading holds market positions on extremely short time scales, shifting exposure constantly, this profit is immediately realised locally resulting in the gradual diffuision of this inefficiency as the increase in price of all instruments. This is a direct result of the cost of trading being factored directly into the agent's local acceptable sale price of held instruments. Every local agent trading action is ideal, but the global market is a divergently inefficient one. Indeed, it is a market in which its pricing inefficiency is maximally concealed from all market participants. In a sense, I conjecture that the estimator is not functioning to increase market efficiency but is, instead amplifying local inefficiency globally, in effect, much like a charge pump would operate in a voltage multiplier circuit. In essence such a scheme acts to conceal increased market cost and overhead (including the profit of market participants) into market instrument pricing. However, it does so in an extremely small and diffuse way so as to make the rise in price of a single instrument, as a result of this activity, extremely difficult to detect as all instruments increase similarly on the same time scale. This behaviour appears to be similar in nature to 'salami slicing', an often effective embezzlement technique - except that, instead of exploiting an information asymmetry created by lack of interest in small quantities in the part of auditing accountants, it exploits the information asymmetry created by the speed of light itself. Of course, the faster the sampling rate, the more efficient the described amplification process would take place. Does this effect correlate between markets with differing but estimable information asymmetry? If there is no correlation, this hypothesis is invalid. It would seem to be an area ripe for research and analysis of market data. Do you see any technical issue with this conjecture by which we may discount it immediately?
raxxorrax 7y ago
That gives these mechanisms way too much credit. Markets can benefit from a more accurate valuation but ms-response times are not needed for this. The real reason are competitors. You have an advantage if you have the faster line. In the name of fairness there are lines of the exact same length* in many trade centers precisely for this reason. There are bots that feign transaction so that others react in a specific way. In the last moment these are canceled again, too late for competitors to still react. This is an example for when you need a faster line. To suggest this arms race in high frequency trade has a serious economic benefit is ridiculous in my opinion. * I do literally mean cable length. Yes, they have become that crazy
- bostik 7y ago
  > There are bots that feign transaction so that others react in a specific way. In the last moment these are canceled again, too late for competitors to still react. In stock exchanges at least, what you describe is called spoofing. The regulators are not friendly towards it, because spoofing enables to skew price discovery. It's generally hard to detect and hard to prove, but there have been cases where the regulators have proved sufficiently well that in certain cases, there were orders never intended to be executed. And yes, if you get caught, there are sanctions. https://en.wikipedia.org/wiki/Spoofing_(finance) https://en.wikipedia.org/wiki/Spoofing_(finance)
branchless 7y ago
No, this is a pointless waste of time. Source: work in low latency tech.
pedrocr 7y ago
I very seriously doubt you'd lose any of those benefits by just trading on 10 second or even 1 minute increments. Everyone has a bunch of time to submit orders, the auction is ran, and then everyone gets the result back. I'd really like to see a use case in any of those real-world markets for pricing with sub-second granularity. It seems much more likely that trading on sub-second granularity is just extracting money from the actual participants in those markets into the pockets of financial firms with no value returned.
- growse 7y ago
  All that'd happen if you did that would be to increase the risk of any market maker having the wrong price at execution time (they have less data to make an informed price discovery), so you'd get wider spreads to compensate. Wider spreads just makes it more expensive for everyone. Personally, I'd like my pension money going towards the actual investment rather than paying for a wider spread, but I'm just strange.
  - trophycase 7y ago
    If you're holding for a pension, wouldn't a wider spread affect you less than any single other market participant? Like a 1 cent different due to spread isn't going to matter in 30 years?
    - growse 7y ago
      If I'm holding a pension, I'm paying in regularly to a fund that's managed over a long period of time. If they manage it actively and trade it a lot over 30 years, 1 cent a time will add up.
      - FabHK 7y ago
        That's not what they're doing, though. The costs will likely be minimal. Spreads used to be much higher, sure, but that was not because there were no HFT shops around back then.
  - 7y ago
patientplatypus 7y ago
If the market is so efficient why can't I afford a house or healthcare? Last I checked, Amazon's P/E ratio is over 80 and Apple's is over 100. Given that we have 10 years to avoid a climate apocalypse (that the market created because it doesn't account for externalities) that means for Apple to be rationally priced their earnings must go up ~10x a year for a decade. Your flowery language aside western capitalism is so fundamentally broken that even an idiot like me can see it. Why can't you? PS fuck Goldman Sachs.
- jodrellblank 7y ago
  Reading that and getting hooked on the narrative because the whole thing sounds like the workings of a giant AI brain and is too cool to refute, then they quietly tack on "and beneficial to the economy"; we economic laypeople are supposed to take that on faith, as if hearing a theologian prescribe how to live and concluding with "and beneficial to your immortal soul". Who are we to argue?
turk73 7y ago
Oh yeah, and make obscene amounts of money, don't forget that part.
bumby 7y ago
Is there any evidence defining where the speed of information reaches a diminishing return? To dilate on your feedback loop comment, physical systems may benefit from a higher sampling rate but usually only to a point. This point is often related to the physical dynamics of the system (e.g., natural frequencies). For example, a small thruster may benefit much more from increasing sampling rates from 1000Hz to 10kHz than a large rocket engine. I assume/wonder if there's a similar analogy to diminished returns in stock information systems, like more volatile markets benefiting more from higher frequency of data. It would be interesting to see where the diminishing returns are.
Junk_Collector 7y ago
I just have a nitpick with your analogy to feedback loops. Assuming the market can be said to have a Nyquist rate, then once you hit that you have all relevant information. Increasing the sample rate past Nyquist does not make a system more stable unless you have a very specific system designed specifically to take advantage of that. More typically, it just increases your noise-bandwidth product and can decrease total system stability and accuracy.
- bumby 7y ago
  I agree with your comment but am curious on how far this analogy goes. What do you suppose defines the Nyquist frequency in a market?
  - Junk_Collector 7y ago
    To stretch the analogy a little further, to take an observation of the market, you have to buy or sell which in turn affects the price. It's very much like the effect of measuring a quantum particle, the impact of a photon is enough to change to observation so you have an inherent uncertainty to everything. Because of this, I don't think you can ever distinguish between noise and meaningful trading with respect to a Nyquist limit in the market. If you tried to for instance, look at the price of all stocks over time and find the 2-D frequency function required to represent that, it is going to creep up based on your measurement rate. Another way to describe it might be that your observability and controllability vectors are not orthogonal. This also skates around the issue of defining what a Nyquist rate of the market even means in real terms. But, if you want to use control theory to model the market, it's important to know that faster does not inherently mean more accurate. In the simple analogy, increasing trading frequency will improve measurement results, up to a point, after which it will actually likely result in decreasing accuracy. I've also ignored the whole conflation of frequency and group delay in these analogies to keep things simpler as well.
    - bumby 7y ago
      I don't want to oversimplify your stance, I'm just trying to distill it to something I understand. In the analogy above, would you think this tipping point where the signal is no longer improved is related to the volume of the buyer compared to the overall volume? In other words, Warren Buffet buying a huge portion of a penny stock will drive past that frequency tipping point easily while lil ol' me buying a few shares of an index fund will have effectively little to no impact?
randomsearch 7y ago
Is this sarcasm?
HSO 7y ago
This is such a confident statement, and I don't mean that as a compliment. For starters, is the evidence behind this Hayekian market efficiency really so strong as to warrant this kind of absolute confidence in the wisdom of markets? > markets work by polling the expertise of many different parties who all understand a piece of how things should be valued. …as well as orders of magnitude more people who do not understand how things should be valued. → noise, which is fine ("excess volatility"), but which can also become highly persistent in the presence of correlated expectations ("bubbles") > This results in millions, if not billions, of interconnected price-discovery feedback loops. Well, there are negative and positive feedback loops, only one of which is stabilizing! > beneficial […] because the price discovery feedback loops get faster This can also backfire. In fact, this is why a number of stock markets have instituted a trading stop if an asset moves "too fast". Slowing things down / reducing liquidity can stabilize a situation. Actually, this reminds me of [1] W. A. Brock, C. H. Hommes, and F. O. Wagener. More hedging instruments may destabilize markets. Journal of Economic Dynamics and Control, 33:1912–1928, 2009. where you have a similar counterintuitive argument. The history of the idea of market efficiency is long and the idea remains controversial or contested. See e.g. Philip Mirowski's writings.
- 1e-9 7y ago
  You misread my post. I never said the markets were efficient. On the contrary, they are generally far from it. What I said was that the markets combine the efforts of many different entities in order to determine prices in a way that is superior to what any one entity could do. > Well, there are negative and positive feedback loops, only one of which is stabilizing! Absolutely. Entities that consistently contribute positive feedback cause harm to markets and they are generally doing something that is either prohibited or foolish. I don't consider either a good long term profit strategy. The market regulation departments work to remove one and large losses tend to remove the other. > This can also backfire. In fact, this is why a number of stock markets have instituted a trading stop if an asset moves "too fast". Slowing things down / reducing liquidity can stabilize a situation. Sure, exchanges use a variety of market integrity controls, including limits on rapid and/or large price changes that can trigger order rejections or trading halts. These controls can be beneficial when the price fluctuation was due to poor trading, but can be damaging to a market when the fluctuation was due to significant new information or because there is a natural high volatility situation such as a derivative that is about to expire or is rarely traded. Consequently, the exchanges have to be careful about how and when halts are invoked. Some exchanges often get it wrong. The main point I was making is that lowering the latency of the multitude of price discovery feedback loops making up the global market can be very beneficial because it allows the pricing dependencies to be more fully determined.
posterboy 7y ago
This is an accurate description of the game theoretic argument. It is not very precise about the premisses, "the market", "the reward" or "the critical products"--variables in a non-linear equation, so to speak, that do not necessarily have a unique solution, or no solution.

by 1e-9 7y ago

NohatCoder 7y ago

Meh, the short term price fluctuations are almost pure noise. Faster trading just incase the frequency of the noise. Increasing frequency only works if the feedback loops are stable.

1e-9 7y ago
Over very short time scales, the price fluctuations vary from almost pure noise to almost pure signal. The almost pure noise situation occurs far more often, but even then, with good estimation techniques, you can extract a signal component that can improve pricing a small amount. Improving many interrelated prices by small amounts can lead to a significant overall improvement to the markets. The almost pure signal situation occurs far less often, but it is almost always extremely important to overall stability. Handling it well can minimize market overreactions and extreme events such as flash crashes and market panics.
- NohatCoder 7y ago
  The faster you can trade, the faster you can flash crash. The current state is that bots can crash the market faster than humans can react, how does that help stability?
  - 1e-9 7y ago
    The faster you trade, the faster you can detect and help limit or prevent a flash crash caused by others trading poorly.

WanderPanda 7y ago

This system is the only one unbiased estimator / decision maker humans ever found. Though it has quite high variance, in the long-term its results are just astounding

jodrellblank 7y ago
If it's unbiased, that's the same as saying it's random? If it leads towards efficiency, productivity, most beneficial allocation, as suggested, those are all biases.
- spenczar5 7y ago
  An unbiased estimator is a technical term. It means that its errors in estimation are equally distributed above and below the true value. Edit to add: I am not sure GP was using the term accurately, either.
  - eyeundersand 7y ago
    I second this. An issue I have found with technical minutiae having,for lack of a better term, general names is that they are prone to being used wrong. And once a critical mass of persons start using it wrongly, there's no going back.
  - 1e-9 7y ago
    An unbiased estimator is one whose expected value equals the true value. Its sampling distribution can be asymmetrical with median not equal to the true value. You seem to be referring to a median-unbiased estimator.
    - spenczar5 7y ago
      You are correct, of course. It’s too late for me to correct my comment; I slipped up while trying to de-technicalize the definition. Thanks for the correction.
- FabHK 7y ago

epiphanitus 7y ago

>> The markets are kind of like a massive, distributed, realtime, ensemble, recursive predictor that performs much better than any one of its individual component algorithms could. That's really interesting, I never thought of it that way. >> markets work by polling the expertise of many different parties who all understand a piece of how things should be valued. Does the whole picture ever become apparent to all of the interested parties after the fact? Or do market movements remain subject to a high degree of interpretation even after they happen?

1e-9 7y ago
Nobody has the whole picture. I would say that the more extreme an individual event is, the easier it tends to be to understand in retrospect. This depends greatly upon one's analytical sophistication, level of market data, fundamental product understanding, and professional network (to know what happened in other firms).
- raxxorrax 7y ago
  If nobody has the whole picture, how can we be sure if it's beneficial? edit: Or asked differently: What do we have now with high frequency trade established compared to the situation before?
  - growse 7y ago
    It's enormously cheaper to do business on markets nowadays. Back in the old days, spreads were sometimes multiple dollars on human-made markets with a lot of inventory. With HFT, people compete to offer the best market, and spreads are in the pennies.
    - deleted 7y ago
      [deleted]
FabHK 7y ago
That argument is basically Hayek's (and Mises's) answer to the economic calculation problem. Hayek argued (in "The Use of Knowledge in Society") that free markets are better than centralised planning, because of dispersed knowledge. https://en.wikipedia.org/wiki/Friedrich_Hayek#The_economic_calculation_problem https://en.wikipedia.org/wiki/Friedrich_Hayek#The_economic_c... https://en.wikipedia.org/wiki/The_Use_of_Knowledge_in_Society https://en.wikipedia.org/wiki/The_Use_of_Knowledge_in_Societ...

sumedh 7y ago

> The reason why shaving a few milliseconds (or even microseconds) can be beneficial is because the price discovery feedback loops get faster Berkshire Hathway has couple of trades every minute and difference in bid/ask prices is huge around $1000+, yet you dont see people complaining about that.

1e-9 7y ago
The bid/ask spread is a separate issue from latency (Edit: although it should tend to be less with better price discovery). To appreciate the benefit of lower latency, I think you have to consider the bigger picture of many interrelated price discovery feedback loops involving many instruments. Many small speedups can result in a much more stable and beneficial system. In the case of Berkshire Hathaway, the value of their stock is dependent on the value of many other equities, interest rates, energies, raw materials, etc.
- sumedh 7y ago
  > the value of their stock is dependent on the value of many other equities, interest rates, energies, raw materials, etc. Exactly, its the same with other stocks. > The bid/ask spread is a separate issue from latency. Yes but it also the second point made by HFT's that they reduce the spread.
  - ycombobreaker 7y ago
    Any market maker's bid-ask spread will be dictated by a combination of factors, including aspects like latency (relative to competitors), expectations on holding time, historical volatility and spread (that one is somewhat circular), availability of information from correlated markets, ability to hedge into correlated markets (and what are _those_ spreads, as the market maker will be crossing that spread to hedge), trading fees, etc. In markets where many of those aspects can be minimized, market makers will end up very tight and the dominant factor becomes latency. a fast market maker can offer a tighter spread to counterparties and will see a virtuous cycle of increased trading opportunities leading to revenue to stay fast. If the other factors remain significant risks, then the benefits of latency optimization fall off. So, spread and latency are related, but other market fundamentals do play a part.

antientropic 7y ago

Is there any empirical evidence that these perceived benefits to society actually ever materialize? It's clear that there is a benefit to a trader from knowing something milliseconds before the rest of the market (otherwise Goldman wouldn't be doing this), but it's not clear at all to me that it helps the rest of us.

grenoire 7y ago
We know, empirically, that a lack of liquidity increases trading costs, which in turn is directly channeled to the prices of goods and services that rely on this liquidity (more or less everything in the world, even more indirectly ones like education). It's difficult to say 'things would be X% more expensive' because of the interconnected complexity the GP was talking about, but there is definitely a very apparent benefit.
- codetrotter 7y ago
  > a lack of liquidity increases trading costs Sure but, does a couple of milliseconds to or from affect that?
  - grenoire 7y ago
    You have to consider some form of multiplier effect that comes into play. Because the prices of assets are derivatives of others (incl. many, many other variables, of course) these millisecond savings end up turning into seconds, even minutes en masse.
  - dcolkitt 7y ago
    Yes, and I'll tell you exactly why. Markets consist of informed and uninformed traders. For example your typical Joe Sixpack rebalancing his 401k is uninformed. His trading does not tell you anything about the underlying value of the stocks. The typical informed trader is a hedge fund manager, who's investing tons of resources in gaining an informational edge. If he's buying a stock at a particular time that is in and of itself a signal that said stock is worth more than you thought it was otherwise. Liquidity providers love trading with uninformed traders. The problem with informed traders is that you don't want to be on the other side of their trades. Since liquidity providers are the immediate counterparties to most order flow, they're the ones that primarily eat this cost. To counter this, liquidity providers invest enormous resources in profiling order flow to try to identify when to what degree its informed. This allows them to provide lower costs and more liquidity to uninformed traders, like Joe Sixpack. It's analogous to how requiring a checkup allows life insurance companies to provide lower premiums, particularly to those who are healthy. One of the most important ways to profile order flow is to quickly adjust quotes as market conditions evolve. For example said hedge fund manager may be trading a thesis that the chipmakers are all undervalued. He may come in and buy Intel, AMD and Nvidia in one swoop. If an HFT sees a huge buy hit Intel, it can bump its quotes on AMD for a few milliseconds. If its a cigar-chomping hedge fund manager executing a basket algorithm, it's quite likely that he'll try to hit AMD and thus pay the higher price. But if its Joe Sixpack the probability that his trade just coincidentally lands in a 5 millisecond time window is vanishingly small.

ypcx 7y ago

The theory sounds great. But why then, our streets are lined with homeless, and our nations are stricken with poverty? Could it be that the only real aim and motivation of market traders is to earn money? One day, maybe.... when these are replaced with DAOs on the blockchain. But until then it's the Wolf of Wall Street.

grenoire 7y ago
The markets on which financial instruments are traded are effectively separate from the ones where policy decisions are being made on, such as labour, vocational education, and healthcare. It's not the derivatives trading that is problematic, it's the market inefficiencies not being resolved by governments.
JDiculous 7y ago
Because having homeless people lining our streets on our commutes to/from our jobs is a daily reminder that if we don't work hard enough to increase corporate profits, then our bosses might lay us off and we'll end up like them. That or moral apathy. At some point in the 80s we decided that markets driven by business profits should dictate every aspect of society. I imagine 100 years from now they'll look back at today in disgust.
- assblaster 7y ago
  100 years after Communism burst on the scene, we currently look at that development with disgust. Right now most people have access to abundant food, cellphone in every pocket, access to a wealth of information, access to transportation, incredible medical advances. I can't imagine that the progress we've made would be scorned. Like other market driven forces, bad players will not be rewarded as information about them increases. However, if information is not increased because of something like a company buying a newspaper so investigative threats can be used against politicians to avoid information gathering, then we have problems. This is more crony capitalism than just capitalism.
  - annabellish 7y ago

dalbasal 7y ago

The upshot of this argument is that this is valuable activity. We need markets to price tradable assets and provide liquidity. The counterargument is that there are diminishing and/or negative returns to increased liquidity and velocity. Take just stocks. Liquidity is not a problem. You have liquidity whether trades take minutes or milliseconds. Pricing? I'd say we have pricing covered too, at least the pricing that more/faster algorithmic trading will contribute. Meanwhile, all this stuff costs money, people, resources that aren't available for actual productive work instead of overhead.

lallysingh 7y ago
> Meanwhile, all this stuff costs money, people, resources that aren't available for actual productive work instead of overhead. That's a very high standard. What's productive? What's productive enough, in your book, to be worth the effort used here?
- dalbasal 7y ago
  Median productiveness is plenty, assuming this is not productive activity.
  - lallysingh 7y ago
    But what's "productive?". Farming? Manufacturing? What if the farmed food is unhealthy? If the manufactured good is wasteful?
1e-9 7y ago
My view is that current liquidity and pricing are far from perfect, particularly when it matters most (e.g. a market panic) and particularly in the global context of the vast universe of interrelated instruments that need better relative pricing. Given that we benefit from realtime pricing, milliseconds matter when you must determine a large vector of prices with complex dependencies using the ensemble recursive system that is the global market.
- 7y ago

justinjlynn 7y ago

Yes, that's the model. So, tell me. Would any profit be possible in an ideally functioning market? If so, how is any item worth more than the total cost of all inputs including externalities? If not, how is such a dynamic system attracted to an ideal state, given that those who would make the market ideally functional are best placed to gain from market inefficiency and dysfunction? Given an answer, do you have a sound game theory explanation for this? Additionally, what is the nyquist limit for such an ideally realised market, if it is indeed to be modelled as a recursive sampled approximator and how is this derived? Given an infinitely recursive network of arbitrarily connected market agents, is any such calculation convergent? If so, why? If not, how does the market ever converge to any appropriate price - a price which accurately reflects the market conditions excluding pricing operations and market costs which aren't directly related to the production of the instrument in question? Keep in mind that, if the market is functioning ideally no market participant will exceed the nyquist rate as all participants knowledge of market conditions converges to zero. How is any sampling rate, excluding zero, convergent? If not, how is any such market realisable? If so, what is the loss function between ideal model and realisable, perfectly imperfect real world implementation? What is the minimum profit, if not zero, and why? However, it does seem that arbitrage opportunities decrease when such high-speed trading is occurring and, does so even more quickly the faster trading speed and market sampling are increased. How can we account for this, if not by increased market efficiency? I conjecture that, by ever increasing the sampling rate and the speed at which transactions complete, markets are not being made more efficient. Instead, I hypothesise that, as markets directly effect the price of the instrument reflexively, the feedback latency produced creates relative local pockets of perceived value - which are only profitable trades in relation to local information asymmetry. As the vast majority of high-speed trading holds market positions on extremely short time scales, shifting exposure constantly, this profit is immediately realised locally resulting in the gradual diffuision of this inefficiency as the increase in price of all instruments. This is a direct result of the cost of trading being factored directly into the agent's local acceptable sale price of held instruments. Every local agent trading action is ideal, but the global market is a divergently inefficient one. Indeed, it is a market in which its pricing inefficiency is maximally concealed from all market participants. In a sense, I conjecture that the estimator is not functioning to increase market efficiency but is, instead amplifying local inefficiency globally, in effect, much like a charge pump would operate in a voltage multiplier circuit. In essence such a scheme acts to conceal increased market cost and overhead (including the profit of market participants) into market instrument pricing. However, it does so in an extremely small and diffuse way so as to make the rise in price of a single instrument, as a result of this activity, extremely difficult to detect as all instruments increase similarly on the same time scale. This behaviour appears to be similar in nature to 'salami slicing', an often effective embezzlement technique - except that, instead of exploiting an information asymmetry created by lack of interest in small quantities in the part of auditing accountants, it exploits the information asymmetry created by the speed of light itself. Of course, the faster the sampling rate, the more efficient the described amplification process would take place. Does this effect correlate between markets with differing but estimable information asymmetry? If there is no correlation, this hypothesis is invalid. It would seem to be an area ripe for research and analysis of market data. Do you see any technical issue with this conjecture by which we may discount it immediately?

raxxorrax 7y ago

That gives these mechanisms way too much credit. Markets can benefit from a more accurate valuation but ms-response times are not needed for this. The real reason are competitors. You have an advantage if you have the faster line. In the name of fairness there are lines of the exact same length* in many trade centers precisely for this reason. There are bots that feign transaction so that others react in a specific way. In the last moment these are canceled again, too late for competitors to still react. This is an example for when you need a faster line. To suggest this arms race in high frequency trade has a serious economic benefit is ridiculous in my opinion. * I do literally mean cable length. Yes, they have become that crazy

bostik 7y ago
> There are bots that feign transaction so that others react in a specific way. In the last moment these are canceled again, too late for competitors to still react. In stock exchanges at least, what you describe is called spoofing. The regulators are not friendly towards it, because spoofing enables to skew price discovery. It's generally hard to detect and hard to prove, but there have been cases where the regulators have proved sufficiently well that in certain cases, there were orders never intended to be executed. And yes, if you get caught, there are sanctions. https://en.wikipedia.org/wiki/Spoofing_(finance) https://en.wikipedia.org/wiki/Spoofing_(finance)

branchless 7y ago

No, this is a pointless waste of time. Source: work in low latency tech.

pedrocr 7y ago

I very seriously doubt you'd lose any of those benefits by just trading on 10 second or even 1 minute increments. Everyone has a bunch of time to submit orders, the auction is ran, and then everyone gets the result back. I'd really like to see a use case in any of those real-world markets for pricing with sub-second granularity. It seems much more likely that trading on sub-second granularity is just extracting money from the actual participants in those markets into the pockets of financial firms with no value returned.

growse 7y ago
All that'd happen if you did that would be to increase the risk of any market maker having the wrong price at execution time (they have less data to make an informed price discovery), so you'd get wider spreads to compensate. Wider spreads just makes it more expensive for everyone. Personally, I'd like my pension money going towards the actual investment rather than paying for a wider spread, but I'm just strange.
- trophycase 7y ago
  If you're holding for a pension, wouldn't a wider spread affect you less than any single other market participant? Like a 1 cent different due to spread isn't going to matter in 30 years?
  - growse 7y ago
    If I'm holding a pension, I'm paying in regularly to a fund that's managed over a long period of time. If they manage it actively and trade it a lot over 30 years, 1 cent a time will add up.
    - FabHK 7y ago
      That's not what they're doing, though. The costs will likely be minimal. Spreads used to be much higher, sure, but that was not because there were no HFT shops around back then.
- 7y ago

patientplatypus 7y ago

If the market is so efficient why can't I afford a house or healthcare? Last I checked, Amazon's P/E ratio is over 80 and Apple's is over 100. Given that we have 10 years to avoid a climate apocalypse (that the market created because it doesn't account for externalities) that means for Apple to be rationally priced their earnings must go up ~10x a year for a decade. Your flowery language aside western capitalism is so fundamentally broken that even an idiot like me can see it. Why can't you? PS fuck Goldman Sachs.

jodrellblank 7y ago
Reading that and getting hooked on the narrative because the whole thing sounds like the workings of a giant AI brain and is too cool to refute, then they quietly tack on "and beneficial to the economy"; we economic laypeople are supposed to take that on faith, as if hearing a theologian prescribe how to live and concluding with "and beneficial to your immortal soul". Who are we to argue?

turk73 7y ago

Oh yeah, and make obscene amounts of money, don't forget that part.

bumby 7y ago

Is there any evidence defining where the speed of information reaches a diminishing return? To dilate on your feedback loop comment, physical systems may benefit from a higher sampling rate but usually only to a point. This point is often related to the physical dynamics of the system (e.g., natural frequencies). For example, a small thruster may benefit much more from increasing sampling rates from 1000Hz to 10kHz than a large rocket engine. I assume/wonder if there's a similar analogy to diminished returns in stock information systems, like more volatile markets benefiting more from higher frequency of data. It would be interesting to see where the diminishing returns are.

Junk_Collector 7y ago

I just have a nitpick with your analogy to feedback loops. Assuming the market can be said to have a Nyquist rate, then once you hit that you have all relevant information. Increasing the sample rate past Nyquist does not make a system more stable unless you have a very specific system designed specifically to take advantage of that. More typically, it just increases your noise-bandwidth product and can decrease total system stability and accuracy.

bumby 7y ago
I agree with your comment but am curious on how far this analogy goes. What do you suppose defines the Nyquist frequency in a market?
- Junk_Collector 7y ago
  To stretch the analogy a little further, to take an observation of the market, you have to buy or sell which in turn affects the price. It's very much like the effect of measuring a quantum particle, the impact of a photon is enough to change to observation so you have an inherent uncertainty to everything. Because of this, I don't think you can ever distinguish between noise and meaningful trading with respect to a Nyquist limit in the market. If you tried to for instance, look at the price of all stocks over time and find the 2-D frequency function required to represent that, it is going to creep up based on your measurement rate. Another way to describe it might be that your observability and controllability vectors are not orthogonal. This also skates around the issue of defining what a Nyquist rate of the market even means in real terms. But, if you want to use control theory to model the market, it's important to know that faster does not inherently mean more accurate. In the simple analogy, increasing trading frequency will improve measurement results, up to a point, after which it will actually likely result in decreasing accuracy. I've also ignored the whole conflation of frequency and group delay in these analogies to keep things simpler as well.
  - bumby 7y ago
    I don't want to oversimplify your stance, I'm just trying to distill it to something I understand. In the analogy above, would you think this tipping point where the signal is no longer improved is related to the volume of the buyer compared to the overall volume? In other words, Warren Buffet buying a huge portion of a penny stock will drive past that frequency tipping point easily while lil ol' me buying a few shares of an index fund will have effectively little to no impact?

randomsearch 7y ago

Is this sarcasm?

HSO 7y ago

This is such a confident statement, and I don't mean that as a compliment. For starters, is the evidence behind this Hayekian market efficiency really so strong as to warrant this kind of absolute confidence in the wisdom of markets? > markets work by polling the expertise of many different parties who all understand a piece of how things should be valued. …as well as orders of magnitude more people who do not understand how things should be valued. → noise, which is fine ("excess volatility"), but which can also become highly persistent in the presence of correlated expectations ("bubbles") > This results in millions, if not billions, of interconnected price-discovery feedback loops. Well, there are negative and positive feedback loops, only one of which is stabilizing! > beneficial […] because the price discovery feedback loops get faster This can also backfire. In fact, this is why a number of stock markets have instituted a trading stop if an asset moves "too fast". Slowing things down / reducing liquidity can stabilize a situation. Actually, this reminds me of [1] W. A. Brock, C. H. Hommes, and F. O. Wagener. More hedging instruments may destabilize markets. Journal of Economic Dynamics and Control, 33:1912–1928, 2009. where you have a similar counterintuitive argument. The history of the idea of market efficiency is long and the idea remains controversial or contested. See e.g. Philip Mirowski's writings.

1e-9 7y ago
You misread my post. I never said the markets were efficient. On the contrary, they are generally far from it. What I said was that the markets combine the efforts of many different entities in order to determine prices in a way that is superior to what any one entity could do. > Well, there are negative and positive feedback loops, only one of which is stabilizing! Absolutely. Entities that consistently contribute positive feedback cause harm to markets and they are generally doing something that is either prohibited or foolish. I don't consider either a good long term profit strategy. The market regulation departments work to remove one and large losses tend to remove the other. > This can also backfire. In fact, this is why a number of stock markets have instituted a trading stop if an asset moves "too fast". Slowing things down / reducing liquidity can stabilize a situation. Sure, exchanges use a variety of market integrity controls, including limits on rapid and/or large price changes that can trigger order rejections or trading halts. These controls can be beneficial when the price fluctuation was due to poor trading, but can be damaging to a market when the fluctuation was due to significant new information or because there is a natural high volatility situation such as a derivative that is about to expire or is rarely traded. Consequently, the exchanges have to be careful about how and when halts are invoked. Some exchanges often get it wrong. The main point I was making is that lowering the latency of the multitude of price discovery feedback loops making up the global market can be very beneficial because it allows the pricing dependencies to be more fully determined.

posterboy 7y ago

This is an accurate description of the game theoretic argument. It is not very precise about the premisses, "the market", "the reward" or "the critical products"--variables in a non-linear equation, so to speak, that do not necessarily have a unique solution, or no solution.