8 ms·
"Computers and algorithms" is just a synonym for people; people implement ideas in code, design the algorithms, pick the weights for training machine learning,
by onion2k 7y ago
"Computers and algorithms" is just a synonym for people; people implement ideas in code, design the algorithms, pick the weights for training machine learning, etc. No matter how many levels of abstraction you go through it's always people at the top. If anything happens in the society there's always a person who made the decision to make it happen somewhere. Deferring that responsibility to "computers did it" is dangerous bullshit.
- swsieber 7y agoHmmm... I both agree and disagree. While humans program the general strategy, and even the bugs they didn't think they were adding, there is a substantial difference: computers can and will act in such a completely stupid way as to be obviously stupid to even the most uninformed human. What I fear the most is a silly bug causing a crash in a way no manually triggered series of trades would.
- dsfyu404ed 7y ago>What I fear the most is a silly bug causing a crash in a way no manually triggered series of trades would. Banks are run by smart people and they worry about the same things. There's dedicated systems checking to ensure algorithms aren't running away. All the big banks and brokers have been buying them for a decade or so. Nobody wants a crash and the algorithmic traders are mostly using a line of credit from the bank with which to do their trading and the bank usually take a commission on the profit so the banks have a strong interest in seeing them make money as opposed to crash the market and disappear into bankruptcy proceedings. Of course the government also swooped in and saved the day by mandating risk checking systems a couple years after everyone who cared was already using them (thereby solidifying the status quo for better or worse).
- benj111 7y agoCould you not say the same kind of thing about mortgage backed securities, Fannie Mae etc? We've had flash crashes, which aren't in peoples interests, they still happened. I don't think its reasonable to hand wave it away, because "smart people". Programmers are supposedly smart people,I've yet to come across a perfect bug free program.
- dsfyu404ed 7y ago>Could you not say the same kind of thing about mortgage backed securities, Fannie Mae etc? Human based checks are easier to subvert than automated ones. >We've had flash crashes Which is exactly the problem that most banks and markets try to prevent. Any system that uses what other people are doing as a strong enough input is going to be susceptible to this to some degree. If you try to buy too many shares of something or for too disparate of a price compared to the market the bank will almost certainly kill your order. The exchange might kill it too. Depending on your contract with the bank and exchange they may kick your session for the day if you do it too much (once may be too much). The limits most banks place on what you can do mean that runaway algorithms can really only be a slow tug in one direction or the other which is no big deal.
- acjohnson55 7y agoAt the end of the day, both systems are subject to issues. Computers are far more disciplined than humans, but that comes with a tradeoff of lack of introspection.
- benj111 7y ago"Human based checks are easier to subvert than automated ones." I disagree. They're easily subvertable, but in different ways. "Which is exactly the problem that most banks and markets try to prevent" But they haven't succeeded so far. Further, as the saying goes, history never repeats itself, but it rhymes. It's all well and good doing all these automated checks based on past issues, that's not going to help with the next issue that isn't enough like the one before.
- selestify 7y agoWhat are the different ways in easily subverting people versus machines?
- mattkrause 7y agoA machine will hold you, strictly, to the letter of the law. A person might consider giving you a bit of wiggle room on the exact letter of the law, but can also ensure you comply with the spirit of the rules.
- swsieber 7y agoThey worry with good cause. We worry about bugs where I work, and they still make it through somehow. Worrying about them doesn't prevent them. See also, Knight Capital: https://dealbook.nytimes.com/2012/08/02/knight-capital-says-trading-mishap-cost-it-440-million/ https://dealbook.nytimes.com/2012/08/02/knight-capital-says-... , where a bug cost it $440 million. It's already been seen on a small scale.
- psychoslave 7y agoI agree with that it's always people at the top, however it doesn't imply that there's always one person who made the decision. Collective decisions exist too. And even looking at individual decision, not all decisions are taken through careful conscious means. Plus, deciding to delegate decisions (to algorithms or anything else) is not the same as deciding through your very own direct perspective.
- mumblemumble 7y agoThat was as true in the days of when pit trading dominated as it is nowadays. It's true that automated trading is fast, and that means that things can go off the rails fast. And, it's true, that has led to some spectacular news-making events. But, for that very reason and others, most trading firms keep a very close eye on what their auto-traders are doing. Closer than they ever kept on their pit traders. I briefly worked at a firm that did both HFT and pit trading, supporting one of the teams in charge of monitoring both groups. And what I saw was that the company was generally much more in control of the algorithmic trading. It was more heavily instrumented than a human could ever be, with humans (who were themselves licensed traders) watching the computers very closely at all times, as many layers of automated fail safes that would kick in if things went a little screwy. By comparison, the pit trading was kind of a mess, and the team was frequently having to scramble to amend all sorts of human errors largely stemming from things like miscommunication and straight-up bad handwriting. Beyond that, I'm sure that, compared to the folks operating the auto-trading systems, the pit traders never had as firm of a real-time grasp of the finer subtleties of their positions, simply because nobody's able to do math in their heads like that. Which, I think, is why spectacular events in the markets stemming from speculative trading going off the rails aren't really any more common nowadays than they were in the days before high frequency trading. Yeah, computers created some new ways to screw things up, but they also mitigated some existing ones. The net effect is not nearly as bad as journalists make it out to be. But, of course, playing to people's fear of the unknown has always been the more lucrative route for journalists to take.
- hef19898 7y agoThat hints at something. Now that you summarize it, it is very similar to my experience in a completely different field. The higher the rate of automation the higher the level of control. If things are done mostly manually it tends to be an uncontrolled mess with inefficient processes in place that create the illusion of control.
- 1980phipsi 7y agoAlso, many of these "computers and algorithms" have a role for the humans when abnormal things happen, which they do in markets, quite often.
- TicklishTiger 7y agoIf anything happens in the society there's always a person who made the decision Imagine a green field of gras. People start walking on it. In all kinds of directions. Slowly paths through the grass start to emerge. Those paths attract more people to walk on those. The stronger the path through the grass, the more people use it. Finally, there is one clearly visible path through the grass everyone uses. Who made the decision?
- cco 7y agoProbably the deer that were there already?
- maxZZzzz 7y agoPeople decided that walking on paths is better then not walking on them.
- raducu 7y agoYes, but you could create the path just as you plant the grass and amuse yourself to no end on how sheep-like people really are. Just because there's a path there, it doesn't mean its optimal or good for the public at large.
- onion2k 7y agoWho made the decision? The first person who walked the final path. Everyone who agreed with that person and walked the same path compounded his decision. Everyone who crossed the field should have thought about what they were doing at the time. Far too many people, and especially developers, are way more focused on absolving themselves from responsibility for anything bad than they are willing to put the thought in at the beginning, and that's the root cause of a lot of problems.
- mattmanser 7y agoThat's a stretch. I know you're enamoured by your synopsis, but realistically emergent systems aren't designed or thought about. They might be tweaked.
- std_throwaway 7y ago"Responsibility" is an abstraction needed for enforcement of the law. It doesn't say much about how things actually are.
- blazespin 7y agoTrue to a point. However, that person doesn't necessarily know why a computer made a decision. The high dimensionality of these questions and decisions can be solved by programming, but the question and solution aren't completely understood. Thus, we enter the age of AI.
- onion2k 7y agoThat doesn't change the fact that they're responsible for the outcome. A reasonable metaphor is someone firing a gun in to the air: they can't claim not to be responsible for where the bullet lands despite having no real control over where it's coming down. If you don't want that responsibility don't pull the trigger.
- blazespin 7y agoYeah, a lot of people warn about the dangers of AI. And it's real - handing over decision making we don't understand to computers comes with real risks.
- nicoburns 7y agoRight, but that doesn't change the fact the stock market has changed from people taking careful aim before pulling the trigger, to a model where people are mostly looking where everybody else is firing, then firing in the same direction.
- XuMiao 7y agoExactly, the high dimensionality is very difficult to grasp for regular people. The nonlinearity of AI models makes the situation even worse.
- mempko 7y agoYou clearly haven't seen my, buy/sell at random algorithm!
- C1sc0cat 7y agoAnd there are algorithms that where/are used by Humans to manage investments
- armitron 7y agoBee colony is just a synonym for bees; bees run the colony, bees build the hive, bees are the hive. No matter how many levels of abstraction you go through it's _always_ bees at the top. If anything happens in the hive, there's _always_ a bee that kickstarted it. A collection of individuals and/or systems giving raise to a metasystem, a superorganism, does not and can not exercise deliberate control over it: the metasystem lies on a higher level. Not to mention as this thread - in its totality - surely proves, that individuals in many cases are completely unaware of the layer of control above them. What does an individual bee know about the colony? What is "dangerous bullshit" is wishful thinking. Nation states have been with us for milennia. Corporations for centuries. Have we learned nothing that we still entertain notions of (human) global control?
- bearer_token 7y agoFrom https://slatestarcodex.com/2014/07/30/meditations-on-moloch/ https://slatestarcodex.com/2014/07/30/meditations-on-moloch/ > if everyone hates the current system, who perpetuates it? And Ginsberg answers: “Moloch”. It’s powerful not because it’s correct – nobody literally thinks an ancient Carthaginian demon causes everything – but because thinking of the system as an agent throws into relief the degree to which the system isn’t an agent.
- danmg 7y agoIt's automation taking away the lower level trader and analyst positions. These were jobs that were very lucrative. This creates an even more rarefied, winner-take-all atmosphere, and makes an industry that was already considered amoral downright sociopathic.
- 1e-9 7y agoFor every automated trading system that is operating on major exchanges today, there are people responsible for its actions. If a system disrupts a market significantly, regulatory agencies work to be sure the responsible people are fined, banned from trading, or even imprisoned. There is no deferment of responsibility to "computers did it". Edit: Just have a look at the disiplinary notices of some of the exchanges to get a feel for this. For example, here are notices for Nasdaq: https://www.nasdaqtrader.com/Trader.aspx?id=NDisciplinaryActions https://www.nasdaqtrader.com/Trader.aspx?id=NDisciplinaryAct...
- asdfman123 7y agoYes, but when computers do things that humans do there is simply less human oversight. If a human is taking action to enforce a policy that will harm other people, there's the chance that he'll resist it or raise awareness. If an algorithm does it, there's no final line of defense. And software is easy to scapegoat because it's murky. You can claim "we didn't know the algorithm would do that," but it's harder to scapegoat your employees when there's emails on record showing you specifically told them to enact a social harm.
- momokoko 7y agoThat is exactly what happens. It's direct human nature to defer that responsibility to "computers did it", and to believe they will not in the future is naive. The second part of that is that the human at the top can only make decisions on what is visible to them. With fairly opaque ML systems the only thing easily visible is the results. Seeing how it came to that result is much more challenging. So we will have humans at the top that can only makes decisions about the results and not the means. Which is the part many, including myself, find troubling.
- karmakaze 7y ago> "Computers and algorithms" is just a synonym for people I don't think we can say this anymore. I wouldn't say that the makers of AlphaGo were the decision makers for any of AlphaGo's moves. You could perhaps say the players of the games AlphaGo trained on were. When it comes to AlphaGo Zero you can't even say that. At most you could say that the people were policy makers or tweakers.
- qvrjuec 7y agoYes, but AlphaGo can't be used as a metaphor for a lot of the algorithms/automated trading strategies which were implemented based on research done by people, and carried out by computers. AlphaGo only trained against itself.
- sgt101 7y agoAnd Alphago only does training when it's asked to by Deepminds programmers. It didn't spontaneously appear and start playing by itself.
- unforeseen9991 7y agoThe point is that almost all algo's that are actually deployed and making 99%+ of these trades are not AI / ML based. So reference to AlphaGo doesn't make sense here.
- landryraccoon 7y ago> If anything happens in the society there's always a person Who decided to make George W Bush president? I highly doubt you can assign responsibility to either a single individual or even fully enumerated clear subgroup of individuals.
- roywiggins 7y agoThere's those who argue it came down to 9 (or, if you like, 5) people.
- funnybeam 7y agoSurely the people that voted for him are responsible?
- landryraccoon 7y agoThat’s fair, but it’s definitely not a single person and not even a set that can be practically enumerated (since voting is anonymous). Id also say that assigning moral responsibility to a group of people that large is really odd as well.
- drited 7y agoThere's also a person who can/will yank the power cord out of the wall when/if it looks like the algo is going haywire and losing a ton of money, like happened during the quant quake.
- Bubbadoo 7y agoI think you're missing the point. Trading and Sales used to be people intensive, in today's parlance, 'high touch.' You had people talking deals on the phone, you had transactions sent to the exchange where a floor trader did the execution, you had a trader's assistant fill out the trade ticket and submit it manually to the mid-office, where you had a person who recorded the transaction then passed it to the back office, where yet more people processed and settled the trade. All of those people are gone. No need, we have automated much of this deal cycle. Your best example is to take a trip to your local exchange floor, which used to be littered with runners, assistants, floor traders, etc. It's eerily quiet these days, as most trades are executed by computers.
- MadWombat 7y ago> If anything happens in the society there's always a person who made the decision This is not true in case of machine learning. Yes, there was a person or people who designed the model, gathered training data, trained the model and deployed the result. But the decisions the model is making as part of its function are not made by these people. Most of the time, people who trained the model cannot even tell you why the model is making the decisions it is making. Tractability of ML models is currently a very hot topic in the field and progress is being made, but a lot of currently deployed models are inherently intractable.
- mattkrause 7y agoSet your spouse’s alarm clock for 4 am. When it goes off, calm them down by pointing out that while you set it and you turned the volume up to 11, it wasn’t your decision to disrupt their sleep; they should blame the timer inside instead. Let me know how the couch feels afterward. Seriously, the responsibility lies with whomever made—or okayed—-the system. One-off things, especially if they are hard to foresee, might get a partial pass. Otherwise you ought to be carefully testing and evaluating your own system instead of blaming gradient descent.
- MadWombat 7y ago> the responsibility lies with whomever made—or okayed—-the system And generally speaking these are very different people, but that is besides the point. I am still unsure why we are talking about testing and evaluating, as if we are only talking about failure cases. If I set the alarm clock for 7AM and it starts ringing at 4AM, the fault is with the clock designer. But if I set it for 4AM and it starts ringing at 4AM, it is somewhat silly of me to start blaming the clock designer for my disrupted sleep, isn't it? And we are not talking alarm clocks, we are talking about systems that make complex decisions based on hugely multidimensional data. Decisions that we provably cannot explain. There is no amount of testing and evaluation that can ensure that these systems are error free. But when they work, they do work well and provide a competitive edge over human analysts.