13 ms·

Ask HN: Automate Stock Trading?

• SandB0x 12y ago
  There is a nice review of related methods in the book "A Random Walk Down Wall Street"
  • nikhilsaraf9 12y ago
    Seems highly rated and a classic. Will get myself a copy.
• bussiere 12y ago
  I'am working on this. Only using python , ipython rethinkdb and postgresql. We are working on a event driven fund to find pattern, but we still need human at the end (the program found pattern and we try to find why and if they are reproductible). Good luck :)
  • nikhilsaraf9 12y ago
    That's very interesting. I'm trying to integrate the concept of "special events" like earnings or dividends or announcements into the mix. Although that isn't the driving force for what I'm trying to do. I'd be interested to learn how it works out for you guys.
    • bussiere 12y ago
      we are event driven here, our program help us to find regular pattern drive by event as annual press conference (we buy before and sell just before) , fda approval for biotech, black friday for amazon , or cultural event (chinese new year). You have annoucement, but don't forget the regular one. We are still working on this (we are around 2% per month) but i'am open to share. Visibly i'am not the only one to do this : http://www.reddit.com/r/stocks/comments/2v7fvk/anyone_make_living_daytrading_stock_market/cof4ulq http://www.reddit.com/r/stocks/comments/2v7fvk/anyone_make_l... Mail me if you want to discuss or share bussiere AT gmail.com regards
• DanBC 12y ago
  "A mathematician plays the stockmarket" is an interesting read of a smart person who made a bunch of mistakes and lost money. http://www.amazon.com/gp/aw/d/0465054811/ref=mp_s_a_1_sc_1?qid=1424599137&sr=8-1-spell&pi=AC_SX110_SY165_QL70&keywords=mathematician+plays+stockmarket http://www.amazon.com/gp/aw/d/0465054811/ref=mp_s_a_1_sc_1?q...
  • nikhilsaraf9 12y ago
    I haven't read the book. From the description on Amazon it seems the investor/mathematician let his emotions come in the way of his investment decisions which led to him losing money. I think that's a risk we run by providing that human touch, which sometimes may even help us if the software is making stupid decisions. My goal is to build a fully-automated system (no human interference) that makes "confident" decisions based on statistics and hard numbers.
    • DanBC 12y ago
      Your emotional response is telling you that your algorithms won't be affected by emotion. Your emotional response has already caused you to prefer one course of action (algorithms) over another (humans picking stocks). Good luck though! It sounds interesting!
    • yummyfajitas 12y ago
      Your automated system will require human input. I lost a bunch of money in Jan - I saw a huge drawdown, discovered a fixable flaw in my strategy, panicked and closed my positions. Human inputtime - do I close positions and shut down? Had I stuck with the flawed strategy, I'd have made (a small amount of) money. Dont think that you dont need to manage your psychology simply because you are running an algorithm.
      • nikhilsaraf9 12y ago
        Yummyfajitas, that's exactly why I do not want human input. Also, I'm not trying to make money on every bet, but to make more money in my winning bets (less fees) than I lose in my losing bets.
• Red_Tarsius 12y ago
  Great work! I haven't done such things, but I'm curious: what books/papers is your software based on?
  • nikhilsaraf9 12y ago
    I haven't based this off of any existing papers yet, I'm intending for it to start off as fresh research, and depending on how it goes, maybe include/test theories from existing papers. Do you have any papers that you would recommend?
• jacquesm 12y ago
  If I could just give you one piece of advice it would be this: don't do this. A bit longer: You're a pig headed for the slaughterhouse. Please do it 'for play money' only for the first 5 years or so, if your method makes money over that period then set yourself a goal and if you reach it stop.
  • nikhilsaraf9 12y ago
    If you could please shed some light as to why you hold that opinion (past experience etc.), it would be of great help!
    • marvin 12y ago
      Beating a stock market index is a zero-sum game, and the large players in this market have hundreds of millions of dollars and huge manpower invested in making money of small players like OP. I suppose it's possible in principle to be smarter than them, but that's the bar.
      • repsilat 12y ago
        It's zero-sum over the short term, but not over the long term -- "beating the index" is (kinda sorta) another way of saying "dragging the index towards efficiency." It means investing in stocks that are going to perform, which (kinda sorta) like investing in companies that will be successful, which increases the size of the pie. Of course, with the amount of money the OP is likely to be investing, it's completely sensible to ignore market impact.
    • PakG1 12y ago
      I'd like to caution that judging from the comments I see from you so far in this, it appears that you have not much knowledge in the area. As such, if you're doing this for fun and learning, great. I'm sure you'll learn a lot. But if you're doing this for money, know that you are playing a difficult game against a lot of very good players. I once attended a predator-prey algorithm seminar on stock trading at the International Conference on Machine Learning. I came away realizing that a ton of people are in this area, and not just the big banks. If you're doing this, go in realizing that you're trying to ride a bike for the first time without training wheels in a race against experienced motorcyclists. These will be very good for you: https://news.ycombinator.com/item?id=8994701 https://news.ycombinator.com/item?id=8994701 https://news.ycombinator.com/item?id=4360742 https://news.ycombinator.com/item?id=4360742 edit: spelling
• njloof 12y ago
  Look for patterns of insider trading and hop on. Oh, and trade offline portfolios for much longer than you think necessary.
• waps 12y ago
  You may want to take a look at http://quantopian.com/ http://quantopian.com/, who make an engine that does just that. A lot of brokers actually support this. The reason appears to be that there are lots of stock trading packages that people want to use (and to be fair, most brokers' interfaces suck somewhat. Also one interface for 2-3 brokers sounds appealing to me). Here is an article about one such trading API: http://www.quantstart.com/articles/Using-Python-IBPy-and-the-Interactive-Brokers-API-to-Automate-Trades http://www.quantstart.com/articles/Using-Python-IBPy-and-the... Most API's also let you play with the "fake money" accounts of the brokers. But keep in mind that they usually cheat on transaction costs in those modes. You may want to read a lot of the algorithms on Quantopian. To be frank, the only one that really makes sense to me is the "auto-rebalancing" approach. What this does it it keeps your investment at what you set it. Say you invest 50% in automotive stocks and 50% in nasdaq stocks. Nasdaq goes up 10% -> it sells 5% of the profit and invests it into automotrive stocks. This is one of the few algorithms that seems to : 1) beat the market 2) actually spreads risk around (or it seems to me it does, depends on your investment spread I guess)
  • nikhilsaraf9 12y ago
    I had actually looked into quantopian when I was getting started, but decided not to use that for my primary source of backtesting because they place limitations on users since that reduces pressure on their systems, allowing them to scale. This restricts backtesting ability and was a dealbreaker for me. The publicly shared algorithms, on the other hand, seem like they can provide a lot of insight, and I will look into those. Thanks!
• balazsdavid987 12y ago
  Just a word of caution: the profitability of stock trading, regardless of the timeframe (high-frequency, daytrade, week long positions), heavily depends on access to information. So even if you build a system that mathematically is better than those built by a team of physicists at large financial institutions, it may easily under-perform the market due to its limited access to data.
  • nikhilsaraf9 12y ago
    Yes. That is a massive handicap that the system will have (it's not going to be co-located and probably won't have access to early information, etc). I'm wondering whether it's possible to develop a system that accommodates these deficiencies while still ending up on top.
• sylvinus 12y ago
  This old HN thread may be relevant: https://news.ycombinator.com/item?id=4748624 https://news.ycombinator.com/item?id=4748624
  • nikhilsaraf9 12y ago
    Thanks for sharing, seems very close to what I'm trying to do. Will give that a thorough read!
• brc 12y ago
  I once set down this path but stopped. Your big problem is that the future market doesn't look like the past market. There is a statistical error inherent in excessively curve fitting past data. You could develop a system which says you'll return 25% per year and then lose 100% in he he first six months. So don't fall into that trap of excessively back testing. What you should be spending time is your money allocation algorithms - the very same set of trades can return high amounts or wipe you out depending on what % of your trading equity you allocate. This is something you can test - by simulating actual trading returns - both upside and downside. I'm not sure if you're looking for trading advice or development advice, but I woudo recommend these books while you are doing system development: - http://www.amazon.com/Trade-Your-Way-Financial-Freedom/dp/0070647623/ref=asap_bc?ie=UTF8 http://www.amazon.com/Trade-Your-Way-Financial-Freedom/dp/00... - trade your way to financial freedom by Van Tharp - the entire 'market wizards' series (http://www.amazon.com/Market-Wizards-Jack-D-Schwager/dp/0887306101 http://www.amazon.com/Market-Wizards-Jack-D-Schwager/dp/0887...) - there are three (or four?) books in the series, all are worth reading. The final problem is designing a system which you truly understand, which you are sure gives you an edge. And which you are willing to stick with if there are periods of high drawdowns. Also remember you are up against people with very deep pockets and huge computers and phd guys working on there same thing. It's a great intellectual puzzle and a worthy challenge, but make sure you know what you are getting into.
  • nikhilsaraf9 12y ago
    Thanks for the advice. I will definitely err on the side of caution and not get caught up with the backtesting and over fitting. Allocating resources (funds) will probably play an important role to manage risk in the system. At the end of the day, I'm trying to answer the question: can you make money in the stock market with a low-touch (eventually no-touch) approach with the help of advanced software and analytics. Hopefully the answer is yes ;-)
    • lmgtfy 12y ago
      >> At the end of the day, I'm trying to answer the question: can you make money in the stock market with a low-touch (eventually no-touch) approach with the help of advanced software and analytics The answer to this has been a resounding "hell yes" for decades across IB/HFT/AlgoTrading shops. The only thing this exercise achieves is whether "you" personally can make money with a low-touch approach using software.
• deleted 12y ago
  [deleted]
• rl3 12y ago
  I wrote a somewhat long-winded comment on this topic a while back, you may find it useful: https://news.ycombinator.com/item?id=6832589 https://news.ycombinator.com/item?id=6832589 The tl;dr version of it is: Risk management, specifically position sizing, is commonly neglected and yet extremely important. Psychology is everything, regardless of whatever your approach may be.
• exelius 12y ago
  This is a bad idea. The "simplistic human investing" you're talking about probably accounts for 0.01% of trades. The vast majority of trading these days is algorithmic, and the HFT guys will eat you for lunch. What you won't see in back testing is how the market reacts to your strategy. The market isn't something that stays static: your activity changes things in subtle ways, and I guarantee there are people smarter than you who have designed algorithms to detect and exploit trading strategies such as yours. Algo trading is very dangerous if you're not coming from an investment finance background. Just be very careful.
  • lmgtfy 12y ago
    This. From OP's blurb it's pretty obvious he's very new to this and most likely hasn't traded much stocks even using the "simplistic human investing" approach. If you're planning on using this exercise to help "mature" your understanding of how you should start tracking/investing stocks then this could be a rewarding effort. If, however, you're planning on jumping straight in to build this to "actually" trade stocks I'm afraid you'll be in a world of hurt before you even know it. If you're serious about the journey, then the choice of programming language or DB is a long long way down the road; start from the basics and work your way up from there.
  • riveralabs 12y ago
    I agree with the fact that it's very unlikely that I can develop a mechanical system that would beat algorithmic trading. The good news is that I don't have to. That's why I'm a discretionary swing trader. In my personal case, trading is about learning how to fish. Learning a non computer skill that I can use for the rest of my life. Even if it sounds cruel. With enough time and hard work, you can train yourself to detect the pigs getting slaughtered by the professional traders and take a bite of the action. That's what swing trading is all about. That being said, it's far from easy. It requires a lot of hard work and determination. Reading books is a good start. The book "Trading Your Way to Financial Freedom" by Dr. Van K. Tharp is required for any modern day trader. The most important lesson from the book is probably that you have to always go into a trade knowing how much you are willing to loose on that trade. Using a stop and respecting it should be deeply engraved into your DNA. But reading books is not enough. Getting a proper trading education is important. The average trading course will cost you anywhere from $2,000 to $5,000. With so many get-rich-quick schemes on the Internet is hard to find a serious an respected trading education program. They are out there. A good place to start would be Trader Planet. They have a list of the Top Trading Schools. Why pay so much for a trading course? Because in the long term it will save you time and money. In the book "How I made $2,000,000 in the Stock Market" by Nicolas Darvas. The author spent many years of failure paying tuition to the markets until he found his way. Many people read this book and get fascinated by the big number. The fascinating part about the book is that in the 1940s and 1950s this guy created his own form of pseudo technical analysis. That system is what made him the $2,000,000. He spent a lot of time an money developing the system and eventually worked out for him. Keep in mind that Moving Averages started to get used in trading in the 1960s (If I'm not mistaken). The moral of the story is that spending $2,000 or $3,000 in advance for a proper trading education will save you a lot of time and money. People with little or no trading experience should not be developing mechanical systems and use them to risk real money. Wanna do it for educational purposes, sure go ahead, but don't risk any money. That shouldn't stop you from diving into the markets though. Once the market bug bites you there's no turning back. You just have to do it the old fashion way. In my case, by reading charts. As a software developer I think technical analysis is the only way to study the markets because that's how my brain is wired.
• Spitfire777 12y ago
  I used to do it with bitcoin, but not to get rich, just for curiosity. The most algorithms is just chart reading, playing around with periods, standard deviations etc. You can achieve something really quickly, but it only works if the market jumps up and down within a short time period and stays on the same level if watching a bigger time frame. What if the market drops by 90% suddenly because some bank decided to do something stupid? The next time I won't work on HFT, but more of risk reduction and long-term investments: - Markowitz diversification model - Multi-Market trading, trade on market differences Another interesting approach would be to exploit trading algorithms, since 99.9% of all trades are algo trades. What you theoretically could do is use popular trading algorithms, develop a tool to monitor their decisions, adjust decisions weights and see how a simulated market performs on comparion to the real market. Now that you have something like a "slave" market and can go on to the next step and calculate scenarios e.g. 5min ahead and can 99.9% predict where the market will be in 5min.
  • kasey_junk 12y ago
    > Now that you have something like a "slave" market and can go on to the next step and calculate scenarios e.g. 5min ahead and can 99.9% predict where the market will be in 5min. What most algorithmic trading models that are looking to prey on other algos do is either wait until their model says the market is already in some bad decision state or move the market in some way to make the bad decision state more likely.
• gaius 12y ago
  gain an advantage over simplistic human investing What makes you think there are any simplistic humans in the market for you to take advantage of?
  • raverbashing 12y ago
    Oh there are a lot. They're the first to be taken advantage of
    • gaius 12y ago
      ... As the OP is about to discover, the hard way.
• smoyer 12y ago
  I started doing this after reading one of the Motley Fools' investment books (Rule Breakers - Rule Makers) ... fortunately I never asked it to trade real money. A few years later and MF has completely disavowed the techniques in that book (and I would have lost whatever I put it). Now I just throw darts ... unless you can get some insider knowledge like the big boys, you'll never be the early mover.
• raverbashing 12y ago
  The only thing you're going to automate is the emptying of your account. > I'm planning to host this on AWS and use a combination of Python and R. Yeah, great idea. NOT. Unless you're doing something like tens of trades per day, or less.
  • nikhilsaraf9 12y ago
    @raverbashing, One trade a day, maybe even less.
• sek 12y ago
  Let's assume your strategies work, then you will earn by a magnitude more by doing this for a trading company than you will ever with your own funds. You can go down that road, but this a whole full time career and it usually takes years to be good enough at it. I could recommend this book (http://www.amazon.com/Building-Algorithmic-Trading-Systems-Website/dp/1118778987 http://www.amazon.com/Building-Algorithmic-Trading-Systems-W...) to get an understanding for the craft.
• koalala 12y ago
  Don't get discouraged by the negative comments. It's completely possible to make a profit (with low risk) without resorting to HFT. I'd recommend you read Quantitative Trading by EP Chan for a great overview of the basic methods and some example implementations. Disclaimer: I've done exactly this successfully for >2yrs now.
  • kasey_junk 12y ago
    Congratulations on your success. What benchmark were you comparing your systems to? How long will you let your system go underperforming your benchmark?
  • jacquesm 12y ago
    > Don't get discouraged by the negative comments. No, of course not. Jump in feet first with all that you've got and to hell with common sense. > It's completely possible to make a profit (with low risk) without resorting to HFT. Yes it is. For a while at least. > I'd recommend you read Quantitative Trading by EP Chan for a great overview of the basic methods and some example implementations. Reading is always good, second that. > Disclaimer: I've done exactly this successfully for >2yrs now. I know gamblers that have been on winning streaks too. Unfortunately that's not how the story ends, especially not for novice gamblers. It's nice you have found a way to consistently beat the market and I really hope that you're outperforming the lowly index funds but without more data 'successfully' is awfully un-specific. I made some money in the stockmarket too, but I'm not going to dress is up as intelligence, I'd rather admit that it was dumb luck.
• anovikov 12y ago
  Someone can please give me some trustworthy, scientific reading material to give people asking me to proof the point 'systematically beating index performance on the stock market, using only public available data, is not possible'? Intuitively i understand this is true, so no need for holywar. Same also applies to commodities trading.
  • kasey_junk 12y ago
    There is no proof of that. Its a restatement of the efficient market hypothesis, which is a long studied and very contentious subject.
  • brc 12y ago
    You don't need scientific proof or reading material - the theory you are trying to prove is wrong. Just look at the number of billionaires who have decades long 20,30,50% average returns - not possible by chance. The efficient market hypothesis believers say that these people were lucky. People who say that don't understand probability. The probability of winning big one year is probably not that rare - lotto win sort of numbers, and given the number of players and the number of iterations - easily explained away. However, the existence of scores of individuals who have been able to profit for long periods of time completely destroys the efficient market hypothesis, because the hypothesis says they can't exist. Take one single example ; Richard Dennis. He started off with $1500 and ran it up into hundreds of millions. He then taught a group of people who had zero trading experience his system and then they went off and made hundreds of millions (some are still going). http://en.wikipedia.org/wiki/Richard_Dennis http://en.wikipedia.org/wiki/Richard_Dennis That is completely non-random outcomes. The efficient market hypothesis is something that academics wet themselves over while other people laugh at their theories and make millions. Essentially, for 'systematically beating index performance on the market using publicly available data is not possible' to be true, then the scores of people who have done just that, over decades, cannot exist. And yet they exist.
    • kasey_junk 12y ago
      Richard Dennis is a weird example to use, given that he ended his career losing tons of other peoples money. If anything he is a perfect example of luck...
• kasey_junk 12y ago
  You have not given nearly enough information to give any reliable tips. Start with a few questions. How often are you targeting trades? Do you anticipate trading thousands of times per day, a few times per day, a few times per week, a few times per month or a few times per year? Know that there are sophisticated algorithmic trading systems in all of those spaces. Once you know how often you will be trading, you can make more appropriate decisions about execution platforms. What kind of data feeds do you want to use? How will you get the data into your systems, how will you get the outputs out of your system. How will you translate those outputs into orders? Who will take your orders? As an aside, most brokerages will want to know if a computer is making all the trades and will either outright ban fully automated trading or will want a kill switch (see the TSA for your broker of choice). This leads us to the actual hardest part about automatic trading, operational risk. What are you going to do to prevent your system from making out trades? What is your shut down procedure? How is it triggered? How will you monitor the system? Know that managing automation risk is the last mile of human intervention in the system, and that lots of people have lost a lot of money by not intervening soon enough, or intervening too soon. You also need to have a target benchmark to compare to. What are you going to benchmark against? How often are you going to evaluate against that benchmark? How long will you allow your system to underperform the benchmark? There are long game algorithmic trading systems that have built into their capital allocation models >10 year underperformance against benchmarks for instance. Finally, ask yourself, what inefficiency are you taking advantage of, that will allow you to profit from trading? What is it that you bring to the table that isn't already there? Once you honestly assess that, ask why wouldn't that same efficiency be more valuable in some other field that is less crowded because automatic stock trading is a very low percentage game right now.
• toddkazakov 12y ago
  My 2 cents - don't reinvent the wheel. Use quantopian.com - it has great community, where you can learn a lot. Their trading system called ZipLine[1] is open sourced, so I guess you can extend it to fit your specific needs. [1] https://github.com/quantopian/zipline https://github.com/quantopian/zipline
• kewin87 12y ago
  You should have a look at MetaTrader. It provide tools for back-testing and dynamic spread etc.. I have been building also my own platform (with nodejs as for high IO), but you easily loose focus when you have to build it ALL by you're self.. (opensource) platforms like MT4 / MT5 give you more time to spend on actual alghorithmes, then on data structures etc
• ndcolorado 12y ago
  Check out omnitrader.Com as a potential platform for building your algorithms. Whilst there also check their product myomnivest. If you want to go big-time as I have, check into their Nirvana club. Not cheap but their auto trading algorithms are extremely sophisticated.
• ndcolorado 12y ago
  Check out omnitrader.Com as a potential platform for building your algorithms. Whilst there also check their product myomnivest. If you want to go big-time as I have, check into their Nirvana club. Not cheap but their auto trading algorithms are extremely sophisticated.
• SEJeff 12y ago
  I've worked in the electronic trading aka hft market now almost 7 years. The big boys are optimizing their trades down to the nanoseconds, they are spending big money on bleeding edge network technologies, colocating their servers inside exchanges, etc. If you expect to compete with them on AWS save yourself the loss and just stop right there. However, if your strategy is more or less buy and hold for weeks / months then this doesn't sound entirely unreasonable. From my experience, simply getting into this industry as a serious player is roughly a 10-12 million investment, and that is starting very very small in only a few exchanges.
• bussiere 12y ago
  One of my friend is working on a same project for a big company and some of my friends are interested in this. I'am thinking does anyone will be interested by a specialised forum or a subreddit to share our experience and tips ?
• briHass 12y ago
  Even though backtesting is only of limited use, you should try your algorithms on fresh (you haven't previously used them) time periods and see what types of returns you can get. When I did this with my algos I was playing with, even though it seemed positive, when I accounted for the cost of the trades and the cost of the data (near-real time market data is damn expensive), I wouldn't come out much ahead of a simple buy-hold index fund approach. Invest the minimum amount you can in this project to learn that it isn't a great idea.
• jshaqaw 12y ago
  Use paper trading for a while on a platform like Interactive Brokers before going live. You will find it's unlikely you discovered the holy grail but hey maybe you have. Just be wary of a common trap for both beginners (excusable) and longtime market pros managing billions (not so excusable) - lots of strategies look great while testing because they have a payoff matrix which yields +1$ 95% of the time and -$1000 5% of the time. Obviously this is a poor ex ante bet but you are unlikely to empirically observe/discover the asymmetry at first when it just seems like you are printing free money.
• billconan 12y ago
  I'm curious how you will link your program with your broker account? do they provide rest api for stock trading?
• nikhilsaraf9 12y ago
  I want to clarify, the intention of this system is specifically to NOT be a HFT system because I'm aware of the large handicap associated with inferior data available at slower speeds. It's intended to primarily help with stock picks since it can look through more data than a human can. This would involve both fundamentals and technical analysis. The most frequent trades would be once a day and may end up being closer to once a month. It is intended to be a low-cost solution. I would like the trades to be done by the system but the approach will not require it. Based on some of the comments it seems such a system will be the underdog and will be the prey for the large HFT shops. And that's one of the risks of such an experiment. Once I put in real money into this (not too much, but just enough to not be consumed by the fee) there will be a hard point after which I'm ready to throw in the towel. I think of this more like an investment management software that has more knowledge than myself (various trading techniques, tax laws, etc.).
• bussiere 12y ago
  I'am thinking btw to make a portal / forum / subreddit about this topic. Anyone interested ? What format will you prefer ? Subreddit / special forum ?
• curran736 12y ago
  Would you like to team up and build something? I have some ideas that can help build better algorithms. curran736 at gmail.com
• deleted 12y ago
  [deleted]