15 ms·

Python For Finance: Algorithmic Trading

379 points by pastefka 9y ago

SirLJ 9y ago
It is a great article, but why on earth someone will use a service like Quantopian or similar service? They are your competitor and who will prevent a disgruntle employee or a hacker to steel your successful trading strategy? Just buy some data from eBay, you can get 20 years of historical stock market data for less than $100 and you can test any trading strategy or idea imaginable, including trend following, buy and hold ETFs, etc... The barrier of entry is pretty low and you can develop a great lifestyle business with no customers, employees and investors around that...
- Blackstone4 9y ago
  Have you done this? If so it would be great to get your perspective
  - SirLJ 9y ago
    Yes, I am doing it as a side project and I am looking to retire in a few years and live off that...
    - sjroot 9y ago
      I second the above comment; you should write about the process and how it is working out for you.
    - gricardo99 9y ago
      I'll third this comment. Some more details on what you trade, where you trade, how long you've been doing it, performance metrics, etc... would be fascinating.
      - SirLJ 9y ago
        As you can see I am paranoid, because the more people trading the same strategy, the less effective it becomes, not to mention, if someone knows your trading strategy it can trade against you... it’s a zero sum game after all... All that said, here are the 2 most important advises I wish someone told me when I started years ago: Have an iron clad risk management strategy for your portfolio, e.g. if you are willing to risk 1% of your capital on any given day and tomorrow happens to be the Black Monday and all stock go down 50% you still have to be sure you’ll loose only 1% if you have to sell everything... This is where the back data comes into play – the goal is to back test not how much money you are going to make, but how much money you are going to loose in the worst case scenario if you have to liquidate everything and move to all cash. If you have this working 100%, you just need a trading strategy with a minimal edge and you’ll make money in the long run in the market and the compounding effect will really make you a rich person (rich person to me is someone who makes more money then he spends, WITHOUT working more than 1 hour a day) And the second advice is never ever trust and buy a trading strategy from someone, the fact he is selling or "teaching" a trading strategy means it is not working and he is trying to make some money by scamming people and selling the dream, all those trading educators cannot make money trading and that’s why they are coming with up with those fake trading courses... just check this website to see the magnitude of this scam and how people are loosing their life savings... I just don’t know why FBI, SEC and the government is not doing something to stop those guys... https://www.tradingschools.org/ https://www.tradingschools.org/ Good luck and as I said before, just treat is as a hobby at first and spend the time testing and you’ll find something that is working for you... EDIT: some spelling errors fixed.
        princeb 9y ago
        > more people trading the same strategy, the less effective it becomes you should pick a more scalable strategy, the kind of strategy that becomes more profitable the more you talk your own book.
  - dsacco 9y ago
    This commenter comes into every single thread about trading and talks about buying data from ebay, then consistently demonstrates that he doesn't know the first thing about due diligence on financial data. Each time I try to ask him about his data quality or methodology at even a high level, he responds by accusing me of wanting to steal his work or stop the democratization of data. I'm going to reiterate this right now: you are not going to find data valuable enough for trading insights on ebay. Go look at the prices of historical data for various time resolutions at TickData or CBOE Livevol and then ask yourself if cheap, competitive data on ebay is too good to be true. If the data is competitive, it's expensive. If it's not competitive, you shouldn't be paying for it, especially not from ebay. The first time I read about him doing this I was puzzled, now I'm getting increasingly frustrated because it's blatantly false information.
    - SirLJ 9y ago
      Unless you are day trading you don't need such a granular data format, so please stop saying you need to spend thousands and thousands of dollars to be able to back test a trading strategy
      - dsacco 9y ago
        If your data doesn't need to operate on an intraday scale, you can get perfectly accurate real time or historical data for free just by using a reputable broker like Interactive Brokers. At that point there's no need to buy the data on ebay, so I still don't understand why you ever would.
        SirLJ 9y ago
        Actually I have checked IB API and it is very restrictive, e.g. you cannot request let say all daily data for all symbols for 2007 for example or even for one day for that matter...
      - lz400 9y ago
        As far as I understand historical daily data is easy to get from yahoo, google, etc. Data is only difficult to find/expensive when you need tick data.
- mi100hael 9y ago
  Mostly a matter of trust. I trust Quantopian more than I trust some random asshole on eBay purporting to sell accurate historical data. Plus, Quantopian is convenient.
  - SirLJ 9y ago
    This is the cheapest data I can find and you can do random checks to verify the data... if you know a cheaper or even better a free place please advice...
- kevstev 9y ago
  Because they do so much work upfront for you. I worked in algorithmic trading for a bit over 10 years but left in 2014. I spent a few months in my spare time building out a trading and backtesting platform, pulling data off of yahoo finance, connecting to Interactive Brokers via their C++ API, etc. It was all very slow and tedious, and during my googling one day, I come across quantopian. I felt like an idiot- they already had free access to morningstar with a decent API around it, tons of other datasets you could subscribe to, and a fully fleshed out backtesting platform with risk measures, pretty graphs, etc. It would have taken me months-years to get to that point. Shortly after they also included integrations to IB and Robinhood so you could very easily take your ideas and actually live trade them. I immediately stopped development on my trading platform and just started using Q's. Q could potentially take my algorithm and start using it themselves, though they actually put money into algorithms they think are promising and share the profits with you. That's a small risk IMHO. Hackers and the like... I did this stuff for 10+ years. People WAY over estimate the value of this code- you have an idea, you backtest, hope it works, and then you put it in the wild. I have never seen an algo that is a real guaranteed money maker (aside from arbitrage/hft stuff). What are these hackers going to do when they take a 10% loss one month and don't understand why? Some "signals" can be used in-line fairly generically with other algorithms to enhance their performance, but I will still bet on the guy that knows/understands the algorithms to not lose his shirt, then some hacker dude that likely has no capital to begin with and may not even understand the basics of financial markets. One thing you seem to overlook is that it is one thing to go buy a bunch of historical data off ebay, but you need to constantly update that if you are going to live trade- Q handles that part as well. If I was looking to make a business out of this, I would build my own platform. But using it as a research platform to invest my own money, Quantopian is awesome.
  - SirLJ 9y ago
    My experience is unfortunately different, I found the data for some delisted stocks like HTZ missing which will massively screw the past results due to the survivorship bias...
    - dsacco 9y ago
- dglass 9y ago
  Not everyone can build their own backtesting tool. I played around with quantopian for a little while and did some live trading with quantopian + robinhood and I never would have been able to do that without quantopian
  - SirLJ 9y ago
    I am not a developer, but using my brokers API, examples and python, it took me few months to build a very robust and automatic trading system... I did learn python by back testing my trading ideas... To use Quantopian you still have to learn python, so I am not sure where is the benefit, considering all the problems mentioned here...
    - davewasthere 9y ago
      Or C#
ncyclopediae 9y ago
Here's a dataset on Kaggle that can be used in this process https://www.kaggle.com/biomimic/periodic-table-of-elements-mapped-to-stocks?03 https://www.kaggle.com/biomimic/periodic-table-of-elements-m...
pmart123 9y ago
Personally, I struggle to see the competitive advantage Quantopian brings. They use retail brokerage platforms to facilitate trading, which rules out anything close to HFT. Then, they are tied to any financial data vendor (Morningstar in this case) to not offer too much visibility on the underlying data. As others have mentioned, this makes it tough to validate aspects like adjusted vs. as reported earnings, how delistings are handled, etc. From my experience, getting/making sure the data is accurate is a ton of work, even if it is from good sources and you can see all the actual data. The moment an investor/trader on the platform gets traction is the moment they want Compustat data, exchange data, Bloomberg for fixed income, and will trade through Instinet/Flextrade, etc. The moment the platform is successful is the moment Morningstar could pull the rug out from them. If someone has better understanding or knowledge on Quantopian in particular, I'd be interested to hear why.
- TACIXAT 9y ago
  My understanding is that Quantopian has no interest in being a platform for HFT. It's there to democratize trading strategies and test them in a sound way. They do a lot to have a good backtesting platform and clean the data that is available in the platform. If I wanted to automate a strategy, I'd have to figure out the Robinhood API and basically recreate what they have. Instead, I can use their platform, their backtests, their free data, and build strategies that efficiently lose money. I think the other goal being that if you create a profitable strategy, you can enter it into competitions, trade with other people's money, and make a profit. I think the platform is a value add, I have an account and use the research notebooks. Some day when I more spare time I'll make some trading algorithms.
  - SirLJ 9y ago
    The main problem with Quantopian is that the data missing especially for delisted stocks, here are some examples you can test yourself: AA, CWH, NIHD, PANL, HTZ, DSL - I have a whole list I found very fast, just imagine what other mistakes could be there... And the other problem is that they are my competitor, so I would never give them my trading strategy or ideas I am testing at the moment...
NicoJuicy 9y ago
I have a simple manual algorithm that took me from 650€ to 1450€ now (as test) in 2 months. Does someone here have experience with algo trading in c#?
- Veratyr 9y ago
  I won't say I have "experience" but quantconnect.com is as good a platform as I've found for C#. Their engine is open source so you don't have to rely on them either, though you'll need to provide your own data if you run it yourself.
  - rl3 9y ago
    QuantConnect also offers tick-level data, though not full resolution if memory serves. Still way ahead of Quantopian in any case.
    - jaredbroad 9y ago
      Futures have millisecond timestamps (trades/quotes), equity trade ticks are rounded to the nearest second, cfd/forex are millisecond quote bars. For options we have minute resolution data =)
      - SirLJ 9y ago
        please use a disclaimer if and when you are promoting your own company...
        thedufer 9y ago
        Does the "we" pronoun not make this sufficiently clear?
      - Veratyr 9y ago
        Worth noting though that Options data is super hard to get running on QuantConnect due to the memory limits and lack of index options.
TACIXAT 9y ago
I went to the Quantopian conference for their basic training on algorithmic trading. This blog post was pretty much what they covered, intro to pandas and a simple strategy. There is a lot of educational material on their site too (which is what you ended up getting in the paid training). My biggest thing with the Python for Finance books - I know Python, I want to learn finance. All these books are the inverse of that, for people who know finance and want to learn Python. There is a good site for quantitative economics [1] that has tutorials in Python and Julia. I would love a mathematics of finance book that had the examples in Python. 1. https://lectures.quantecon.org/py/ https://lectures.quantecon.org/py/
- ice109 9y ago
  those are indeed good lectures but they are not math for finance per se - more like dynamic programming for econ/finance. my impression is that finance as such is a hodge-podge of econ/accounting/business.
- Maven911 9y ago
  Did you find the conference worth it ? I was thinking of going to it as well or even going to the online version since it's a lot cheaper but ended up not taking part.
  - TACIXAT 9y ago
    I went to the conference 1 or 2 years ago and it was good. Worth the student pricing. I only did the training this year, also on student pricing, and it was pretty disappointing.
- wtvanhest 9y ago
  The problem with learning finance that it takes a lot og effort. A good place to start would be taking the 3 CFA exams. After about 1000 hours of study, you will have a basic grasp of finance.
  - dsacco 9y ago
    This is utterly the wrong way to learn finance for quantitative trading. That would be an unproductive use of time. Read Options, Futures and Other Derivatives and Algorithmic Trading and DMA and you basically know everything you'll get from public sources that could be meaningful for trading. If he knows how to code and is looking for "mathematics of finance" he should start with those, not the CFA.
nickthemagicman 9y ago
How is algorithmic trading not equivalent to astrology? Nothing can be predicted because there's way too many confounding factors.
- fenwick67 9y ago
  The net gain in stock trading is usually positive in the long run, even if you just buy and sell randomly. Just look at the DJIA.
- PaulRobinson 9y ago
  Algo trading typically utilises technical analysis which is basically patterns proven to repeat in markets for a variety of fundamental reasons, or fundamental analysis (e.g. algorithmically valuing and pricing options based on underlying fundamental data, sentiment analysis, etc.), and buying/shorting as appropriate. It is based on scientific methods: empirical evidence being used to validate hypothesis that produce results that can be shared and repeated by others. Astrology is story-telling with no scientific grounding.
  - gnaritas 9y ago
    > technical analysis which is basically patterns proven to repeat in markets for a variety of fundamental reasons Technical analysis is merely another name for hindsight bias. Those patterns only look like they repeat in hindsight because you're ignoring all the failed patterns that setup right but failed to play out and thus don't look like the pattern in hindsight. Technical analysis is exactly like astrology and is in no way grounded in science. Quantitative analysis is based in the scientific process, technical analysis is superstition created by visual traders who think they see patterns but fail to understand biases. You are confusing the two, technical analysis is a pattern library of nonsense like "head and shoulders" or "dead cat bounce" based purely on visual patterns in price data that only "appear" to repeat due to hindsight bias. It is in no way remotely scientific. I've spend thousands of hours testing trading patterns in Forex, they're all nonsense that lose as much as they win over any decent sample size which equates to random. Edit: to the guy below, You should ask for sources that back up the claims that it does work. You won't find any, it's trader superstition, not science. You don't have to prove things don't work; you have to prove they do. At least if you want to call it based in science. However, the field of quantitative analysis itself is proof technical analysis doesn't work; it's what happened with people who actually knew math got into trading and discovered the technical analysis superstition that visual traders who lack math skills made up.
tejaswiy 9y ago
On a side note, the market continues to do well and I've been noticing this trend of active-trading, real-estate investing gurus crawl out of the wood work selling services. Please do not try to trade actively unless that's your full-time job. Passive investing using index funds is definitely not sexy, but it gets the job done.
- gnaritas 9y ago
  > Passive investing using index funds is definitely not sexy, but it gets the job done. Only if you have a significant amount of capital to play with. People looking into active trading are doing it because they don't have the necessary capital to make passive investing meaningful; they're looking for much much larger returns that you can get from an index fund, which necessarily comes with more risk. Traders are gamblers.
  - slyfocks 9y ago
    This is extremely reckless investment advice. Active trading, by definition, incurs higher transaction costs than passive investing. Since transaction costs are generally priced per trade, active trading is more costly on a percentage basis for individuals with less capital.
    - gnaritas 9y ago
      > This is extremely reckless investment advice. It wasn't investment advice, nor did I say it was a good idea; I'm merely telling you why people choose active investing: they believe they can do better and there's enough stories out there of the "lucky" person who did better to keep that hope alive. In all likelihood they're going to lose their money, but you'll never convince people trying to win a lottery that they're better off accepting they can't. I literally said traders are gamblers, obviously that means I think active trading is gambling, that's not investment advice.
- rootsudo 9y ago
  Don't worry, we have cryptocurrency now. A new, unregulated frontier.
jaredbroad 9y ago
QuantConnect recently announced full python library support; and we have launched https://www.quantconnect.com/tutorials https://www.quantconnect.com/tutorials to help people write quantiative strategies in Python. QuantConnect & LEAN gives you ability to do tick->daily resolutions; for equity, morning-star, future, option, forex and cfd trading - all with a fully open source project which includes samples of data to get you started. The grunt work is still done in C# so its faster than other full python based backtesting engines. Edit: I'm the Founder @ QC.
- SirLJ 9y ago
  When promoting your company, please say so, also on QuantConnect one cannot actually see the data, so there is no way to verify how good the data is...
  - rl3 9y ago
    >When promoting your company, please say so ... His bio and pronoun usage said as much. >also on QuantConnect one cannot actually see the data, so there is no way to verify how good the data is... Can you not load up the data and test it however you wish? They even say where they get their data from.
    - SirLJ 9y ago
      No you cannot
      - dsacco 9y ago
        First of all, you're criticizing QuantConnect's data when you claim to get your data from ebay. Second, yes you absolutely can verify QuantConnect's data. You can use it as much as you want within the context of their platform, you just can't download the data en masse from their platform and use it on your own. But if you have tick data (equities) or minute data (options) yourself, you can certainly verify it (which of course, you don't, because you think ebay is a good source for financial data). I am going to continue griefing you in threads like this where you spread blatant misinformation, because at this point I'm convinced you have an ulterior motive or are in fact selling this data you keep talking about on ebay.
lowpro 9y ago
The main issue I found in algo and financial aspects of programming is that the market is a zero sum game, and my intro knowledge of finance and algorithms, even when I know python, are no match for MIT PHD Quants who does it full time. There's no real way to compete with that, and therefore I would lose money, even if the data showed it might be successful in the future, firms and full time workers on algo trading would simply be faster, more focused, have more funding, and be able to quickly and constantly adapt at the scale an individual could not. So despite the fact that the subject is interesting, I'd consider it a waste of time to try and gain anything but a basic understanding of the industry and how algo trading works.
- jlg23 9y ago
  You are oversimplifying and overcomplicating at the same time. If it was an easy/obvious zero-sum game, we'd not have people going into finance at all. MIT does not teach "HFT", those "MIT PHD Quants" are just as untrained in finance as you are. The rest is math, something that requires time to study but not necessarily a certificate. And creativity. I've only seen a single HFT-algo that made me say "wow, that's creative"; all others were just straight implementations of finance "wisdom".
  - _e 9y ago
    1.) Bet what you can afford to lose 2.) The less you know the better
- SirLJ 9y ago
  Sorry, but this is like saying those CalTech PHD in Computers Science are the best, so no point for me to go into that field... This is like everything in life, not easy, but if you work on it it'll pay out in the end
  - zootam 9y ago
    >but if you work on it it'll pay out in the end unless its one of the many things in your life that don't
hendzen 9y ago
If you want to build a real algorithmic trading strategy.. please pick up a real textbook such as Qian's Quantitative Equity Portfolio Management. You should understand the following concepts at a minimum: - Markowitz portfolio optimization (mean-variance analysis) - Beta-neutral portfolios (i.e. using MSCI BARRA, sector ETFs or PCA factors, etc) - Alpha decay - Time series analysis (autocorrelation, GARCH, ARMA processes) - Basic price-based signals (momentum, volatility, value, etc) - etc, etc.
- ndesaulniers 9y ago
  Any others you (or anyone else) recommend?
  - remyp 9y ago
    Evidence-based Technical Analysis by David Aronson is a great intro.
- samolang 9y ago
  thanks for the recommendation
kyleblarson 9y ago
This is nothing more than gambling. Let's say you have 20k to play with. You would be far better off in the long run to put 15k in Wealthfront and use the other 5k as a bankroll of 25 buyins for 1/2 no limit holdem to learn the game.
JabavuAdams 9y ago
So, I got semi-seriously interested in this around the end of last year. My takeaway is: it's not about implementing a couple of trading strategies. It's about implementing a pipeline that rapidly allows you to test what-if scenarios. I might have like 10 ideas a day for strategies. How many of those can I rigorously validate per week? What about variations? I.e. tuning various hyper-parameters? Combinations? How quickly can I recombine data from various sources into exactly the layout I need to train and test this or that strat? Me: 1) Build some simple forecasting NNs. 2) Realize I need to be able to generate and test ideas waaay faster. Start working on infrastructure. 3) Get impatient -- get some "bright" ideas, do some manual trading. 4) Make some money on the first day, lose double that on the next two days. 5) Decide to ban myself from manual trading as obviously I'm an addict. Resolve to only do algorithmic trading. 6) Back to coding infrastructure. Get bored ... do some manual trades. 7) After losing around $20k, stop the trading madness, and go back to buying and holding great tech companies. 8) Enjoy my 25% returns.
- ndesaulniers 9y ago
  > 5) Decide to ban myself from manual trading as obviously I'm an addict. > 7) After losing around $20k
hagakure0c 9y ago
I use R for almost all quantitative tasks but not as a trading platform. It's great for research and testing though.
brobinson 9y ago
The article mentions a few of the pitfalls of backtesting, but it does not mention one of the best tools at your disposal in backtesting: Walk Forward Optimization/Analysis https://en.wikipedia.org/wiki/Walk_forward_optimization https://en.wikipedia.org/wiki/Walk_forward_optimization See also: "The Evaluation and Optimization of Trading Strategies" by Robert Pardo
moonforeshot 9y ago
I'm wondering... why not scala? scala gives me type-checking tools (eg dependent types) so I don't shoot myself on foot
- dsacco 9y ago
  Scala is absolutely used in finance, especially for interoperability with Java in Java-heavy firms. But it's data analysis libraries aren't as mature as the pandas/scipy stack. You'd use them for different parts of a trading pipeline.

379 points by pastefka 9y ago

It is a great article, but why on earth someone will use a service like Quantopian or similar service? They are your competitor and who will prevent a disgruntle employee or a hacker to steel your successful trading strategy? Just buy some data from eBay, you can get 20 years of historical stock market data for less than $100 and you can test any trading strategy or idea imaginable, including trend following, buy and hold ETFs, etc... The barrier of entry is pretty low and you can develop a great lifestyle business with no customers, employees and investors around that...

Blackstone4 9y ago
Have you done this? If so it would be great to get your perspective
- SirLJ 9y ago
  Yes, I am doing it as a side project and I am looking to retire in a few years and live off that...
  - sjroot 9y ago
    I second the above comment; you should write about the process and how it is working out for you.
  - gricardo99 9y ago
    I'll third this comment. Some more details on what you trade, where you trade, how long you've been doing it, performance metrics, etc... would be fascinating.
    - SirLJ 9y ago
      As you can see I am paranoid, because the more people trading the same strategy, the less effective it becomes, not to mention, if someone knows your trading strategy it can trade against you... it’s a zero sum game after all... All that said, here are the 2 most important advises I wish someone told me when I started years ago: Have an iron clad risk management strategy for your portfolio, e.g. if you are willing to risk 1% of your capital on any given day and tomorrow happens to be the Black Monday and all stock go down 50% you still have to be sure you’ll loose only 1% if you have to sell everything... This is where the back data comes into play – the goal is to back test not how much money you are going to make, but how much money you are going to loose in the worst case scenario if you have to liquidate everything and move to all cash. If you have this working 100%, you just need a trading strategy with a minimal edge and you’ll make money in the long run in the market and the compounding effect will really make you a rich person (rich person to me is someone who makes more money then he spends, WITHOUT working more than 1 hour a day) And the second advice is never ever trust and buy a trading strategy from someone, the fact he is selling or "teaching" a trading strategy means it is not working and he is trying to make some money by scamming people and selling the dream, all those trading educators cannot make money trading and that’s why they are coming with up with those fake trading courses... just check this website to see the magnitude of this scam and how people are loosing their life savings... I just don’t know why FBI, SEC and the government is not doing something to stop those guys... https://www.tradingschools.org/ https://www.tradingschools.org/ Good luck and as I said before, just treat is as a hobby at first and spend the time testing and you’ll find something that is working for you... EDIT: some spelling errors fixed.
      - princeb 9y ago
        > more people trading the same strategy, the less effective it becomes you should pick a more scalable strategy, the kind of strategy that becomes more profitable the more you talk your own book.
- dsacco 9y ago
  This commenter comes into every single thread about trading and talks about buying data from ebay, then consistently demonstrates that he doesn't know the first thing about due diligence on financial data. Each time I try to ask him about his data quality or methodology at even a high level, he responds by accusing me of wanting to steal his work or stop the democratization of data. I'm going to reiterate this right now: you are not going to find data valuable enough for trading insights on ebay. Go look at the prices of historical data for various time resolutions at TickData or CBOE Livevol and then ask yourself if cheap, competitive data on ebay is too good to be true. If the data is competitive, it's expensive. If it's not competitive, you shouldn't be paying for it, especially not from ebay. The first time I read about him doing this I was puzzled, now I'm getting increasingly frustrated because it's blatantly false information.
  - SirLJ 9y ago
    Unless you are day trading you don't need such a granular data format, so please stop saying you need to spend thousands and thousands of dollars to be able to back test a trading strategy
    - dsacco 9y ago
      If your data doesn't need to operate on an intraday scale, you can get perfectly accurate real time or historical data for free just by using a reputable broker like Interactive Brokers. At that point there's no need to buy the data on ebay, so I still don't understand why you ever would.
      - SirLJ 9y ago
        Actually I have checked IB API and it is very restrictive, e.g. you cannot request let say all daily data for all symbols for 2007 for example or even for one day for that matter...
    - lz400 9y ago
      As far as I understand historical daily data is easy to get from yahoo, google, etc. Data is only difficult to find/expensive when you need tick data.
mi100hael 9y ago
Mostly a matter of trust. I trust Quantopian more than I trust some random asshole on eBay purporting to sell accurate historical data. Plus, Quantopian is convenient.
- SirLJ 9y ago
  This is the cheapest data I can find and you can do random checks to verify the data... if you know a cheaper or even better a free place please advice...
kevstev 9y ago
Because they do so much work upfront for you. I worked in algorithmic trading for a bit over 10 years but left in 2014. I spent a few months in my spare time building out a trading and backtesting platform, pulling data off of yahoo finance, connecting to Interactive Brokers via their C++ API, etc. It was all very slow and tedious, and during my googling one day, I come across quantopian. I felt like an idiot- they already had free access to morningstar with a decent API around it, tons of other datasets you could subscribe to, and a fully fleshed out backtesting platform with risk measures, pretty graphs, etc. It would have taken me months-years to get to that point. Shortly after they also included integrations to IB and Robinhood so you could very easily take your ideas and actually live trade them. I immediately stopped development on my trading platform and just started using Q's. Q could potentially take my algorithm and start using it themselves, though they actually put money into algorithms they think are promising and share the profits with you. That's a small risk IMHO. Hackers and the like... I did this stuff for 10+ years. People WAY over estimate the value of this code- you have an idea, you backtest, hope it works, and then you put it in the wild. I have never seen an algo that is a real guaranteed money maker (aside from arbitrage/hft stuff). What are these hackers going to do when they take a 10% loss one month and don't understand why? Some "signals" can be used in-line fairly generically with other algorithms to enhance their performance, but I will still bet on the guy that knows/understands the algorithms to not lose his shirt, then some hacker dude that likely has no capital to begin with and may not even understand the basics of financial markets. One thing you seem to overlook is that it is one thing to go buy a bunch of historical data off ebay, but you need to constantly update that if you are going to live trade- Q handles that part as well. If I was looking to make a business out of this, I would build my own platform. But using it as a research platform to invest my own money, Quantopian is awesome.
- SirLJ 9y ago
  My experience is unfortunately different, I found the data for some delisted stocks like HTZ missing which will massively screw the past results due to the survivorship bias...
  - dsacco 9y ago
dglass 9y ago
Not everyone can build their own backtesting tool. I played around with quantopian for a little while and did some live trading with quantopian + robinhood and I never would have been able to do that without quantopian
- SirLJ 9y ago
  I am not a developer, but using my brokers API, examples and python, it took me few months to build a very robust and automatic trading system... I did learn python by back testing my trading ideas... To use Quantopian you still have to learn python, so I am not sure where is the benefit, considering all the problems mentioned here...
  - davewasthere 9y ago
    Or C#

ncyclopediae 9y ago

Here's a dataset on Kaggle that can be used in this process https://www.kaggle.com/biomimic/periodic-table-of-elements-mapped-to-stocks?03 https://www.kaggle.com/biomimic/periodic-table-of-elements-m...

pmart123 9y ago

Personally, I struggle to see the competitive advantage Quantopian brings. They use retail brokerage platforms to facilitate trading, which rules out anything close to HFT. Then, they are tied to any financial data vendor (Morningstar in this case) to not offer too much visibility on the underlying data. As others have mentioned, this makes it tough to validate aspects like adjusted vs. as reported earnings, how delistings are handled, etc. From my experience, getting/making sure the data is accurate is a ton of work, even if it is from good sources and you can see all the actual data. The moment an investor/trader on the platform gets traction is the moment they want Compustat data, exchange data, Bloomberg for fixed income, and will trade through Instinet/Flextrade, etc. The moment the platform is successful is the moment Morningstar could pull the rug out from them. If someone has better understanding or knowledge on Quantopian in particular, I'd be interested to hear why.

TACIXAT 9y ago
My understanding is that Quantopian has no interest in being a platform for HFT. It's there to democratize trading strategies and test them in a sound way. They do a lot to have a good backtesting platform and clean the data that is available in the platform. If I wanted to automate a strategy, I'd have to figure out the Robinhood API and basically recreate what they have. Instead, I can use their platform, their backtests, their free data, and build strategies that efficiently lose money. I think the other goal being that if you create a profitable strategy, you can enter it into competitions, trade with other people's money, and make a profit. I think the platform is a value add, I have an account and use the research notebooks. Some day when I more spare time I'll make some trading algorithms.
- SirLJ 9y ago
  The main problem with Quantopian is that the data missing especially for delisted stocks, here are some examples you can test yourself: AA, CWH, NIHD, PANL, HTZ, DSL - I have a whole list I found very fast, just imagine what other mistakes could be there... And the other problem is that they are my competitor, so I would never give them my trading strategy or ideas I am testing at the moment...

NicoJuicy 9y ago

I have a simple manual algorithm that took me from 650€ to 1450€ now (as test) in 2 months. Does someone here have experience with algo trading in c#?

Veratyr 9y ago
I won't say I have "experience" but quantconnect.com is as good a platform as I've found for C#. Their engine is open source so you don't have to rely on them either, though you'll need to provide your own data if you run it yourself.
- rl3 9y ago
  QuantConnect also offers tick-level data, though not full resolution if memory serves. Still way ahead of Quantopian in any case.
  - jaredbroad 9y ago
    Futures have millisecond timestamps (trades/quotes), equity trade ticks are rounded to the nearest second, cfd/forex are millisecond quote bars. For options we have minute resolution data =)
    - SirLJ 9y ago
      please use a disclaimer if and when you are promoting your own company...
      - thedufer 9y ago
        Does the "we" pronoun not make this sufficiently clear?
    - Veratyr 9y ago
      Worth noting though that Options data is super hard to get running on QuantConnect due to the memory limits and lack of index options.

TACIXAT 9y ago

I went to the Quantopian conference for their basic training on algorithmic trading. This blog post was pretty much what they covered, intro to pandas and a simple strategy. There is a lot of educational material on their site too (which is what you ended up getting in the paid training). My biggest thing with the Python for Finance books - I know Python, I want to learn finance. All these books are the inverse of that, for people who know finance and want to learn Python. There is a good site for quantitative economics [1] that has tutorials in Python and Julia. I would love a mathematics of finance book that had the examples in Python. 1. https://lectures.quantecon.org/py/ https://lectures.quantecon.org/py/

ice109 9y ago
those are indeed good lectures but they are not math for finance per se - more like dynamic programming for econ/finance. my impression is that finance as such is a hodge-podge of econ/accounting/business.
Maven911 9y ago
Did you find the conference worth it ? I was thinking of going to it as well or even going to the online version since it's a lot cheaper but ended up not taking part.
- TACIXAT 9y ago
  I went to the conference 1 or 2 years ago and it was good. Worth the student pricing. I only did the training this year, also on student pricing, and it was pretty disappointing.
wtvanhest 9y ago
The problem with learning finance that it takes a lot og effort. A good place to start would be taking the 3 CFA exams. After about 1000 hours of study, you will have a basic grasp of finance.
- dsacco 9y ago
  This is utterly the wrong way to learn finance for quantitative trading. That would be an unproductive use of time. Read Options, Futures and Other Derivatives and Algorithmic Trading and DMA and you basically know everything you'll get from public sources that could be meaningful for trading. If he knows how to code and is looking for "mathematics of finance" he should start with those, not the CFA.

nickthemagicman 9y ago

How is algorithmic trading not equivalent to astrology? Nothing can be predicted because there's way too many confounding factors.

fenwick67 9y ago
The net gain in stock trading is usually positive in the long run, even if you just buy and sell randomly. Just look at the DJIA.
PaulRobinson 9y ago
Algo trading typically utilises technical analysis which is basically patterns proven to repeat in markets for a variety of fundamental reasons, or fundamental analysis (e.g. algorithmically valuing and pricing options based on underlying fundamental data, sentiment analysis, etc.), and buying/shorting as appropriate. It is based on scientific methods: empirical evidence being used to validate hypothesis that produce results that can be shared and repeated by others. Astrology is story-telling with no scientific grounding.
- gnaritas 9y ago
  > technical analysis which is basically patterns proven to repeat in markets for a variety of fundamental reasons Technical analysis is merely another name for hindsight bias. Those patterns only look like they repeat in hindsight because you're ignoring all the failed patterns that setup right but failed to play out and thus don't look like the pattern in hindsight. Technical analysis is exactly like astrology and is in no way grounded in science. Quantitative analysis is based in the scientific process, technical analysis is superstition created by visual traders who think they see patterns but fail to understand biases. You are confusing the two, technical analysis is a pattern library of nonsense like "head and shoulders" or "dead cat bounce" based purely on visual patterns in price data that only "appear" to repeat due to hindsight bias. It is in no way remotely scientific. I've spend thousands of hours testing trading patterns in Forex, they're all nonsense that lose as much as they win over any decent sample size which equates to random. Edit: to the guy below, You should ask for sources that back up the claims that it does work. You won't find any, it's trader superstition, not science. You don't have to prove things don't work; you have to prove they do. At least if you want to call it based in science. However, the field of quantitative analysis itself is proof technical analysis doesn't work; it's what happened with people who actually knew math got into trading and discovered the technical analysis superstition that visual traders who lack math skills made up.

tejaswiy 9y ago

On a side note, the market continues to do well and I've been noticing this trend of active-trading, real-estate investing gurus crawl out of the wood work selling services. Please do not try to trade actively unless that's your full-time job. Passive investing using index funds is definitely not sexy, but it gets the job done.

gnaritas 9y ago
> Passive investing using index funds is definitely not sexy, but it gets the job done. Only if you have a significant amount of capital to play with. People looking into active trading are doing it because they don't have the necessary capital to make passive investing meaningful; they're looking for much much larger returns that you can get from an index fund, which necessarily comes with more risk. Traders are gamblers.
- slyfocks 9y ago
  This is extremely reckless investment advice. Active trading, by definition, incurs higher transaction costs than passive investing. Since transaction costs are generally priced per trade, active trading is more costly on a percentage basis for individuals with less capital.
  - gnaritas 9y ago
    > This is extremely reckless investment advice. It wasn't investment advice, nor did I say it was a good idea; I'm merely telling you why people choose active investing: they believe they can do better and there's enough stories out there of the "lucky" person who did better to keep that hope alive. In all likelihood they're going to lose their money, but you'll never convince people trying to win a lottery that they're better off accepting they can't. I literally said traders are gamblers, obviously that means I think active trading is gambling, that's not investment advice.
rootsudo 9y ago
Don't worry, we have cryptocurrency now. A new, unregulated frontier.

jaredbroad 9y ago

QuantConnect recently announced full python library support; and we have launched https://www.quantconnect.com/tutorials https://www.quantconnect.com/tutorials to help people write quantiative strategies in Python. QuantConnect & LEAN gives you ability to do tick->daily resolutions; for equity, morning-star, future, option, forex and cfd trading - all with a fully open source project which includes samples of data to get you started. The grunt work is still done in C# so its faster than other full python based backtesting engines. Edit: I'm the Founder @ QC.

SirLJ 9y ago
When promoting your company, please say so, also on QuantConnect one cannot actually see the data, so there is no way to verify how good the data is...
- rl3 9y ago
  >When promoting your company, please say so ... His bio and pronoun usage said as much. >also on QuantConnect one cannot actually see the data, so there is no way to verify how good the data is... Can you not load up the data and test it however you wish? They even say where they get their data from.
  - SirLJ 9y ago
    No you cannot
    - dsacco 9y ago
      First of all, you're criticizing QuantConnect's data when you claim to get your data from ebay. Second, yes you absolutely can verify QuantConnect's data. You can use it as much as you want within the context of their platform, you just can't download the data en masse from their platform and use it on your own. But if you have tick data (equities) or minute data (options) yourself, you can certainly verify it (which of course, you don't, because you think ebay is a good source for financial data). I am going to continue griefing you in threads like this where you spread blatant misinformation, because at this point I'm convinced you have an ulterior motive or are in fact selling this data you keep talking about on ebay.

lowpro 9y ago

The main issue I found in algo and financial aspects of programming is that the market is a zero sum game, and my intro knowledge of finance and algorithms, even when I know python, are no match for MIT PHD Quants who does it full time. There's no real way to compete with that, and therefore I would lose money, even if the data showed it might be successful in the future, firms and full time workers on algo trading would simply be faster, more focused, have more funding, and be able to quickly and constantly adapt at the scale an individual could not. So despite the fact that the subject is interesting, I'd consider it a waste of time to try and gain anything but a basic understanding of the industry and how algo trading works.

jlg23 9y ago
You are oversimplifying and overcomplicating at the same time. If it was an easy/obvious zero-sum game, we'd not have people going into finance at all. MIT does not teach "HFT", those "MIT PHD Quants" are just as untrained in finance as you are. The rest is math, something that requires time to study but not necessarily a certificate. And creativity. I've only seen a single HFT-algo that made me say "wow, that's creative"; all others were just straight implementations of finance "wisdom".
- _e 9y ago
  1.) Bet what you can afford to lose 2.) The less you know the better
SirLJ 9y ago
Sorry, but this is like saying those CalTech PHD in Computers Science are the best, so no point for me to go into that field... This is like everything in life, not easy, but if you work on it it'll pay out in the end
- zootam 9y ago
  >but if you work on it it'll pay out in the end unless its one of the many things in your life that don't

hendzen 9y ago

If you want to build a real algorithmic trading strategy.. please pick up a real textbook such as Qian's Quantitative Equity Portfolio Management. You should understand the following concepts at a minimum: - Markowitz portfolio optimization (mean-variance analysis) - Beta-neutral portfolios (i.e. using MSCI BARRA, sector ETFs or PCA factors, etc) - Alpha decay - Time series analysis (autocorrelation, GARCH, ARMA processes) - Basic price-based signals (momentum, volatility, value, etc) - etc, etc.

ndesaulniers 9y ago
Any others you (or anyone else) recommend?
- remyp 9y ago
  Evidence-based Technical Analysis by David Aronson is a great intro.
samolang 9y ago
thanks for the recommendation

kyleblarson 9y ago

This is nothing more than gambling. Let's say you have 20k to play with. You would be far better off in the long run to put 15k in Wealthfront and use the other 5k as a bankroll of 25 buyins for 1/2 no limit holdem to learn the game.

JabavuAdams 9y ago

So, I got semi-seriously interested in this around the end of last year. My takeaway is: it's not about implementing a couple of trading strategies. It's about implementing a pipeline that rapidly allows you to test what-if scenarios. I might have like 10 ideas a day for strategies. How many of those can I rigorously validate per week? What about variations? I.e. tuning various hyper-parameters? Combinations? How quickly can I recombine data from various sources into exactly the layout I need to train and test this or that strat? Me: 1) Build some simple forecasting NNs. 2) Realize I need to be able to generate and test ideas waaay faster. Start working on infrastructure. 3) Get impatient -- get some "bright" ideas, do some manual trading. 4) Make some money on the first day, lose double that on the next two days. 5) Decide to ban myself from manual trading as obviously I'm an addict. Resolve to only do algorithmic trading. 6) Back to coding infrastructure. Get bored ... do some manual trades. 7) After losing around $20k, stop the trading madness, and go back to buying and holding great tech companies. 8) Enjoy my 25% returns.

ndesaulniers 9y ago
> 5) Decide to ban myself from manual trading as obviously I'm an addict. > 7) After losing around $20k

hagakure0c 9y ago

I use R for almost all quantitative tasks but not as a trading platform. It's great for research and testing though.

brobinson 9y ago

The article mentions a few of the pitfalls of backtesting, but it does not mention one of the best tools at your disposal in backtesting: Walk Forward Optimization/Analysis https://en.wikipedia.org/wiki/Walk_forward_optimization https://en.wikipedia.org/wiki/Walk_forward_optimization See also: "The Evaluation and Optimization of Trading Strategies" by Robert Pardo

moonforeshot 9y ago

I'm wondering... why not scala? scala gives me type-checking tools (eg dependent types) so I don't shoot myself on foot

dsacco 9y ago
Scala is absolutely used in finance, especially for interoperability with Java in Java-heavy firms. But it's data analysis libraries aren't as mature as the pandas/scipy stack. You'd use them for different parts of a trading pipeline.