4 ms·

I hope no-one minds me touting a free course I'm offering on bandit algorithms: http://noelwelsh.com/data/2013/08/05/why-you-should-know-about-bandit-algorithm

I hope no-one minds me touting a free course I'm offering on bandit algorithms: http://noelwelsh.com/data/2013/08/05/why-you-should-know-about-bandit-algorithms/ http://noelwelsh.com/data/2013/08/05/why-you-should-know-abo... I was literally working on the post when Patrick's email arrived. It's more focused on the algorithms and will, I think, be a good compliment to what Patrick is offering.

patio11 13y ago
I upvoted you from zero, in the interest of HN remaining a supportive and congenial place for people who are doing stuff to move the industry's state of play forward.
- noelwelsh 13y ago
  Thanks! I appreciate your support, especially given you don't agree with me on the benefits of bandit testing.
badclient 13y ago
I signed up as someone who has a deep need for this. We are trying to run through 4 months of cold calling data to identify the ideal customer profile / call frequency etc. that has the highest odds of converting into a sale. I vaguely remember learning some of it in my Marketing Research class but have since forgotten. From your post it sounded like a free online course but it seems like the free course will be presented in person? I wish you had a more formal online offering.
- noelwelsh 13y ago
  The free course is online. Let me give the backstory, which I hope will clarify things: I'm giving a talk at Strata London (a "Big Data" conference) in a few months. I won't be able to go into details for the talk, so I want to create supplementary material. I seems sharing this material online will be good for others (and for me). I don't want to plonk it all on the web because 1) I don't want to make all my material available till after the talk and 2) I want to build a mailing list of people interested in this kind of stuff. I may build the material into a more formal course if there is sufficient interest.
rfergie 13y ago
I've been doing some simulations with bandit algorithms. Early results make them look pretty good (http://www.eanalytica.com/blog/split-test-methodologies-preview/ http://www.eanalytica.com/blog/split-test-methodologies-prev...). However, this result could also be explained by the bandit having a lot more variations to play with. I have a lot more work to do on this before I feel sure of an answer
- noelwelsh 13y ago
  Interesting. I'd like to know more about how you setup your experiments.
  - rfergie 13y ago
    Have chucked the simulation code up on github: https://github.com/richardfergie/split-test-simulator https://github.com/richardfergie/split-test-simulator Current things that I think are wrong with it: * Successive tests should not be independent; picking the wrong option should have repercussions beyond the next test * The distribution of conversion rates from which samples are drawn was picked because it is convenient not because it is correct * Takes no account of resource issues around creating new variations e.g. it is unlikely that people can create a new variation every day but some strategies assume they can. Please let me know your thoughts

patio11 13y ago

I upvoted you from zero, in the interest of HN remaining a supportive and congenial place for people who are doing stuff to move the industry's state of play forward.

noelwelsh 13y ago
Thanks! I appreciate your support, especially given you don't agree with me on the benefits of bandit testing.

badclient 13y ago

I signed up as someone who has a deep need for this. We are trying to run through 4 months of cold calling data to identify the ideal customer profile / call frequency etc. that has the highest odds of converting into a sale. I vaguely remember learning some of it in my Marketing Research class but have since forgotten. From your post it sounded like a free online course but it seems like the free course will be presented in person? I wish you had a more formal online offering.

noelwelsh 13y ago
The free course is online. Let me give the backstory, which I hope will clarify things: I'm giving a talk at Strata London (a "Big Data" conference) in a few months. I won't be able to go into details for the talk, so I want to create supplementary material. I seems sharing this material online will be good for others (and for me). I don't want to plonk it all on the web because 1) I don't want to make all my material available till after the talk and 2) I want to build a mailing list of people interested in this kind of stuff. I may build the material into a more formal course if there is sufficient interest.

rfergie 13y ago

I've been doing some simulations with bandit algorithms. Early results make them look pretty good (http://www.eanalytica.com/blog/split-test-methodologies-preview/ http://www.eanalytica.com/blog/split-test-methodologies-prev...). However, this result could also be explained by the bandit having a lot more variations to play with. I have a lot more work to do on this before I feel sure of an answer

noelwelsh 13y ago
Interesting. I'd like to know more about how you setup your experiments.
- rfergie 13y ago
  Have chucked the simulation code up on github: https://github.com/richardfergie/split-test-simulator https://github.com/richardfergie/split-test-simulator Current things that I think are wrong with it: * Successive tests should not be independent; picking the wrong option should have repercussions beyond the next test * The distribution of conversion rates from which samples are drawn was picked because it is convenient not because it is correct * Takes no account of resource issues around creating new variations e.g. it is unlikely that people can create a new variation every day but some strategies assume they can. Please let me know your thoughts