3 ms·

ShrimpTest: A/B Testing for WordPress

2 points by joelhaus 16y ago

Documentation: http://shrimptest.com/docs/

joelhaus 16y ago
Plugin Documentation: http://shrimptest.com/docs/ http://shrimptest.com/docs/ Demo (per user24's comment): http://shrimptest.com/tag/video/ http://shrimptest.com/tag/video/
- user24 16y ago
  Nice. I watched the demo up to 10 mins in, it told me what I wanted to know. The first 3 mins weren't very engaging for me and I switched tabs while leaving the audio running, but when you started talking about "here's a demo" I flicked back and watched the whole demo section. Just some feedback on the video. But yes, that's very good - shows me nicely what it does without me having to go through any effort. You definitely need to improve the z-score thing though. How do I know 2.4375637465% translates to 96% confidence? With conversion rates so low, I'm surprised you think the difference is significant actually. 0.003% as opposed to 0.009? This month my highest CTR was 0.74% higher than my lowest - with nothing different* (over 100,000 impressions). So I wouldn't trust a 0.006% increase as significant at all! * well - I say nothing different. In fact the content is probably always changing, but I would want to test different positions. edit: PS: you keep saying "zero point six".... isn't it "zero point zero zero six"?
  - joelhaus 16y ago
    Didn't mean to imply that it's my project (I wish); just thought others might find it useful too. Good feedback though, hopefully the author sees this.
user24 16y ago
This is exactly what I was thinking about earlier. But you need to require less of people, at the moment I need to actually download, install and fiddle with it to know what it does and how well. Give me screenshots, show me features. Can I test an entirely new theme with it or just widget placement? Can I set up a series of timed tests so I can test out 10 new themes over 20 weeks? How easy is it to create, etc etc. These questions can be answered right on the homepage, why force users to do anything else before answering them?

2 points by joelhaus 16y ago

Documentation: http://shrimptest.com/docs/

Plugin Documentation: http://shrimptest.com/docs/ http://shrimptest.com/docs/ Demo (per user24's comment): http://shrimptest.com/tag/video/ http://shrimptest.com/tag/video/

user24 16y ago
Nice. I watched the demo up to 10 mins in, it told me what I wanted to know. The first 3 mins weren't very engaging for me and I switched tabs while leaving the audio running, but when you started talking about "here's a demo" I flicked back and watched the whole demo section. Just some feedback on the video. But yes, that's very good - shows me nicely what it does without me having to go through any effort. You definitely need to improve the z-score thing though. How do I know 2.4375637465% translates to 96% confidence? With conversion rates so low, I'm surprised you think the difference is significant actually. 0.003% as opposed to 0.009? This month my highest CTR was 0.74% higher than my lowest - with nothing different* (over 100,000 impressions). So I wouldn't trust a 0.006% increase as significant at all! * well - I say nothing different. In fact the content is probably always changing, but I would want to test different positions. edit: PS: you keep saying "zero point six".... isn't it "zero point zero zero six"?
- joelhaus 16y ago
  Didn't mean to imply that it's my project (I wish); just thought others might find it useful too. Good feedback though, hopefully the author sees this.

user24 16y ago

This is exactly what I was thinking about earlier. But you need to require less of people, at the moment I need to actually download, install and fiddle with it to know what it does and how well. Give me screenshots, show me features. Can I test an entirely new theme with it or just widget placement? Can I set up a series of timed tests so I can test out 10 new themes over 20 weeks? How easy is it to create, etc etc. These questions can be answered right on the homepage, why force users to do anything else before answering them?