5 ms·

Show HN: A/Bingo for PHP

33 points by sdrinf 16y ago

sdrinf 16y ago
Cog is a little webdev framework of mine, a work-in-progress steal from spending couple of years in different companies. Since A/B testing is considered pretty fundamental to most of my clients, I thought I'd share with the dear community of HN one possible implementation of it. You can see A/B testing in action on the sample page: http://github.com/waverz/cog/blob/master/cog-guestbook/pages/guestbook.html http://github.com/waverz/cog/blob/master/cog-guestbook/pages... When used together with scriptor, an A/Btest is about 1 line in the design file (db rows, and counters are automagically set up on first page visiting). Stand-alone you might want to integrate it with a templating engine of your own :) Code is PD, and review is more, than welcome.
- Nycto 16y ago
  > review is more, than welcome. You opened the door, so here is my feedback: - You never have to explicitly define any of the super globals as global... you can delete any line that says "global $_SESSION;", for example - The database interactions look like they are heavily dependent on global state. It is also a tightly coupled dependency - Use __construct for constructors instead of the class name: http://www.php.net/manual/en/language.oop5.decon.php http://www.php.net/manual/en/language.oop5.decon.php - Foreach is better than a for loop for iterating over arrays - Single character variable names only make code more confusing. You write code once, you read it a dozen times. I challenge you to rewrite this with only using classes (no global functions) and to write unit tests for your code. Remember, if you have to jump through hoops to test your code, you need to rethink what you're doing.
- lmz 16y ago
  That resolvequery() function in cog_database.php looks interesting. Wonder what happens if you call resolvequery('INSERT INTO ... VALUES (@email, @example)', array('email' => 'test@example.com', 'example' => 123));
patio11 16y ago
Thanks -- PHP developers need better alternatives for A/B testing than Google Website Optimizer. Your code is about the minimum viable A/B testing library for a developer: you can use it to start getting your feet wet with A/B testing done the right way. I think if you start using it in production you will quickly understand that some of the implementation choices you've made are going to cause you pain, but that is less important than doing A/B testing. You can always rewrite the internals later. (I've done it a few times.)
- sdrinf 16y ago
  Thanks. Would you mind pointing out implementation choices, that will come back biting us down the road? :)
  - deleted 16y ago
    [deleted]
  - patio11 16y ago
    Your mileage may vary, and I don't guarantee I'm reading your code right, but: 1) You will find it useful to be able to coerce A/B test values, particularly in testing and sometimes (for CS purposes) on the live site. 2) I do not see a convenient method of having Ms. Smith get the same website at home and at her workplace. That may or may not be a problem for you. 3) You may find "control" and "experiment" to be limiting once you start doing this frequently -- they aren't nearly as self-documenting six months down the line when you're staring at your web page wondering "What possessed me to make this button teal?!" 4) I'm pretty sure you have a race condition for creation of new tests. (This has bitten A/Bingo a few times.)
    - tomjen3 16y ago
      I do not see a convenient method of having Ms. Smith get the same website at home and at her workplace. That may or may not be a problem for you. How did get this to work with A/Bingo? Did you "just" write the A/B tests such that they are tied together with the user database?
aliasaria 16y ago
As an alternative, I wrote a port of Vanity for PHP that we use for A/B testing. It uses redis to store data http://github.com/aliasaria/PHP-redis-A-B-Testing http://github.com/aliasaria/PHP-redis-A-B-Testing
- armandososa 16y ago
  This is an honest question: is REDIS the best option for storing A/B tests or is just a matter of taste? MongoDB or even MySql could work?
  - patio11 16y ago
    I try to keep most of the A/Bingo work in a cache store (Redis, by choice) because sites with traffic two orders of magnitude or so higher than mine would blow up if they used MySQL for it. At something like Facebook game scales I think it would be smart to take A/Bingo and rip the database out entirely. Writes hurt. A/B testing requires, minimally, O(tests * participants) writes. (And more reads, but the disk will be on fire from writes first.) However, despite the cost, A/B testing benefits minimally from ACID. If you lose 2% of writes, oh well, it comes out in the wash anyhow. Thus the huge hardware/maintenance budget you'd need for web scale A/B testing on an ACID architecture makes little sense.

33 points by sdrinf 16y ago

Cog is a little webdev framework of mine, a work-in-progress steal from spending couple of years in different companies. Since A/B testing is considered pretty fundamental to most of my clients, I thought I'd share with the dear community of HN one possible implementation of it. You can see A/B testing in action on the sample page: http://github.com/waverz/cog/blob/master/cog-guestbook/pages/guestbook.html http://github.com/waverz/cog/blob/master/cog-guestbook/pages... When used together with scriptor, an A/Btest is about 1 line in the design file (db rows, and counters are automagically set up on first page visiting). Stand-alone you might want to integrate it with a templating engine of your own :) Code is PD, and review is more, than welcome.

Nycto 16y ago
> review is more, than welcome. You opened the door, so here is my feedback: - You never have to explicitly define any of the super globals as global... you can delete any line that says "global $_SESSION;", for example - The database interactions look like they are heavily dependent on global state. It is also a tightly coupled dependency - Use __construct for constructors instead of the class name: http://www.php.net/manual/en/language.oop5.decon.php http://www.php.net/manual/en/language.oop5.decon.php - Foreach is better than a for loop for iterating over arrays - Single character variable names only make code more confusing. You write code once, you read it a dozen times. I challenge you to rewrite this with only using classes (no global functions) and to write unit tests for your code. Remember, if you have to jump through hoops to test your code, you need to rethink what you're doing.
lmz 16y ago
That resolvequery() function in cog_database.php looks interesting. Wonder what happens if you call resolvequery('INSERT INTO ... VALUES (@email, @example)', array('email' => 'test@example.com', 'example' => 123));

patio11 16y ago

Thanks -- PHP developers need better alternatives for A/B testing than Google Website Optimizer. Your code is about the minimum viable A/B testing library for a developer: you can use it to start getting your feet wet with A/B testing done the right way. I think if you start using it in production you will quickly understand that some of the implementation choices you've made are going to cause you pain, but that is less important than doing A/B testing. You can always rewrite the internals later. (I've done it a few times.)

sdrinf 16y ago
Thanks. Would you mind pointing out implementation choices, that will come back biting us down the road? :)
- deleted 16y ago
  [deleted]
- patio11 16y ago
  Your mileage may vary, and I don't guarantee I'm reading your code right, but: 1) You will find it useful to be able to coerce A/B test values, particularly in testing and sometimes (for CS purposes) on the live site. 2) I do not see a convenient method of having Ms. Smith get the same website at home and at her workplace. That may or may not be a problem for you. 3) You may find "control" and "experiment" to be limiting once you start doing this frequently -- they aren't nearly as self-documenting six months down the line when you're staring at your web page wondering "What possessed me to make this button teal?!" 4) I'm pretty sure you have a race condition for creation of new tests. (This has bitten A/Bingo a few times.)
  - tomjen3 16y ago
    I do not see a convenient method of having Ms. Smith get the same website at home and at her workplace. That may or may not be a problem for you. How did get this to work with A/Bingo? Did you "just" write the A/B tests such that they are tied together with the user database?

aliasaria 16y ago

As an alternative, I wrote a port of Vanity for PHP that we use for A/B testing. It uses redis to store data http://github.com/aliasaria/PHP-redis-A-B-Testing http://github.com/aliasaria/PHP-redis-A-B-Testing

armandososa 16y ago
This is an honest question: is REDIS the best option for storing A/B tests or is just a matter of taste? MongoDB or even MySql could work?
- patio11 16y ago
  I try to keep most of the A/Bingo work in a cache store (Redis, by choice) because sites with traffic two orders of magnitude or so higher than mine would blow up if they used MySQL for it. At something like Facebook game scales I think it would be smart to take A/Bingo and rip the database out entirely. Writes hurt. A/B testing requires, minimally, O(tests * participants) writes. (And more reads, but the disk will be on fire from writes first.) However, despite the cost, A/B testing benefits minimally from ACID. If you lose 2% of writes, oh well, it comes out in the wash anyhow. Thus the huge hardware/maintenance budget you'd need for web scale A/B testing on an ACID architecture makes little sense.