4 ms·

Causes open sources Buffet, a Ruby testing framework.

26 points by johnfn 15y ago

grimen 15y ago
So the question that remains after reading the README is...what does it do? :)
- johnfn 15y ago
  Ah, didn't realize that was ambiguous. I edited the opening - does that make it more clear?
  - grimen 15y ago
    Yes now I got it. :)
mattraibert 15y ago
How are people using this? For having an incredibly fast suite of unit tests (i.e. speed up your suite from 25 seconds to 5 seconds)? Or are we talking about a 25 hour build that now runs in 5 hours?
- johnfn 15y ago
  It speeds up about linearly; 5 more machines will make it run about 5 times as fast, etc. For incredibly fast suites, it isn't very helpful, because the startup and sync times across machines outweigh the benefits. At causes, our tests run in about 20 minutes on 1 machine, and 4 minutes with buffet and 5.
kunley 15y ago
While this is cool, I must admin it's a bit "worse is better" approach. Wouldn't be cooler just to unbloat existing test tools?
- johnfn 15y ago
  I don't think so. I mean, RSpec is pretty fast as is, at least for running only hundreds of tests. The problem we were encountering is that if you want to get full coverage, some tests will have to hit the database or do other (mildly) time consuming tasks, and if you have a large enough test suite doing these things all over the place, they tend to add up into ridiculously long test runs.
rawsyntax 15y ago
sounds like another hydra ( https://github.com/ngauthier/hydra https://github.com/ngauthier/hydra ). It distributes your tests across processor cores, and optionally remote workers (other machines) Why is Buffet better? (I'm genuinely interested)
- johnfn 15y ago
  This is a good question, because we actually were using Hydra for a while. We ran into a lot of problems where hydra workers would just hang once they finished their test (used strace to see what they were doing, and it looked like they were stuck in some sort of infinite loop). Hydra caused enough headaches that we just stopped using it; out of that frustration, Buffet was born. We tried pretty hard to make Buffet 'just work' and stable.
- RobertLowe 15y ago
  As far as I know Hydra can't run on 1.9 ( https://github.com/ngauthier/hydra/issues/29 https://github.com/ngauthier/hydra/issues/29 ) Alternatively, you have parallel_tests ( https://github.com/grosser/parallel_tests https://github.com/grosser/parallel_tests ) which can run on multiple cores. That's great if you use a mac pro, or some behemoth machine. Little known there's testbot ( https://github.com/joakimk/testbot https://github.com/joakimk/testbot ) which can run tests across machines & cores. They've even worked in a bootstrapping process https://github.com/joakimk/testbot_cloud https://github.com/joakimk/testbot_cloud From ( https://github.com/joakimk/testbot/wiki/How-testbot-is-used https://github.com/joakimk/testbot/wiki/How-testbot-is-used ) "... 60 minutes of tests to run in 10 minutes using 16 medium EC2 instances (32 cores), read the blog post." reports thoughtbot! Code softly hackers.
tonycoco 15y ago
Isn't this exactly like travisci?

26 points by johnfn 15y ago

grimen 15y ago

So the question that remains after reading the README is...what does it do? :)

johnfn 15y ago
Ah, didn't realize that was ambiguous. I edited the opening - does that make it more clear?
- grimen 15y ago
  Yes now I got it. :)

mattraibert 15y ago

How are people using this? For having an incredibly fast suite of unit tests (i.e. speed up your suite from 25 seconds to 5 seconds)? Or are we talking about a 25 hour build that now runs in 5 hours?

johnfn 15y ago
It speeds up about linearly; 5 more machines will make it run about 5 times as fast, etc. For incredibly fast suites, it isn't very helpful, because the startup and sync times across machines outweigh the benefits. At causes, our tests run in about 20 minutes on 1 machine, and 4 minutes with buffet and 5.

kunley 15y ago

While this is cool, I must admin it's a bit "worse is better" approach. Wouldn't be cooler just to unbloat existing test tools?

johnfn 15y ago
I don't think so. I mean, RSpec is pretty fast as is, at least for running only hundreds of tests. The problem we were encountering is that if you want to get full coverage, some tests will have to hit the database or do other (mildly) time consuming tasks, and if you have a large enough test suite doing these things all over the place, they tend to add up into ridiculously long test runs.

rawsyntax 15y ago

sounds like another hydra ( https://github.com/ngauthier/hydra https://github.com/ngauthier/hydra ). It distributes your tests across processor cores, and optionally remote workers (other machines) Why is Buffet better? (I'm genuinely interested)

johnfn 15y ago
This is a good question, because we actually were using Hydra for a while. We ran into a lot of problems where hydra workers would just hang once they finished their test (used strace to see what they were doing, and it looked like they were stuck in some sort of infinite loop). Hydra caused enough headaches that we just stopped using it; out of that frustration, Buffet was born. We tried pretty hard to make Buffet 'just work' and stable.
RobertLowe 15y ago
As far as I know Hydra can't run on 1.9 ( https://github.com/ngauthier/hydra/issues/29 https://github.com/ngauthier/hydra/issues/29 ) Alternatively, you have parallel_tests ( https://github.com/grosser/parallel_tests https://github.com/grosser/parallel_tests ) which can run on multiple cores. That's great if you use a mac pro, or some behemoth machine. Little known there's testbot ( https://github.com/joakimk/testbot https://github.com/joakimk/testbot ) which can run tests across machines & cores. They've even worked in a bootstrapping process https://github.com/joakimk/testbot_cloud https://github.com/joakimk/testbot_cloud From ( https://github.com/joakimk/testbot/wiki/How-testbot-is-used https://github.com/joakimk/testbot/wiki/How-testbot-is-used ) "... 60 minutes of tests to run in 10 minutes using 16 medium EC2 instances (32 cores), read the blog post." reports thoughtbot! Code softly hackers.

tonycoco 15y ago

Isn't this exactly like travisci?