4 ms·

Square open sources Kochiku

67 points by xophe 13y ago

derwiki 13y ago
When I was at Causes, we wrote a similar tool called Buffet (https://github.com/causes/buffet https://github.com/causes/buffet, pronounced like the investor) for running test cases in parallel on an arbitrary number of machines; dropped out total build suite time from ~20 minutes on a single machine to 2-3 minutes. We certainly didn't add the level of polish that Kochiku seems to have, though.
- aiiane 13y ago
  Yeah, we wound up adding similar functionality to Testify (https://github.com/Yelp/Testify https://github.com/Yelp/Testify) before I left Yelp.
- Aleran 13y ago
  I looked at Buffet but I was turned away by the lack of documentation for it. There is a short readme but that's it.
- deleted 13y ago
  [deleted]
- Aaronneyer 13y ago
  Haha, I spent the whole summer pronouncing it wrong then.
vscarpenter 13y ago
I'm curious as to why Square would spend engineering talent on a distributed CI system when you can buy something like Jetbrains TeamCity that does what they wanted to do. This is not meant as a slam and I am also a developer who loves to build things but I would think Square and other companies for that matter would want to apply their limited engineering talents in solving bigger problems that grow the business. Not looking for a flame war - just curious to get different perspectives.
- j_s 13y ago
  Would you mind starting the conversation by addressing the 'What about existing solutions?' portion of the article? edit: rephrase
- deleted 13y ago
  [deleted]
- hkarthik 13y ago
  It makes a lot of sense. I'm currently working on a large Rails app and we've felt a lot of pain with an off the shelf CI System (Jenkins) and the idea of writing our own CI has been mentioned a few times. For Team City, here's the cost breakdown: TeamCity Enterprise = $1,999 229 Build agents @ $299/each = $68,741 Total yearly license cost ~ $70K This doesn't include the time that a team of developers will spend configuring the build and babysitting it during through the early growing pains of adopting a CI system. Also you'll probably still need a build/release engineering team to manage the server when you have a team of 100+ devs like Square does. When you add it all up, building a custom CI setup makes a lot of sense. And I'm sure they grew it over time, with an initial version that was usable being completed in just a few months. What they've open sourced is the end result of spending that time. When you really adopt testing as part of your culture, CI becomes totally critical so engineering should spend the time to make it a solid and viable solution.
  - MasterScrat 13y ago
    I'm currently considering Jenkins for my next project; what were its shortcomings for you?
    - hkarthik 13y ago
      For us it has just become extremely fragile. We often lose executors and the Jenkins processes tend to leak memory. In general we probably restart executor machines quite often just due to the above issues alone. Also our Devops folks have complained about the lack of automation around Jenkins. If they could monitor it better, we could probably be a lot more proactive about dealing with our problems around it. If you're looking at CI for your next project, I would look at either a hosted solution like CircleCI or TravisCI Pro. Don't jump straight to self-hosted CI until you outgrow these solutions.
polskibus 13y ago
For distributed build, testing this is the best tool I've used: http://www.incredibuild.com/ http://www.incredibuild.com/. Unfortunately, it is not open source. Does anyone know of a free alternative to incredibuild with a similar feature set?
joeblau 13y ago
One of the things i'm really loving about Square is the utilities that they are open sourcing that are not core to their business. I think this is great for the community.
- aguynamedrich 13y ago
  Their Android libraries and their involvement in the Android dev community are incredible.
nobodysfool 13y ago
So now we've gone from adding an 'er' to everything and dropping the 'e' to using Japanese words and proper names for our products. And I was just going to iPost that to my iBlog.
pbiggar 13y ago
CircleCI (https://circleci.com https://circleci.com) offers this right now. We automatically partition your test suite and run it across multiple machines. Except since it's in the cloud, you don't need to own 10 Mac minis to make this happen. And if you decide you want to split it 20 ways, not 10, you can do that in about 2 clicks. [disclaimer: I work at https://circleci.com https://circleci.com, if that wasn't already obvious]
- statusgraph 13y ago
  It sounds like Square doesn't use 10 mac minis but rather EC2 spot instances....
- Aleran 13y ago
  The nice thing about Kochiku is that it is a free open source project whereas CircleCI is a paid hosted solution. If you have hardware available to use as workers on then Kochiku is free to use. There is a real cost to the time required to setup and maintain Kochiku but, at scale, within a larger company, this route can make sense. I am sure CircleCI takes security very seriously but it also nice (and a requirement for certain companies) that with a self hosted CI server, that your source code does not leave the company.

67 points by xophe 13y ago

When I was at Causes, we wrote a similar tool called Buffet (https://github.com/causes/buffet https://github.com/causes/buffet, pronounced like the investor) for running test cases in parallel on an arbitrary number of machines; dropped out total build suite time from ~20 minutes on a single machine to 2-3 minutes. We certainly didn't add the level of polish that Kochiku seems to have, though.

aiiane 13y ago
Yeah, we wound up adding similar functionality to Testify (https://github.com/Yelp/Testify https://github.com/Yelp/Testify) before I left Yelp.
Aleran 13y ago
I looked at Buffet but I was turned away by the lack of documentation for it. There is a short readme but that's it.
deleted 13y ago
[deleted]
Aaronneyer 13y ago
Haha, I spent the whole summer pronouncing it wrong then.

vscarpenter 13y ago

I'm curious as to why Square would spend engineering talent on a distributed CI system when you can buy something like Jetbrains TeamCity that does what they wanted to do. This is not meant as a slam and I am also a developer who loves to build things but I would think Square and other companies for that matter would want to apply their limited engineering talents in solving bigger problems that grow the business. Not looking for a flame war - just curious to get different perspectives.

j_s 13y ago
Would you mind starting the conversation by addressing the 'What about existing solutions?' portion of the article? edit: rephrase
deleted 13y ago
[deleted]
hkarthik 13y ago
It makes a lot of sense. I'm currently working on a large Rails app and we've felt a lot of pain with an off the shelf CI System (Jenkins) and the idea of writing our own CI has been mentioned a few times. For Team City, here's the cost breakdown: TeamCity Enterprise = $1,999 229 Build agents @ $299/each = $68,741 Total yearly license cost ~ $70K This doesn't include the time that a team of developers will spend configuring the build and babysitting it during through the early growing pains of adopting a CI system. Also you'll probably still need a build/release engineering team to manage the server when you have a team of 100+ devs like Square does. When you add it all up, building a custom CI setup makes a lot of sense. And I'm sure they grew it over time, with an initial version that was usable being completed in just a few months. What they've open sourced is the end result of spending that time. When you really adopt testing as part of your culture, CI becomes totally critical so engineering should spend the time to make it a solid and viable solution.
- MasterScrat 13y ago
  I'm currently considering Jenkins for my next project; what were its shortcomings for you?
  - hkarthik 13y ago
    For us it has just become extremely fragile. We often lose executors and the Jenkins processes tend to leak memory. In general we probably restart executor machines quite often just due to the above issues alone. Also our Devops folks have complained about the lack of automation around Jenkins. If they could monitor it better, we could probably be a lot more proactive about dealing with our problems around it. If you're looking at CI for your next project, I would look at either a hosted solution like CircleCI or TravisCI Pro. Don't jump straight to self-hosted CI until you outgrow these solutions.

polskibus 13y ago

For distributed build, testing this is the best tool I've used: http://www.incredibuild.com/ http://www.incredibuild.com/. Unfortunately, it is not open source. Does anyone know of a free alternative to incredibuild with a similar feature set?

joeblau 13y ago

One of the things i'm really loving about Square is the utilities that they are open sourcing that are not core to their business. I think this is great for the community.

aguynamedrich 13y ago
Their Android libraries and their involvement in the Android dev community are incredible.

nobodysfool 13y ago

So now we've gone from adding an 'er' to everything and dropping the 'e' to using Japanese words and proper names for our products. And I was just going to iPost that to my iBlog.

pbiggar 13y ago

CircleCI (https://circleci.com https://circleci.com) offers this right now. We automatically partition your test suite and run it across multiple machines. Except since it's in the cloud, you don't need to own 10 Mac minis to make this happen. And if you decide you want to split it 20 ways, not 10, you can do that in about 2 clicks. [disclaimer: I work at https://circleci.com https://circleci.com, if that wasn't already obvious]

statusgraph 13y ago
It sounds like Square doesn't use 10 mac minis but rather EC2 spot instances....
Aleran 13y ago
The nice thing about Kochiku is that it is a free open source project whereas CircleCI is a paid hosted solution. If you have hardware available to use as workers on then Kochiku is free to use. There is a real cost to the time required to setup and maintain Kochiku but, at scale, within a larger company, this route can make sense. I am sure CircleCI takes security very seriously but it also nice (and a requirement for certain companies) that with a self hosted CI server, that your source code does not leave the company.