8 ms·
Launch HN: Karate Labs (YC W22) – Open-Source API and UI Test Automation
Hi HN, Peter here, founder of Karate Labs (https://karatelabs.io https://karatelabs.io) joined by my co-founder Kapil. Karate is an open-source solution unifying API and UI test automation, including mock servers and performance testing (https://github.com/karatelabs/karate https://github.com/karatelabs/karate).
Back in 2016, I was part of the API platform team at Intuit. An issue had been slowing down the team: a particular test for a set of key services would randomly fail, and it was not clear if this was a problem with the test or if there was a genuine defect. The deeper I looked, the more the complexity around the test-suite freaked me out. It was using an in-house framework, which had evolved over years and the test depended on code in multiple files scattered across the workspace. It was clear that many programmers had attempted to fix it over the years. It was next to impossible to understand what the test was doing. There had to be a better way to express web-service functional tests, and I started thinking hard about it.
This gave birth to Karate, a scriptable framework combining API and UI test automation. It has seen world wide adoption as an open-source project, including 37 of the Fortune 500 companies (so far!). Companies that have written about how they use Karate include Walmart [1], Expedia [2], Adobe [3], Trivago [4], and Oktana [5].
Karate has its own Domain Specific Language, focused on writing tests with less code and in less time. This results in easy-to-read, maintainable tests, which are often simple enough for product owners to be able to contribute to. Karate also has powerful assertions (https://twitter.com/getkarate/status/1515657727913377798 https://twitter.com/getkarate/status/1515657727913377798 ), runs tests in parallel, and can reuse API tests as performance tests, which saves time compared to rewriting performance tests using a second tool. The UI automation space is crowded, but there are very few tools that do all three: API, UI and performance testing.
Last year, we decided to leave our day-jobs and work full-time on Karate. We incorporated Karate Labs as a for-profit company with an open-core business model in mind. In recent weeks, we've released our first two open-core products.
Karate Studio can import Postman collections, Swagger, OpenAPI, HAR and cURL. Once imported, you can preview an API sequence and edit it using an intuitive no-code interface. You can then export it as a ready-to-run Karate feature file that you can integrate into your existing CI/CD or DevOps pipeline. If you already have a set of Postman collections, you can migrate them to Karate and get the benefits of parallel execution, powerful assertions and performance testing. If team members prefer Postman for exploratory testing, they can use Studio to convert their draft collections into full-fledged API automation suites, complete with assertions for complex business logic, and then use them in regression test suites. Studio can also export back to Postman if needed. It is available for a 7-day free trial at https://studio.karatelabs.io https://studio.karatelabs.io, and you can see a demo video here: https://www.youtube.com/watch?v=aJCgtnhektA https://www.youtube.com/watch?v=aJCgtnhektA.
Our second new product is an IntelliJ plugin (https://www.karatelabs.io/intellij-plugin https://www.karatelabs.io/intellij-plugin) that integrates the auto-complete experience and syntax hints that developers love. Until now, Karate support in IntelliJ was via the built-in Cucumber and Gherkin support, which was very basic. Teams have wished for a better option that would take advantage of all the Karate capabilities such as embedded JSON, JS and data assertions. Now you can write, debug, and maintain Karate tests even faster than before. The plugin is available from the JetBrains Marketplace with a 30-day free trial: https://plugins.jetbrains.com/plugin/19232-karate https://plugins.jetbrains.com/plugin/19232-karate.
When it comes to "build vs buy", many teams tend to build test automation frameworks. The fact that maintenance of an in-house framework eventually becomes prohibitive in terms of effort and cost, tends to be overlooked. We are trying to increase awareness that choosing a mature open-source framework like Karate is the right move for any team wanting to improve developer velocity.
We thank the community, developers and enterprise users of Karate for having helped us achieve broad adoption and earn credibility in the test-automation domain. We look forward to your support, feedback and suggestions.
[1] https://medium.com/walmartglobaltech/kafka-automation-using-karate-6a129cfdc210 https://medium.com/walmartglobaltech/kafka-automation-using-...
[2] https://medium.com/expedia-group-tech/karate-5-reasons-why-you-should-try-it-87539e003840 https://medium.com/expedia-group-tech/karate-5-reasons-why-y...
[3] https://adapt.to/2018/en/schedule/karate-the-black-belt-of-http-api-testing.html https://adapt.to/2018/en/schedule/karate-the-black-belt-of-h...
[4] https://tech.trivago.com/post/2019-11-14-apitestautomationusingkarate/ https://tech.trivago.com/post/2019-11-14-apitestautomationus...
[5] https://oktana.com/api-testing-using-karate-framework/ https://oktana.com/api-testing-using-karate-framework/
- FloatArtifact 4y agoI wish there was a unified back end for accessibility that screen readers could plug into and other accessibility automated tools. Their domain use cases are close in use but not the same. However, automation for testing has money behind it. Perhaps the use case could be bridged where both domains could benefit.
- ptrthomas 4y agoVery interesting point. I'm aware of the Microsoft UI Automation interface, which at least on Windows I think does exactly this: https://en.wikipedia.org/wiki/Microsoft_UI_Automation https://en.wikipedia.org/wiki/Microsoft_UI_Automation - and most Windows UI test-automation tools depend on this.
- uday_nandam 4y agoAmazing work Peter, congrats on all the success that you've had so far and future success! I've watched this project from my time at Intuit (PCG), and the impact that it had there
- ptrthomas 4y agoThanks Uday!
- orliesaurus 4y agoHow does it compare to Postman or Stoplight for testing your APIs?
- ptrthomas 4y agoPostman is a "UI driven" tool, it is great for exploratory testing of APIs and sharing the knowledge on how you call an API (e.g. collections) with others. But Karate tests are scripts that can be checked into version-control just like any other code in your team. Karate allows you to write hybrid tests, which means you can switch between API calls and UI actions within the same test script. You can also re-use API tests as performance tests with Karate. Tests are executed in parallel and you get a single HTML report. Finally, in my opinion - the assertions that Karate gives you out of the box for complex JSON payloads are hard to beat.