4 ms·

Yep, I like Locust but been a while since I’ve needed it again. It’s main advantage over simpler tools like ab or httperf is being able to simulate most any ki

Yep, I like Locust but been a while since I’ve needed it again. It’s main advantage over simpler tools like ab or httperf is being able to simulate most any kind of user interaction. Whereas other tools tend to just spam static predefined HTTP requests at a server (which is great when that’s all you need), Locust is great at simulating complex user behavior, such as users clicking through different pages of your site, or clients that make common sequences of API requests, perhaps to a variety of servers at different urls. In my case, I was load testing a RESTful DNS management api over http and, as part of the same performance test, simulating DNS traffic to nameservers managed by that API. Locust is horizontally scalable and incredibly flexible since you can write arbitrary Python code to make it do whatever you want.

hardwaresofton 6y ago
> In my case, I was load testing a RESTful DNS management api over http and, as part of the same performance test, simulating DNS traffic to nameservers managed by that API. Locust is horizontally scalable and incredibly flexible since you can write arbitrary Python code to make it do whatever you want. This sounds really interesting and like good engineering (tm), would you mind sharing which company this was for? Maintaining good blackbox e2e tests and the infrastructure is one thing, finding time (and space in the roadmap) to do that and load test on top of that is a great engineering indicator IMO. I wonder if it would be possible to translate regular recorded user activity (assuming the API was instrumented) for it and find some way to randomize & generate it (a la quickcheck) and re-feed it into the system. There are some sticky points like pre-existing state (if you take the recording of any given day), but you could probably discover/evaluate that from API requirements (if you saw a `GET /resources/1` which succeeded, then obviously a `POST /resources` must have occurred before hand).
- quacker 6y ago
  > This sounds really interesting and like good engineering (tm), would you mind sharing which company this was for? Maintaining good blackbox e2e tests and the infrastructure is one thing, finding time (and space in the roadmap) to do that and load test on top of that is a great engineering indicator IMO. You’re really hearing about a unique case. The engineering org there had basically no consistency across teams/projects. Every product team was independent and responsible for their own services. So different teams deployed their services to different infrastructure with different programming languages, different frameworks, different databases, different monitoring and logging solutions, and so on. I was in the QA org and was embedded as an SDET on a particularly well-functioning product team and had a lot of independence to solve problems as I saw fit with support from great forward-thinking devs. I got their e2e/integration tests to a point where it was super easy for devs to add at least some new tests themselves, which freed up some of my time to slowly build out a performance test. But, very few other teams were getting to the point of doing any non-trivial load testing. I tried evangelizing Locust a little bit, but there wasn’t enough demand for it without a turnkey solution - something a bit hard to make without much consistency across teams. Not to mention the culture and processes weren’t there to consolidate on a single solution anyhow. So, grass is always greener. That company’s engineering org has since been decimated by layoffs and outsourcing unfortunately.
  - hardwaresofton 6y ago
    oof, thanks for sharing -- was hoping it wasn't this but looks like this: > I was in the QA org and was embedded as an SDET on a particularly well-functioning product team and had a lot of independence to solve problems as I saw fit with support from great forward-thinking devs. I got their e2e/integration tests to a point where it was super easy for devs to add at least some new tests themselves, which freed up some of my time to slowly build out a performance test. is hard to find sustainably happening in the wild.
- tortila 6y ago
  I’m not sure if this is what you’re looking for, but me and my colleagues created a tool (https://github.com/zalando-incubator/transformer https://github.com/zalando-incubator/transformer) to convert actions recorded in a browser (in har format) to Locust scenarios. Recorded state (like cookies) can be programmatically removed by user-defined plugins.
  - hardwaresofton 6y ago
    Whoa this looks awesome, thanks -- more awesome tech out of Zalando (I'm familiar with patroni/postgres-operator)!

hardwaresofton 6y ago

> In my case, I was load testing a RESTful DNS management api over http and, as part of the same performance test, simulating DNS traffic to nameservers managed by that API. Locust is horizontally scalable and incredibly flexible since you can write arbitrary Python code to make it do whatever you want. This sounds really interesting and like good engineering (tm), would you mind sharing which company this was for? Maintaining good blackbox e2e tests and the infrastructure is one thing, finding time (and space in the roadmap) to do that and load test on top of that is a great engineering indicator IMO. I wonder if it would be possible to translate regular recorded user activity (assuming the API was instrumented) for it and find some way to randomize & generate it (a la quickcheck) and re-feed it into the system. There are some sticky points like pre-existing state (if you take the recording of any given day), but you could probably discover/evaluate that from API requirements (if you saw a `GET /resources/1` which succeeded, then obviously a `POST /resources` must have occurred before hand).

quacker 6y ago
> This sounds really interesting and like good engineering (tm), would you mind sharing which company this was for? Maintaining good blackbox e2e tests and the infrastructure is one thing, finding time (and space in the roadmap) to do that and load test on top of that is a great engineering indicator IMO. You’re really hearing about a unique case. The engineering org there had basically no consistency across teams/projects. Every product team was independent and responsible for their own services. So different teams deployed their services to different infrastructure with different programming languages, different frameworks, different databases, different monitoring and logging solutions, and so on. I was in the QA org and was embedded as an SDET on a particularly well-functioning product team and had a lot of independence to solve problems as I saw fit with support from great forward-thinking devs. I got their e2e/integration tests to a point where it was super easy for devs to add at least some new tests themselves, which freed up some of my time to slowly build out a performance test. But, very few other teams were getting to the point of doing any non-trivial load testing. I tried evangelizing Locust a little bit, but there wasn’t enough demand for it without a turnkey solution - something a bit hard to make without much consistency across teams. Not to mention the culture and processes weren’t there to consolidate on a single solution anyhow. So, grass is always greener. That company’s engineering org has since been decimated by layoffs and outsourcing unfortunately.
- hardwaresofton 6y ago
  oof, thanks for sharing -- was hoping it wasn't this but looks like this: > I was in the QA org and was embedded as an SDET on a particularly well-functioning product team and had a lot of independence to solve problems as I saw fit with support from great forward-thinking devs. I got their e2e/integration tests to a point where it was super easy for devs to add at least some new tests themselves, which freed up some of my time to slowly build out a performance test. is hard to find sustainably happening in the wild.
tortila 6y ago
I’m not sure if this is what you’re looking for, but me and my colleagues created a tool (https://github.com/zalando-incubator/transformer https://github.com/zalando-incubator/transformer) to convert actions recorded in a browser (in har format) to Locust scenarios. Recorded state (like cookies) can be programmatically removed by user-defined plugins.
- hardwaresofton 6y ago
  Whoa this looks awesome, thanks -- more awesome tech out of Zalando (I'm familiar with patroni/postgres-operator)!