4 ms·

Yes, that's where I'd like to start (and I sort of did, but the approach I took was nowhere near good enough). Like you say, it does take time to build the har

by clarry 7y ago

Yes, that's where I'd like to start (and I sort of did, but the approach I took was nowhere near good enough). Like you say, it does take time to build the harness. Kind of a hard sell with our client.. I'd need to mock hardware I don't have, possibly mock some kernel interfaces, set up a physical test endpoint that can observe things on the wire (as in raw ethernet packets, but also application level protocols..). And well, really capturing timing issues and such is kinda important because things like flow control, multipath routing & failover are critical features of the software. Of course some of the things you want to test are inherently timing related issues. Like, a certain signal must be asserted if a particular message isn't received within a specified (possibly configurable or not) time window after some event. The core of the software is a heavily asynchronous & multithreaded mess.. Getting all the scaffolding in place and then automating everything is going to require lots and lots of time. And reverse engineering. I tried playing with bash scripts and docker based virtual deployments but it completely misses the hardware side of things and really doesn't get me far.

akavel 7y ago
Hm, do you know of any tool/framework for capturing and then re-running (mocking) syscall responses? Could be interesting to keep in a toolbox for future...
pydry 7y ago
I did this once on a large project I recently joined. It took a long time to build these sophisticated mocks (3-4 weeks) and it was a fairly hard sell but it did have the advantage that it's somewhat immune from mythical man month issues. That is, you can hire a new person, they can do all this work and it doesn't affect the speed that the rest of the team delivers at much. They don't need to know extensive amounts about the underlying core code - just the way it interfaces with the outside world. It's also a pretty effective way of onboarding somebody on to the project. That isn't the case for any kind of unit testing approach, where you have to be able to work with, understand and refactor the core code. That requires core dev team members and will take their attention away from customer facing tasks.

by clarry 7y ago

akavel 7y ago

Hm, do you know of any tool/framework for capturing and then re-running (mocking) syscall responses? Could be interesting to keep in a toolbox for future...

pydry 7y ago

I did this once on a large project I recently joined. It took a long time to build these sophisticated mocks (3-4 weeks) and it was a fairly hard sell but it did have the advantage that it's somewhat immune from mythical man month issues. That is, you can hire a new person, they can do all this work and it doesn't affect the speed that the rest of the team delivers at much. They don't need to know extensive amounts about the underlying core code - just the way it interfaces with the outside world. It's also a pretty effective way of onboarding somebody on to the project. That isn't the case for any kind of unit testing approach, where you have to be able to work with, understand and refactor the core code. That requires core dev team members and will take their attention away from customer facing tasks.