3 ms·
An example of how to do this is the documentation for the PHP framework Symfony. Code examples from the documentation are run in the CI server. If a pull reques
by blowski 5y ago
An example of how to do this is the documentation for the PHP framework Symfony. Code examples from the documentation are run in the CI server. If a pull request breaks a code example, then that example must also be fixed as part of the pull request.
That's a fantastic feature for a popular open-source framework, as it means the documentation remains up to date.
I'm not sure at what point it's worth the effort for an internal project, though. If you have a cultural problem with incorrect, out-of-date or missing documentation this could make things worse. I'd look for the root cause of that first (training, motivation?), before trying to enforce it with technology.
- lucb1e 5y agoTesting your examples from documentation is actually a great idea. As a security tester, I can't even count the number of times I've gotten API documentation with omitted info like perhaps-trivial-to-them-but-blocker-to-me how to actually authenticate against the API. If they actually had it running somewhere, that sort of thing can't be missing. (Also, the number of times I've asked for API docs in an API-only test and they go "umm, let me task someone to write that real quick"... like, what did you think I was going to work with, balloons and thin air?) When appropriate, I'll definitely be recommending clients to include their documentation examples in testing. But they will probably ignore it like the rest of our non-high-risk advice (today's 'low' findings are tomorrow's stepping stones for ransomware).