3 ms·

Aren't many of those tests? Why define them as linters?

by ukuina 9mo ago

bobjordan 9mo ago
Great question, I let Claude help answer this...see below: The key differences are: 1. Static vs Runtime Analysis Linters use AST parsing to analyze code structure without executing it. Tests verify actual runtime behavior. Example from our datetime_linter: tree = ast.parse(file_path.read_text()) for node in ast.walk(tree): if isinstance(node, ast.Import): if alias.name == "datetime": # Violation: should use pendulum This catches import datetime syntactically. A test would need to actually execute code and observe wrong datetime behavior. 2. Feedback Loop Speed - Linters: Run in pre-commit hooks. Agent writes code → instant feedback → fix → iterate in seconds - Tests: Run in CI. Commit → push → wait minutes/hours → fix in next session For AI agents, this is critical. A linter that blocks commit keeps them on track immediately rather than discovering violations after a test run. 3. Structural Violations For example, our `fastapi_security_linter` catches things like "route missing TenantRouter decorator". These are structural violations - "you forgot to add X" - not "X doesn't work correctly." Tests verify the behavior of X when it exists. 4. Coverage Exhaustiveness Linters scan all code paths structurally. Tests only cover scenarios you explicitly write. Our org_scope_linter catches every unscoped platform query across the entire codebase in one pass. Testing that would require writing a test for each query. 5. The Hybrid Value We actually have both. The linter catches "you forgot the security decorator" instantly. The test (test_fastapi_authorization.py) verifies "the security decorator actually blocks unauthorized users at runtime." Different failure modes, complementary protections. Think of it like: linters are compile-time checks, tests are runtime checks. TypeScript catches string + number at compile time; you don't write a test for that.

Aren't many of those tests? Why define them as linters?

by ukuina 9mo ago

Aren't many of those tests? Why define them as linters?

Great question, I let Claude help answer this...see below: The key differences are: 1. Static vs Runtime Analysis Linters use AST parsing to analyze code structure without executing it. Tests verify actual runtime behavior. Example from our datetime_linter: tree = ast.parse(file_path.read_text()) for node in ast.walk(tree): if isinstance(node, ast.Import): if alias.name == "datetime": # Violation: should use pendulum This catches import datetime syntactically. A test would need to actually execute code and observe wrong datetime behavior. 2. Feedback Loop Speed - Linters: Run in pre-commit hooks. Agent writes code → instant feedback → fix → iterate in seconds - Tests: Run in CI. Commit → push → wait minutes/hours → fix in next session For AI agents, this is critical. A linter that blocks commit keeps them on track immediately rather than discovering violations after a test run. 3. Structural Violations For example, our `fastapi_security_linter` catches things like "route missing TenantRouter decorator". These are structural violations - "you forgot to add X" - not "X doesn't work correctly." Tests verify the behavior of X when it exists. 4. Coverage Exhaustiveness Linters scan all code paths structurally. Tests only cover scenarios you explicitly write. Our org_scope_linter catches every unscoped platform query across the entire codebase in one pass. Testing that would require writing a test for each query. 5. The Hybrid Value We actually have both. The linter catches "you forgot the security decorator" instantly. The test (test_fastapi_authorization.py) verifies "the security decorator actually blocks unauthorized users at runtime." Different failure modes, complementary protections. Think of it like: linters are compile-time checks, tests are runtime checks. TypeScript catches string + number at compile time; you don't write a test for that.