Testing
The test suite is broad in directory coverage even where some implementations remain thin.
Audience: Developer
Context: Use this page when deciding where a test belongs, which markers to use, and what existing coverage areas already exist.
Core Explanation
The repository test tree is intentionally split by test depth:
| Area | Purpose |
|---|---|
tests/unit | isolated module and command tests |
tests/integration | multi-component behavior |
tests/e2e | end-to-end workflows |
tests/security | abuse, fuzzing, and regression coverage |
tests/sdk | SDK-specific placeholders and future coverage |
tests/validate_*.py | test-infrastructure validation scripts |
Markers defined in pyproject.toml:
unitintegratione2esecuritypropertyasyncioslow
Practical Usage
Common runs:
pytest -m unit
pytest -m integration
pytest -m security
pytest --cov=caracal --cov-report=term-missing
Infrastructure validation:
python tests/validate_all.py
python tests/simulate_ci.py
Internal Behavior
The suite is organized around repository subsystems, not just test types. There are dedicated areas for:
- CLI
- core authority logic
- DB and migrations
- deployment
- Flow
- MCP
- Merkle
- monitoring
- provider and Redis behavior
This structure reflects the repository layout and should remain aligned with it.
Writing Strategy
Place tests where they match both:
- the behavioral depth
- the subsystem under test
Examples:
- pure validation logic belongs in
tests/unit/core - database-backed authority flows belong in
tests/integration/core - full command workflows belong in
tests/e2e/cli - attack or malformed-input cases belong in
tests/security
Edge Cases And Constraints
- Some e2e and broader coverage areas are still placeholders. Do not assume a directory implies complete behavioral coverage.
- Coverage reporting in the repo is currently configured broadly, but coverage thresholds and actual implemented tests may not yet reflect the long-term target.