30. Use Patch for mocking and testing in Elixir

30. Use Patch for mocking and testing in Elixir

Date

2025-06-16

Status

Accepted

Context

The Bartendie project uses a sophisticated CQRS/Event Sourcing architecture built with the Commanded framework in Elixir. As we develop and test complex domain logic, aggregates, projections, and event handlers, we need robust testing capabilities that can handle the unique challenges of this architecture.

Key testing challenges identified:

• Complex Domain Logic: Testing CQRS aggregates with complex business rules and state transitions
• Event Sourcing Patterns: Mocking external dependencies while testing event handlers and projections
• Private Function Testing: Need to test private functions in aggregates without exposing them publicly
• Process Communication: Testing GenServer interactions and message passing in the CQRS system
• External Dependencies: Mocking HTTP clients, database calls, and third-party services
• Concurrent Testing: Ensuring mocks work correctly in concurrent test scenarios

Technical constraints:

• Must work seamlessly with ExUnit and the existing test infrastructure
• Should integrate well with Commanded’s testing patterns
• Must support both unit and integration testing approaches
• Should not require significant changes to production code
• Must be maintainable and have good developer experience

Current testing gaps:

• No standardized mocking/stubbing approach across the team
• Difficulty testing private functions in domain aggregates
• Challenges mocking external services in integration tests
• Limited ability to test error scenarios and edge cases
• No consistent approach for testing process interactions

Stakeholders:

• Development team writing and maintaining tests
• QA team validating test coverage and quality
• DevOps team ensuring tests run reliably in CI/CD
• Future developers who need to understand and extend tests

Decision

We will adopt Patch as our primary mocking and testing library for the Elixir codebase.

Options Considered

1. Patch - Ergonomic mocking library with unique “Super Powers”

   • ✅ Pros: Works with both local and remote calls, can patch private functions, excellent process support
   • ✅ Easy-to-use API with sensible defaults, comprehensive documentation
   • ✅ First-class support for testing private functions without changing visibility
   • ✅ Unique ability to patch functions that always resolve to the patch (local and remote calls)
   • ❌ Cons: Not compatible with async: true tests, relatively newer library

2. Mox - Explicit contracts and behavior-based mocking

   • ✅ Pros: Excellent for behavior verification, works with async: true, José Valim endorsed
   • ✅ Explicit contracts prevent interface drift, good for testing boundaries
   • ❌ Cons: Requires defining behaviors/contracts, more setup overhead
   • ❌ Cannot mock private functions, limited to module boundaries

3. Mock - Simple mocking library based on meck

   • ✅ Pros: Simple API, lightweight, works with existing code
   • ❌ Cons: Limited functionality, doesn’t handle all edge cases
   • ❌ No support for private function testing, basic process support

4. Mimic - Copy-based mocking approach

   • ✅ Pros: Works with async: true, good performance
   • ❌ Cons: Limited functionality compared to Patch, less mature ecosystem
   • ❌ Cannot patch private functions, requires more setup

5. Bypass - HTTP mocking specifically

   • ✅ Pros: Excellent for HTTP testing, realistic HTTP server simulation
   • ❌ Cons: Limited to HTTP testing only, doesn’t solve general mocking needs

Rationale

Patch was selected because it uniquely addresses our CQRS/Event Sourcing testing challenges:

CQRS/Event Sourcing Advantages:

• Private Function Testing: Can test complex aggregate business logic without exposing private functions
• Process Testing: Excellent support for testing GenServer interactions and message passing
• Event Handler Testing: Easy to mock external dependencies in event handlers and projections
• Command Testing: Can mock repository calls and external services in command handlers

Technical Advantages:

• Super Powers: Unique ability to patch functions that work for both local and remote calls
• Ergonomic API: Simple, intuitive interface that reduces test boilerplate
• Comprehensive Features: 11 core functions, 10 assertions, 7 value builders
• Excellent Documentation: Comprehensive guides, examples, and cheatsheets

Developer Experience:

• Easy Setup: Simple use Patch in test modules
• Rich Assertions: Built-in assertions for call verification and history
• Value Builders: Sophisticated mock return values (cycles, sequences, callables)
• Debug Support: Built-in debugging capabilities for troubleshooting tests

Implementation Plan

1. ✅ Add Patch dependency to mix.exs (dev/test only)
2. ✅ Update .formatter.exs to include Patch formatting rules
3. 🔄 Create testing guidelines and best practices documentation
4. 🔄 Update existing tests to use Patch where appropriate
5. 🔄 Train team on Patch usage and patterns
6. 🔄 Establish conventions for mocking in CQRS/Event Sourcing contexts

Consequences

Positive Outcomes

Enhanced Testing Capabilities:

• Private Function Testing: Can thoroughly test aggregate business logic without compromising encapsulation
• Comprehensive Mocking: Easy mocking of external dependencies, HTTP clients, and database calls
• Process Testing: Robust testing of GenServer interactions and message passing
• Error Scenario Testing: Easy simulation of failure conditions and edge cases

Improved Developer Experience:

• Intuitive API: Simple, readable test code with minimal boilerplate
• Rich Assertions: Built-in assertions for verifying call patterns and history
• Excellent Documentation: Comprehensive guides and examples for common patterns
• Debug Support: Built-in debugging tools for troubleshooting test issues

CQRS/Event Sourcing Benefits:

• Aggregate Testing: Thorough testing of complex domain logic and state transitions
• Event Handler Testing: Easy mocking of external services in event projections
• Command Testing: Simplified testing of command handlers with mocked dependencies
• Integration Testing: Better control over external dependencies in integration scenarios

Code Quality Improvements:

• Higher Test Coverage: Ability to test previously hard-to-test private functions
• Better Test Isolation: Proper mocking prevents tests from affecting each other
• Faster Test Execution: Mocked dependencies eliminate slow external calls
• More Reliable Tests: Consistent behavior regardless of external service availability

Potential Challenges

Technical Limitations:

• No Async Support: Cannot use async: true in tests that use Patch
• Global State Changes: Patch alters global execution environment through recompilation
• Test Execution Time: May slightly increase test execution time due to recompilation
• Memory Usage: Additional memory overhead from maintaining mock state

Learning Curve:

• New API: Team needs to learn Patch-specific functions and patterns
• Best Practices: Need to establish conventions for when and how to use mocking
• CQRS Patterns: Understanding how to effectively mock in CQRS/Event Sourcing contexts
• Debugging: Learning to debug issues with mocked vs real implementations

Maintenance Considerations:

• Mock Maintenance: Mocks need to be updated when interfaces change
• Test Brittleness: Over-mocking can make tests brittle and hard to maintain
• Documentation: Need to maintain examples and guidelines for effective usage
• Version Updates: Need to track Patch updates and potential breaking changes

Risk Mitigation

Technical Risks:

• Performance Monitoring: Track test execution times and optimize where needed
• Memory Monitoring: Monitor test memory usage and optimize mock usage patterns
• Async Alternative: Use Mox for scenarios where async testing is critical
• Gradual Adoption: Introduce Patch incrementally to minimize disruption

Team Adoption:

• Training Sessions: Provide comprehensive training on Patch usage and best practices
• Documentation: Create project-specific guidelines for mocking in CQRS contexts
• Code Reviews: Ensure proper mock usage through peer review
• Examples: Maintain a library of common mocking patterns for the team

Quality Assurance:

• Mock Validation: Regular review of mock accuracy against real implementations
• Integration Testing: Maintain integration tests with real dependencies
• Test Coverage: Monitor test coverage to ensure mocks don’t hide untested code
• Refactoring Support: Establish patterns for updating mocks during refactoring

Future Considerations

Potential Enhancements:

• Custom Matchers: Develop project-specific matchers for CQRS patterns
• Test Helpers: Create helper functions for common mocking scenarios
• CI Integration: Optimize CI pipeline for tests using Patch
• Performance Optimization: Investigate ways to minimize recompilation overhead

Alternative Approaches:

• Hybrid Strategy: Use Mox for boundary testing, Patch for internal testing
• Behavior Testing: Complement mocking with property-based testing where appropriate
• Contract Testing: Consider contract testing for external service interactions
• Test Doubles: Evaluate when to use mocks vs stubs vs fakes

Monitoring and Review:

• Quarterly Reviews: Assess Patch effectiveness and team satisfaction
• Performance Metrics: Track test execution time and reliability
• Coverage Analysis: Monitor test coverage trends and quality
• Team Feedback: Regular feedback sessions on developer experience

Success Metrics

Quantitative Metrics:

• Increased test coverage, especially for private functions
• Reduced test execution time (excluding external calls)
• Decreased test flakiness due to external dependencies
• Faster development cycle with reliable test feedback

Qualitative Metrics:

• Improved developer confidence in test suite
• Easier testing of complex CQRS scenarios
• Better test maintainability and readability
• Enhanced ability to test error conditions and edge cases