Dave Farley’s videos on acceptance testing and deployment pipelines changed how I think about testing. Even though they were posted over four years ago, they remain highly relevant today.
One of the biggest takeaways was Executable Specifications, particularly Dave Farley’s brilliant Four-Layer Approach to writing them, which is most often discussed in the context of acceptance tests. But why stop there? What if we reused the same test cases across unit, integration, and acceptance tests to gain confidence earlier in the pipeline?
TL;DR: Running Executable Specifications at unit and integration levels provides the highest confidence and fastest feedback possible from commit time to test completion, long before acceptance tests run. While acceptance tests offer the highest overall confidence, they are slower and more expensive to run. By validating the same business rules earlier, we catch issues sooner, reduce debugging time, and prevent costly late-stage failures. The trade-off is that maintaining multiple test layers requires effort, but if fast feedback and early confidence matter, this is one of the best investments you can make in your deployment pipeline.
Defining Some Terms
People define these concepts differently, so here’s what I mean when I refer to them:
Unit tests: These test only your own code, using mocks or test doubles when needed. The number of classes tested doesn’t matter, as long as there are no external dependencies (like databases or frameworks), they remain fast.
Integration tests: These test interactions with third-party code, such as databases, libraries, or frameworks.
Acceptance tests: As Dave Farley describes, these verify business rules in a production-like environment.
Executable Specification (ES): There are many definitions for it, but I’ll refer to ES specifically as Dave Farley’s Four-Layer approach, a brilliant model for writing and reusing tests aimed at verifying business rules. It separates test logic from system implementation by using clear test cases, a readable abstraction (DSL), a protocol for interacting with the system (e.g., HTTP, direct calls), and the actual system under test. This structure allows tests to be reused, targeting different entry points of a system through different protocols, but also targeting different systems, which we will explore later.
Deployment pipeline: a pipeline that goes from commit to releasable code, as Dave Farley describes. In this post, I’ll focus on the commit phase, where tests must be fast but reliable and provide high confidence for the release candidate.
Why Limit ES to Acceptance Tests?
From what I’ve seen, Dave Farley primarily discusses ES in the context of acceptance tests. While he may have addressed their use at other levels, I haven’t come across it and I believe that limiting ES to acceptance tests is a missed opportunity. Acceptance tests provide the highest confidence in business rules before deployment, but they happen much later in the pipeline. The same principles can be applied earlier for faster feedback and the highest confidence possible from commit time to test completion.
We can apply ES in integration tests by spinning up test containers for databases, APIs, and external services while running a test version of our system locally within the test environment. This setup allows us to validate functional requirements without needing a full production-like environment. These tests provide the best confidence we can get for business rules without a deployment, often completing in just a few minutes. The protocol layer can even be reused as you interface with your system just like you would with a live environment.
Even when ES is applied in integration tests, applying it to unit tests is valuable. Here, we initialize the system under test much like we would in a dependency injection framework, but we replace external dependencies with test doubles (mocks, fakes).In unit tests, the protocol layer, instead of HTTP requests to WireMock, sets expectations on mocks. Instead of interacting with the system via an API request, we directly invoke the responsible methods.These tests provide the best possible confidence in business rules at unit test speed, ensuring that even before integration tests run, we already have high confidence in our changes. When paired with lightweight integration tests, this combination offers the highest confidence possible in seconds, making it a perfect candidate for fast feedback before committing changes.
Conclusion
Running Executable Specifications at the integration or even unit test level provides the best possible confidence in business rules at the earliest stage of the pipeline.
Unit tests complete almost immediately after commit, offering the fastest feedback. Integration tests take longer due to container and service initialization but provide greater confidence. Acceptance tests take the longest, as they require full deployment, but they offer the highest overall confidence before release.
By validating the same business rules earlier in the pipeline, we catch issues sooner, reduce debugging time, and prevent costly failures later on. This ensures that feedback is received at the fastest possible stage, allowing developers to make corrections before issues escalate.
Maintaining multiple test layers requires effort, and if ES in acceptance tests is already efficient, running them at lower levels might not add significant value. But for teams that prioritize fast feedback and early confidence, bringing these tests earlier in the pipeline is one of the smartest investments you can make in your deployment pipeline.
What Do You Think? Do you use Executable Specifications in your testing strategy? At what layer do you run them, and why?
I’d love to hear your thoughts.
Komentarze