Writing
Verified, Not Trusted: Expense Analysis, Part Four
Evals measure reliability with reference keys, code graders, model judges and pass-rate gates. Expense examples show what to check and how structured output expands coverage.
Topic
Acceptance Testing
Exercising a system through its front door as a black box, with attention to where test data comes from and what keeps a scenario true over time.
22 pages
Writing
Verified, Not Trusted: Expense Analysis, Part Three
Evals are acceptance tests for prompts and skills: repeated isolated runs, a pass rate for each check, and a minimum pass rate as the gate. Code grades extracted facts and traces failures.
Writing
Stampede at the Gates
AI generation exposed an already flawed SDLC: faster code review moves the queue to verification, which needs functional acceptance testing at the boundary.
Writing
A Suite the Team Can Keep Trusting
A growing acceptance suite stays trustworthy through explicit scenarios and owned resources. Organize by feature, keep Act visible, review complete comparisons, and retain useful CI failures.
Writing
Arranging Scenarios That Stay True
Random data must preserve the named test scenario. Constrained catalog sampling, immutable builders, deliberate threshold guards, and owned identities keep arrangements correct.
Writing
Assertions That Verify the Whole Outcome
Assertions must compare every required field, count, and effect. Pure asserters separate observations from comparisons and report available failures together with verified expectations.
Writing
From a Green Suite to a Release
A green behavioral suite supports release when its evidence belongs to the build being advanced. Preserve execution receipts and verify deployed capabilities, configuration, and other release duties.
Writing
Functional Acceptance Testing at the Boundary
Functional acceptance tests verify complete outcomes through the assembled system. Correct arrangements, deep assertions, and controlled observations support release confidence.
Writing
Idempotency and Concurrent Requests
Repeated and concurrent order requests must preserve accepted outcomes without duplicate work. Compare canonical request identity, allowed race outcomes, stored facts, and outgoing effects.
Writing
Knowing Which Scenarios a Feature Requires
A green suite needs correct scenarios for established requirements. An obligation map connects each rule to its arrangement, expected outcome, independent observation, and assertion.
Writing
Learning a System Through Its Tests
Acceptance tests let teammates learn requirements and production behavior together. Follow one order through Arrange, the visible Act, independent observations, and complete assertions.
Writing
One Order, Every Obligation
One ordering scenario verifies the caller result, stored lines, fulfillment, and confirmation. Follow its complete C# test, independent observations, and accumulated failure reports.
Writing
Questions About Functional Acceptance Testing at the Boundary
Boundary testing questions concern evidence, isolation, failures, and release confidence. Compare the practice with mocks, TDD, generated checks, formal models, and production observation.
Writing
Refactoring and Adopting Boundary Testing
A stable Domain Facade lets acceptance scenarios and assertions survive internal refactoring. Adapt test support behind unchanged observations and adopt complete boundary tests feature by feature.
Writing
Refusals, Failures, and Work Still Owed
Failure timing determines what an order still owes. Verify refusal, real transaction rollback, post-commit action failures, and accepted effects whose completion could not be recorded.
Writing
Testing Against Real Infrastructure
Real infrastructure must participate when transactions and messages are required outcomes. Provision SQL and broker topology, isolate runs, define observation windows, and verify actual rollback.
Writing
Testing the Service Interface
Interface tests verify wire conversion and exception translation at their owning boundaries. Controller Hooks, middleware checks, and hosted HTTP scenarios complement domain acceptance.
Writing
The Regression Suite the Business Is Paying For
The maintained regression suite protects earlier business behavior as new features change the system. Keep approved expectations, run the required suite, and preserve lessons as permanent scenarios.
Writing
The Test Mediator and the Transport Spy
A Test Mediator and transport spy control delivery and expose observations while production gateways run. Capture email content, script retries, and measure bounded response reads.
Writing
AI-native SDLC
Engineering both gates requires certified code review, functional acceptance tests at the boundary ratified by humans, and verification against the specification.
Writing
Programming to Exceptions, Part 3: Logging and Progress
Structured logs preserve the facts needed to investigate failures. Application Insights custom dimensions make them searchable, while typed progress events track long-running work.
Writing
Mocking? No Thank You!
Test whole features through the public surface so regression tests survive later design changes. Class mocks bind tests to internals; transport doubles cover rare external failures.