QA Testing Strategy (Jan 2026)

Risk-based quality engineering strategy for modern software delivery.

Core references: curated links in data/sources.json (SLOs/error budgets, contracts, E2E, OpenTelemetry). Start with references/operational-playbook.md for a compact, navigable overview.

Scope

• Create or update a risk-based test strategy (what to test, where, and why)

• Define quality gates and release criteria (merge vs deploy)

• Select the smallest effective layer (unit → integration → contract → E2E)

• Make failures diagnosable (artifacts, logs/traces, ownership)

• Operationalize reliability (flake SLO, quarantines, suite budgets)

Use Instead

Need

Skill

Debug failing tests or incidents

qa-debugging

Test LLM agents/personas

qa-agent-testing

Perform security audit/threat model

software-security-appsec

Design CI/CD pipelines and infra

ops-devops-platform

Quick Reference

Test Type

Goal

Typical Use

Unit

Prove logic and invariants fast

Pure functions, core business rules

Component

Validate UI behavior in isolation

UI components and state transitions

Integration

Validate boundaries with real deps

API + DB, queues, external adapters

Contract

Prevent breaking changes cross-team

OpenAPI/AsyncAPI/JSON Schema/Protobuf

E2E

Validate critical user journeys

1–2 “money paths” per product area

Performance

Enforce budgets and capacity

Load, stress, soak, regression trends

Visual

Catch UI regressions

Layout/visual diffs on stable pages

Accessibility

Automate WCAG checks

axe smoke + targeted manual audits

Security

Catch common web vulns early

DAST smoke + critical checks in CI

Default Workflow

• Clarify scope and risk: critical journeys, failure modes, and non-functional risks (latency, data loss, auth).

• Define quality signals: SLOs/error budgets, contract/schema checks, and what blocks merge vs blocks deploy.

• Choose the smallest effective layer (unit → integration → contract → E2E).

• Make failures diagnosable: artifacts + correlation IDs (logs/traces/screenshots), clear ownership, deflake runbook.

• Operationalize: flake SLO, quarantine with expiry, suite budgets (PR gate vs scheduled), dashboards.

Test Pyramid

/\

          /E2E\          5-10% - Critical journeys

         /------\

        /Integr. \       15-25% - API, DB, queues

       /----------\

      /Component \       20-30% - UI modules

     /------------\

    /   Unit      \      40-60% - Logic and invariants

   /--------------\

Decision Tree: Test Strategy

Need to test: [Feature Type]

    │

    ├─ Pure business logic/invariants? → Unit tests (mock boundaries)

    │

    ├─ UI component/state transitions? → Component tests

    │   └─ Cross-page user journey? → E2E tests

    │

    ├─ API Endpoint?

    │   ├─ Single service boundary? → Integration tests (real DB/deps)

    │   └─ Cross-service compatibility? → Contract tests (schema/versioning)

    │

    ├─ Event-driven/API schema evolution? → Contract + backward-compat tests

    │

    └─ Performance-critical? → k6 load testing

Core QA Principles

Definition of Done

• Strategy is risk-based: critical journeys + failure modes explicit

• Test portfolio is layered: fast checks catch most defects

• CI is economical: fast pre-merge gates, heavy suites scheduled

• Failures are diagnosable: actionable artifacts (logs/trace/screenshots)

• Flakes managed with SLO and deflake runbook

Shift-Left Gates (Pre-Merge)

• Contracts: OpenAPI/AsyncAPI/JSON Schema validation

• Static checks: lint, typecheck, secret scanning

• Fast tests: unit + key integration (avoid full E2E as PR gate)

Shift-Right (Post-Deploy)

• Synthetic checks for critical paths (monitoring-as-tests)

• Canary analysis: compare SLO signals and key metrics before ramping

• Feature flags for safe rollouts and fast rollback

• Convert incidents into regression tests (prefer lower layers first)

CI Economics

Budget

Target

PR gate

p50 ≤ 10 min, p95 ≤ 20 min

Mainline health

≥ 99% green builds/day

Flake Management

• Define: test fails without product change, passes on rerun

• Track weekly: flaky_failures / total_test_executions (where flaky_failure = fail_then_pass_on_rerun)

• SLO: Suite flake rate ≤ 1% weekly

• Quarantine policy with owner and expiry

• Use the deflake runbook: template-flaky-test-triage-deflake-runbook.md

Common Patterns

AAA Pattern

it('should apply discount', () => {

  // Arrange

  const order = { total: 150 };

  // Act

  const result = calculateDiscount(order);

  // Assert

  expect(result.discount).toBe(15);

});

Page Object Model (E2E)

class LoginPage {

  async login(email: string, password: string) {

    await this.page.fill('[data-testid="email"]', email);

    await this.page.fill('[data-testid="password"]', password);

    await this.page.click('[data-testid="submit"]');

  }

}

Anti-Patterns

Anti-Pattern

Problem

Solution

Testing implementation

Breaks on refactor

Test behavior

Shared mutable state

Flaky tests

Isolate test data

sleep() in tests

Slow, unreliable

Use proper waits

Everything E2E

Slow, expensive

Use test pyramid

Ignoring flaky tests

False confidence

Fix or quarantine

Do / Avoid

Do

• Write tests against stable contracts and user-visible behavior

• Treat flaky tests as P1 reliability work

• Make "how to debug this failure" part of every suite

Avoid

• "Everything E2E" as default

• Sleeps/time-based waits (use event-based)

• Coverage % as primary quality KPI

Feature Matrix vs Test Matrix Gate (Release Blocking)

Before release, run a coverage audit that maps product features/backlog IDs to direct test evidence.

Gate Rules

• Every release-scoped feature must map to at least one direct automated test, or an explicit waiver with owner/date.

• Evidence must include file path and test identifier (suite/spec/case).

• "Covered indirectly" is not accepted without written rationale and risk acknowledgment.

• If critical features have no direct evidence, release is blocked.

Minimal Audit Output

• feature/backlog id

• coverage status (direct, indirect, none)

• evidence reference

• risk level

• owner and due date for gaps

Resources

Resource

Purpose

comprehensive-testing-guide.md

End-to-end playbook across layers

operational-playbook.md

Testing pyramid, BDD, CI gates

shift-left-testing.md

Contract-first, BDD, continuous testing

test-automation-patterns.md

Reliable patterns and anti-patterns

playwright-webapp-testing.md

Playwright patterns

chaos-resilience-testing.md

Chaos engineering

observability-driven-testing.md

OpenTelemetry, trace-based

contract-testing-2026.md

Pact, Specmatic

synthetic-test-data.md

Privacy-safe, ephemeral test data

test-environment-management.md

Environment provisioning and lifecycle

quality-metrics-dashboard.md

Quality metrics and dashboards

compliance-testing.md

SOC2, HIPAA, GDPR, PCI-DSS testing

feature-matrix-vs-test-matrix-gate.md

Release-blocking feature-to-test coverage audit

Templates

Template

Purpose

template-test-case-design.md

Given/When/Then and test oracles

test-strategy-template.md

Risk-based strategy

template-flaky-test-triage.md

Flake triage runbook

template-jest-vitest.md

Unit test patterns

template-api-integration.md

API + DB integration tests

template-playwright.md

Playwright E2E

template-visual-testing.md

Visual regression testing

template-k6-load-testing.md

k6 performance

automation-pipeline-template.md

CI stages, budgets, gates

template-cucumber-gherkin.md

BDD feature files and steps

template-release-coverage-audit.md

Feature matrix vs test matrix release audit

Data

File

Purpose

sources.json

External references

Related Skills

• qa-debugging — Debugging failing tests

• qa-agent-testing — Testing AI agents

• software-backend — API patterns to test

• ops-devops-platform — CI/CD pipelines

Ops Gate: Release-Safe Verification Sequence

Use this sequence for feature branches that touch user flows, pricing, localization, or analytics.

# 1) Static checks

npm run lint

npm run typecheck

# 2) Fast correctness

npm run test:unit

# 3) Critical path checks

npm run test:e2e -- --grep "@critical"

# 4) Instrumentation gate (if configured)

npm run test:analytics-gate

# 5) Production build

npm run build

If a Gate Fails

• Capture exact failing command and first error line.

• Classify: environment issue, baseline known failure, or regression.

• Re-run only the failed gate once after fix.

• Do not continue to later gates while earlier required gates are red.

Agent Output Contract for QA Handoff

Always report:

• commands run,

• pass/fail per gate,

• whether failures are pre-existing or introduced,

• next blocking action.

Fact-Checking

• Use web search/web fetch to verify current external facts, versions, pricing, deadlines, regulations, or platform behavior before final answers.

• Prefer primary sources; report source links and dates for volatile information.

• If web access is unavailable, state the limitation and mark guidance as unverified.