@crossplatformai/skills
Version:
Reusable Agent Skills for CrossPlatform.ai projects.
124 lines (100 loc) • 8.12 kB
Markdown
---
name: test-layer-fit
description: Choose the lowest meaningful test layer and design deterministic test harness ownership. Use for unit, component, adapter-contract, integration, or E2E placement; suite-load or CI-only failures; timeout, retry, sleep, serialization, warmup, timing-workaround, or suite/config-splitting proposals; and browser, Electron, mobile, subprocess, server, port, process, isolation, readiness, diagnostic, or cleanup ownership. Covers cross-platform proof boundaries and integration-suite reachability. Not for test-framework selection, coverage thresholds, general mocking or assertion style, visual verification, host-OS path or quoting mechanics, or Post.Build.Ship gate and receipt policy.
---
# Test Layer Fit
Move the assertion before moving the timeout. Treat a failure that appears only under suite load or
in CI as evidence about layer, ownership, or synchronization until the actual boundary proves
otherwise. A longer timeout does not prove product correctness.
## Choose The Lowest Meaningful Boundary
Name the behavior contract without referring to the current test, then place its proof on this
boundary ladder:
1. **Deterministic unit or component tests** — prove logic, timing, ordering, loading, progress, and
transient state with controlled time and explicit collaborators.
2. **Adapter contract tests** — prove arguments, environment, scheduling, lifecycle decisions, and
translation at a process, network, storage, or platform adapter without requiring the external
system to perform the behavior.
3. **Real integration tests** — prove subprocess, filesystem, network, database, or operating-system
semantics against the real boundary in an isolated fixture.
4. **Minimal client E2E tests** — prove UI, IPC, platform, and external wiring through the smallest
user-visible scenario that crosses the required boundary.
Start at the first rung that can fail when the contract is broken. Climb only when the lower rung
cannot observe part of the contract. Keep detailed lower-layer proof when an E2E smoke is also
needed; one does not substitute for the other.
Use a humble adapter: keep environment-coupled code thin and delegate decisions to deterministic
code. When a test is slow only in a full suite, use logs or traces to separate product latency from
cold compilation, worker contention, background throttling, resource collisions, and machine
scheduling before changing the test or runtime.
Do not add a retry, sleep, forced serialization, warmup, larger timeout, harness timing mechanism,
or production behavior merely to preserve a misplaced assertion. Require an independent runtime or
product contract and prove that reason directly.
## Design The Harness Around Ownership
### Artifact And Resource Ownership
- Test the deployable artifact when production build or serving behavior is the contract. Use a
development server only when development behavior is itself under test.
- Classify every resource as **exclusive** (the harness must start and own it), **borrowed or reused**
(the harness may use a healthy existing resource but owns only resources it starts), or **external**
(the harness verifies it but never starts or stops it).
- Resolve dependent external targets atomically. A web and API pair, for example, must be supplied
together or rejected rather than mixing an external half with a local half.
- Preflight required dependencies before starting owned processes. Wait for observable readiness at
the real boundary, and fail early if an owned process exits; do not synchronize with sleeps.
### Isolation And Observation
- Isolate mutable state per test. Share only prerequisites that are explicitly suite-owned and safe
to reuse.
- Install observers before the action that can emit a transient result. Await the observed event or
state instead of polling after it may already have disappeared.
- Keep the smallest surface-specific wiring proof and put shared behavior at a deterministic shared
layer.
### Processes, Cleanup, And Diagnostics
- Make cleanup idempotent across success, failure, partial startup, and termination signals.
- Terminate every owned process tree, then verify exit. Never stop borrowed, reused, or external
resources.
- Preserve the primary test or startup failure when cleanup also fails. Report cleanup as secondary
evidence; surface it as primary only when no earlier failure exists.
- Drain child output streams so processes cannot block. Retain only bounded, allowlisted,
secret-safe diagnostics needed to attribute lifecycle and readiness failures.
### Timeouts, Bail, And Suite Splits
- Place each timeout at the operation that can block: process start, readiness probe, subprocess,
test body, hook, or cleanup. Treat the value as a failure ceiling, not synchronization.
- Use bail only when a shared-environment failure would make later results cascading and
misleading. Bail never replaces per-test isolation.
- When real integration work needs separate configuration or ceilings, split it from deterministic
unit tests. Verify that the integration suite remains reachable from package scripts and every
authoritative QA path that is supposed to run it.
## Preserve Cross-Platform Proof
- Prove shared behavior once at the shared deterministic layer.
- Prove web, desktop, and mobile wiring on each surface. Evidence from one client cannot establish
another client's UI, IPC, platform adapter, or external wiring.
- Launch scripted Electron E2E with Playwright-owned applications, such as `_electron.launch`, and
isolated user-data directories. Reserve stable CDP attachment for interactive development and
requested visible-app verification; use `desktop-electron-dev-attach` for that workflow.
- Default native mobile E2E to Maestro on supported targets, with repository-owned flows reusable
locally and by configured CI. For Expo apps, keep the flows with the app's EAS project so local and
configured EAS Workflows use the same assets.
- When a repository has no Maestro flow or command, keep guidance at the contract level, report the
absence, and confirm the minimal Maestro approach instead of inventing CLI or flow syntax or
inferring mobile proof from web or desktop.
- Route host-OS process and path mechanics, including separators, shell quoting, signals, and
Windows process-tree behavior, to the Post.Build.Ship `host-portability` reference when that
workflow is active. Do not duplicate its platform mechanics here.
- Use `playwright-visual-verification` for screenshots, accessibility snapshots, computed styles,
and visual-only proof. Visual verification does not decide test-layer placement or harness
lifecycle ownership.
## Keep Workflow Enforcement Separate
Use this skill for test and harness reasoning. When Post.Build.Ship is active, its
`quality-assurance` reference remains authoritative for blocking failures, approval chronology,
receipts, candidate identity, QA reachability, final-QA widening, and specification-gaming refusal.
This skill cannot waive or reinterpret a required check.
## Evidence
- In CrossPlatform.ai commit `8878711`, a preview-driven update-toast sequence hit a 20-second E2E
ceiling under suite load. Detailed checking, available, downloading, ready, complete, and dismissed
ordering moved to a deterministic shared component test while E2E retained a narrow wiring smoke.
- Commit `833e9ae` separated real Git subprocess behavior into an integration suite with its own
configuration and narrowed Electron scenarios so unit and E2E failures stayed attributable.
- Commit `7d1734a` made web E2E build the production artifact, classified exclusive, reused, and
external target ownership, required paired external web/API targets, preflighted dependencies,
observed readiness, and verified owned-process cleanup.
- Commit `f6c736d` replaced scripted Electron CDP ownership with Playwright-owned applications,
isolated user data, exclusive local API startup, deterministic scheduling tests, and bounded
lifecycle diagnostics that preserve the primary failure.