Design Verification and Test

A requirement-to-evidence workflow for credible qualification, acceptance, failure handling, and test records.

Mechanical and Manufacturing · Verification

Verification asks whether the design meets its requirements. Validation asks whether the resulting system serves its intended use in the real operating context. A test is useful only when it supports a defined decision.

Build a requirement-to-evidence map

Each requirement should identify:

  • a unique requirement ID,
  • a measurable condition and limit,
  • operating state and environmental boundary conditions,
  • verification method,
  • test level and article,
  • instrumentation and relevant uncertainty,
  • pass/fail logic,
  • evidence location,
  • deviation status and disposition.

Common verification methods are analysis, inspection, demonstration, and test. Select the method that produces sufficient evidence at acceptable risk. Do not use a demonstration when quantitative performance is required.

Carry uncertainty into the decision

For a measurement model $y = f(x_1,\ldots,x_n)$ with independent inputs, a first-order combined standard uncertainty is:

\[u_c(y) = \sqrt{ \sum_{i=1}^{n} \left( \frac{\partial f}{\partial x_i}u(x_i) \right)^2 }\]

Include covariance terms when inputs are correlated. Expanded uncertainty is:

\[U = k u_c\]

Three independent contributions of 0.20 mm, 0.10 mm, and 0.05 mm give $u_c \approx 0.23$ mm. With a documented coverage factor $k=2$, $U \approx 0.46$ mm.

For an upper specification limit $L$, a simple conservative guard-band rule is:

\[y + U \leq L\]

The correct decision rule depends on the contract, risk allocation, distribution assumptions, and measurement capability. Always report the result, uncertainty, units, coverage factor, and method rather than stating only pass or fail.

Illustrative traceability record

The table below shows the minimum useful connection between requirement and evidence. Values are illustrative.

Requirement Verification definition Result Guarded decision Evidence
PWR-014: logic rail remains 3.201 to 3.399 V at the load test at minimum and maximum input, idle and peak load, temperature extremes 3.287 V minimum, $U=0.012$ V $3.287-0.012=3.275$ V, pass raw waveform, DMM log, calibration, configuration
THM-006: enclosure surface stays at or below 70.0 °C test at maximum continuous load after thermal equilibrium 66.8 °C, $U=1.4$ °C $66.8+1.4=68.2$ °C, pass chamber log, sensor map, thermal images
MASS-002: assembled unit mass stays at or below 1.500 kg inspection with calibrated balance and production configuration 1.472 kg, $U=0.003$ kg $1.472+0.003=1.475$ kg, pass balance export, configuration photo, serial record

The voltage row still needs evidence from every required input, load, and temperature corner. A single passing point does not close a multi-condition requirement.

Before execution, the traceability record should already name the article, configuration, method, limits, uncertainty target, raw-data location, and deviation process. Filling those fields after the result is known invites accidental result-dependent criteria.

Establish test article pedigree

Distinguish between:

  • breadboard,
  • prototype,
  • engineering unit,
  • qualification unit,
  • production-representative unit,
  • acceptance unit.

Evidence transfers between articles only when hardware, software, materials, processes, and operating conditions are equivalent or the differences are explicitly justified. Record serial number, revision, firmware, calibration state, rework, and prior exposure.

Design the test around the decision

  1. State the requirement and pass/fail rule.
  2. Map service loads and boundary conditions into the test setup.
  3. Identify fixture influence and predict its important modes or losses.
  4. Select sensors from range, bandwidth, loading, environment, and calibration needs.
  5. Predict the expected data before running the test.
  6. Define safe abort limits and who has authority to stop the test.
  7. Freeze configuration and preserve raw data.
  8. Inspect the article and setup before and after the run.

Record what would invalidate the result. Examples include a fixture resonance inside the control band, a clipped sensor, lost time synchronization, or an unplanned configuration change.

Qualification and acceptance are different

Qualification demonstrates that a design has adequate performance and margin using representative hardware, processes, and environments. Acceptance screens individual units for workmanship and functional conformity.

Combining the two without a defined philosophy can consume production life by over-testing every unit, or leave design margin unproven by under-testing the qualification article.

Plan sample size and confidence

One passing article establishes that one article passed. A population claim needs a sampling model tied to the claim, variability, acceptable risk, and unit selection.

For independent pass/fail trials with zero observed failures, a one-sided lower reliability bound is:

\[R_L = \alpha^{1/n}\]

where $n$ is the number of trials and $1-\alpha$ is the confidence level. Ten successes with zero failures give only:

\[R_L = 0.05^{1/10} \approx 0.741\]

at 95% confidence. To demonstrate at least 90% reliability at the same confidence with zero failures requires:

\[n \geq \frac{\ln(0.05)}{\ln(0.90)} \approx 28.4\]

so at least 29 valid independent trials are needed under this model.

This calculation does not fix an unrepresentative sample, dependent trials, changing configurations, censored life data, or an undefined mission duration. Continuous-life claims require an explicit failure-time model and accumulated test exposure. Stratify samples across relevant lots, suppliers, processes, and configurations when those factors are part of the target population. Define before testing how failures, invalid runs, rework, and regression units affect the inference.

Environmental test traps

  • Fixture resonance can amplify or hide vibration response.
  • Chamber air temperature is not necessarily part temperature.
  • Thermocouple attachment can miss the critical gradient or alter it.
  • Pressure and leak fixtures add compliance and dead volume.
  • Cable restraints can change structural modes and heat flow.
  • An accelerated test can introduce a failure mechanism absent from service.
  • Repeated testing consumes fatigue and thermal-cycle life.

Measure the boundary conditions the article actually experiences. Setpoint values alone are not evidence.

Handle failures without destroying evidence

  1. Stop safely and preserve setup, logs, and raw data.
  2. Confirm instrumentation, clocks, and configuration.
  3. Record exact failure time, operating state, and preceding events.
  4. Quarantine the article from undocumented rework or power cycling.
  5. Inspect nondestructively before teardown when useful.
  6. Separate the observed symptom, failure mechanism, and root cause.
  7. Update the design, analysis, requirement, or process as the evidence requires.
  8. Define regression scope and the evidence needed to close the failure.

Evidence package

test/
  plan.md
  configuration/
  setup/
  calibration/
  raw-data/
  analysis/
  photos/
  report.md
  deviations.md

The report should link to immutable raw evidence and record analysis versions. A selected plot without provenance is not a complete test record.

Measurement and Instrumentation, PCB Design and Bring-Up, and UUV Navigation and State Estimation.

Sources