2.4: Verification and Validation

Verification Strategy (SWE.4-5)

V-Model Test Levels

The following diagram maps the V-Model test levels (unit, integration, system, acceptance) to their corresponding ASPICE processes (SWE.4-6), showing how each left-side development phase pairs with its right-side verification activity.

Unit Testing (SWE.4)

Test Environment Setup

Tool: VectorCAST/C++ (certified for ISO 26262)

Alternative Unit Testing Tools: VectorCAST is used in this example, but alternatives include Tessy (Razorcat), Cantata (QA Systems), and Parasoft C/C++test. All provide MC/DC coverage analysis required for ASIL-B. Tool selection often depends on existing OEM/Tier-1 toolchain agreements.

Coverage Metrics (ASIL-B requirements):

• Statement coverage: ≥100%
• Branch coverage: ≥100%
• MC/DC coverage: ≥85%

Example Unit Test Report:

VectorCAST Unit Test Report
Project: ACC_Control
Date: 2025-08-15
ASPICE: SWE.4 BP3 (Test software units)

Function: ACC_PID_Controller()
  Total Statements: 28
  Covered Statements: 28 (100%)

  Total Branches: 12
  Covered Branches: 12 (100%)

  MC/DC Coverage: 89% (11/12 decision points)
  [PASS] PASS (ASIL-B requires ≥85%)

Uncovered Scenarios:
  None - Full coverage achieved

Test Cases Executed: 15
  [PASS] Passed: 15
  [FAIL] Failed: 0

Execution Time: 2.3 seconds

Integration Testing (SWE.5)

HIL (Hardware-in-the-Loop) Test Bench

Setup: dSPACE SCALEXIO Real-Time Simulator

Configuration:

HIL Test Bench: dSPACE SCALEXIO
├── Real ECU: Infineon AURIX TC397 (production hardware)
├── Simulated Environment:
│   ├── Vehicle Dynamics Model (IPG CarMaker)
│   ├── Radar Sensor Model (77 GHz, Continental ARS540)
│   ├── Camera Sensor Model (Mobileye EyeQ4)
│   ├── CAN Bus Simulator (vehicle network)
│   └── Traffic Scenarios (500+ test cases)
└── Measurement & Control:
    ├── dSPACE ControlDesk (test execution)
    ├── TRACE32 Debugger (on-chip debugging)
    └── CANalyzer (CAN message logging)

Test Scenario Example:

# HIL Test Case: Following Distance Regulation
# Implements: [SWE-5] Integration Test for SensorFusion + ACC_Control
# ASPICE: SWE.5 BP3

import dspace_api as ds

def test_following_distance_regulation():
    """
    Scenario: Lead vehicle drives at constant 100 km/h, ACC maintains 2.0s gap
    Expected: ACC keeps distance at 55.6m ± 10%
    """
    # Initialize HIL environment
    ds.reset_simulation()
    ds.load_scenario("highway_cruise_100kmh.xml")

    # Set ACC parameters
    ds.set_variable("ACC_SetSpeed", 100.0)  # km/h
    ds.set_variable("ACC_TimeGap", 2.0)     # seconds
    ds.set_variable("ACC_Enable", True)

    # Run simulation for 60 seconds
    ds.start_simulation(duration=60.0)

    # Measure distance to lead vehicle (sampled at 20 Hz)
    distances = ds.get_signal_data("Radar_LeadVehicleDistance", sample_rate=20)

    # Calculate statistics
    mean_distance = np.mean(distances)
    std_distance = np.std(distances)

    # Verify: Target = 2.0s × 27.8 m/s = 55.6 meters
    target_distance = 55.6  # meters
    tolerance = 0.10        # ±10% (SYS-002.2 requirement)

    assert abs(mean_distance - target_distance) <= (target_distance * tolerance), \
        f"Distance error: {mean_distance:.1f}m (expected: {target_distance}m ±10%)"

    assert std_distance < 2.0, \
        f"Distance oscillation too high: {std_distance:.2f}m (max: 2.0m)"

    print(f"[PASS] PASS: Mean distance {mean_distance:.1f}m, Std {std_distance:.2f}m")

# Run test
test_following_distance_regulation()

Test Results:

HIL Test Case: Following Distance Regulation
─────────────────────────────────────────────────────────────
Requirement: [SYS-002] Maintain Safe Following Distance
Expected: 55.6m ± 10% (50.0 - 61.2m)
Actual:   55.2m (std: 1.3m)

Status: [PASS] PASS

Performance Metrics:
  - Distance error: -0.4m (0.7% of target)
  - Oscillation: 1.3m std (acceptable, <2.0m limit)
  - Control stability: [PASS] No overshoot

HIL Test Campaign (500+ Scenarios)

Test Coverage (SWE.5 BP2):

Total: 491/500 Pass (98.2% pass rate)

Failure Analysis (9 failures):

1. TC-HIL-234: Cut-in vehicle at high speed (120 km/h) → ACC brakes too late (gap 1.8s instead of 2.0s)

   • Root Cause: Kalman filter converges slowly for abrupt distance changes
   • Fix: Increase process noise Q matrix for faster adaptation
   • Retest: [PASS] Pass after fix

2. TC-HIL-456: Camera blinded by low sun → ACC disables unexpectedly

   • Root Cause: Confidence metric too sensitive (threshold too high)
   • Fix: Lower confidence threshold from 0.8 to 0.6
   • Retest: [PASS] Pass

Qualification Testing (SWE.6)

System-Level Validation

ASPICE SWE.6: Software Qualification Testing (validates against SYS.2 requirements)

Test Environment:

• Vehicle: Test vehicle (prototype with ACC ECU installed)
• Track: Proving ground (closed circuit)
• Conditions: Dry, wet, night, fog (varied weather)
• Driver: Professional test driver + safety engineer

Test Procedure Example (SYS-002 validation):

## Qualification Test Procedure: QTP-SYS-002

**Requirement**: [SYS-002] Maintain Safe Following Distance
**Safety Level**: ASIL-B
**Test Method**: Real-world driving test

### Setup
- Location: Proving ground (5 km closed loop)
- Weather: Dry, daytime
- Lead vehicle: Instrumented test car (GPS + data logger)
- Own vehicle: ACC prototype (ECU v1.0.0)

### Procedure
1. Accelerate to 100 km/h in cruise mode (no lead vehicle)
2. Lead vehicle enters lane ahead at 100 km/h
3. ACC detects lead vehicle, switches to following mode
4. Driver sets time gap to 2.0s via steering wheel button
5. Maintain following mode for 5 minutes
6. Measure actual distance every 1 second (GPS-based)

### Acceptance Criteria
1. Distance maintained: 55.6m ± 10% (50.0 - 61.2m)
2. No safety violations (distance never <45m)
3. Comfort: Jerk <1.5 m/s³ (smooth acceleration/deceleration)
4. Driver intervention: None required (system stable)

### Results (Execution Date: 2025-10-05)
- Mean distance: 54.8m [PASS] (within 50.0 - 61.2m)
- Min distance: 51.2m [PASS] (above 45m safety limit)
- Max distance: 59.1m [PASS]
- Jerk: 0.9 m/s³ max [PASS] (below 1.5 m/s³ limit)
- Driver interventions: 0 [PASS]

**Verdict**: [PASS] PASS

**Signed**: Test Engineer (John Doe), Safety Engineer (Jane Smith)
**Date**: 2025-10-05

Qualification Test Campaign Summary

Total Tests: 120 qualification tests (SYS-001 through SYS-120)

Overall: 118/120 Pass (98.3%)

Failures:

1. QTP-SYS-015: ACC speed control oscillation at low speed (30 km/h)

   • Root Cause: PID tuning suboptimal for low-speed regime
   • Fix: Adaptive PID gains (different Kp/Ki/Kd below 50 km/h)
   • Retest: [PASS] Pass

2. QTP-SYS-305: CAN message latency >20ms (exceeds 15ms spec)

   • Root Cause: CAN bus load from other ECUs (not ACC issue)
   • Resolution: OEM adjusted CAN arbitration IDs (lower priority for non-critical messages)
   • Retest: [PASS] Pass

ISO 26262 Compliance Testing

Safety Testing (ASIL-B)

Fault Injection Tests (ISO 26262-6, Table 10):

Results: 6/6 safety tests passed (100% compliance with ISO 26262-6)

SOTIF Validation (ISO 21448)

Safety of the Intended Functionality - Edge cases where system behaves as designed but unsafe:

Example SOTIF Scenario:

## SOTIF Test Case: TC-SOTIF-012

**Scenario**: Dense fog, visibility <20 meters
**System Limitation**: Radar range 150m, but camera range only 30m in fog

**Test Procedure**:
1. Enable ACC at 100 km/h in clear weather
2. Enter dense fog zone (visibility: 15m)
3. Lead vehicle present at 50m distance

**Expected Behavior** (SOTIF requirement):
- System recognizes degraded sensor performance (camera confidence <0.3)
- Proactively reduces max ACC speed to 80 km/h (safe speed for reduced visibility)
- Alerts driver: "Reduced visibility, ACC speed limited"

**Actual Result**:
- [PASS] Camera confidence: 0.18 (correctly detected degradation)
- [PASS] Speed limited to 80 km/h
- [PASS] Driver alerted via instrument cluster
- [PASS] No safety violation (maintained safe distance despite fog)

**Verdict**: [PASS] PASS

SOTIF Test Coverage: 50 scenarios (fog, rain, snow, low sun, tunnels, etc.) Pass Rate: 92% (46/50 pass)

SIL/MIL/HIL Integration: This project used HIL testing primarily. For earlier validation, consider Software-in-the-Loop (SIL) and Model-in-the-Loop (MIL) testing using tools like dSPACE VEOS or MathWorks Simulink Test. SIL/MIL enables testing before hardware availability, shifting defect detection left.

Verification Summary

Test Metrics (All Levels)

Total: 826 test cases, 811 pass (98.2% overall pass rate)

Defect Tracking

Defect Metrics (Month 3-12)

Target: ≤2.0 defects/KLOC (automotive industry average) Achieved: 1.6 defects/KLOC (20% better than target)

Summary

Verification & Validation Deliverables:

Next: Lessons learned and project retrospective (25.06).