4.1: MLE.1 ML Requirements Analysis
Process Definition
Purpose
MLE.1 Purpose: To establish a set of ML requirements from system requirements that will guide ML model development.
Outcomes
| Outcome | Description |
|---|---|
| O1 | The ML requirements including ML data requirements are identified and specified based on the software requirements and the components of the software architecture |
| O2 | ML requirements are structured and prioritized |
| O3 | ML requirements are analyzed for correctness and verifiability |
| O4 | The impact of ML requirements on the ML operating environment is analyzed |
| O5 | Consistency and bidirectional traceability are established between ML requirements and software requirements, and between ML requirements and software architecture |
| O6 | The ML requirements are agreed and communicated to all affected parties |
Base Practices with AI Integration
| BP | Base Practice | AI Level | AI Application |
|---|---|---|---|
| BP1 | Specify ML requirements | L1 | Requirement derivation, data requirements |
| BP2 | Structure ML requirements | L1 | Structuring, prioritization |
| BP3 | Analyze ML requirements | L1-L2 | Quality analysis, verifiability check |
| BP4 | Analyze the impact on the ML operating environment | L1 | Environment impact analysis |
| BP5 | Ensure consistency and establish bidirectional traceability | L2 | Link suggestion, consistency checking |
| BP6 | Communicate agreed ML requirements | L1 | Documentation generation |
ML Requirement Types
Functional ML Requirements
| Category | Example |
|---|---|
| Detection | "ML shall detect pedestrians with 99% recall" |
| Classification | "ML shall classify road signs with 95% accuracy" |
| Prediction | "ML shall predict collision risk within 2 seconds" |
| Segmentation | "ML shall segment drivable area with 98% IoU" |
| Regression | "ML shall estimate distance with ±5% error" |
Performance Requirements
| Metric | Example Requirement |
|---|---|
| Accuracy | "Classification accuracy ≥ 95% on test set" |
| Precision | "Pedestrian detection precision ≥ 98%" |
| Recall | "Vehicle detection recall ≥ 99.9%" |
| F1-Score | "F1 score ≥ 0.97 for object detection" |
| Latency | "Inference time ≤ 50ms on target hardware" |
| Throughput | "Process ≥ 30 frames per second" |
Data Requirements
The following diagram outlines the ML data requirements structure, covering dataset size, distribution, labeling quality, and ODD coverage criteria that must be specified for each ML component.
Operational Design Domain (ODD)
ODD Specification
Note: This comprehensive ODD example can be adapted for different domains (urban, parking, etc.).
# Operational Design Domain Definition
odd_specification:
id: MLE-ODD-001
name: Highway ADAS Perception ODD
version: 1.2
environmental_conditions:
weather:
clear: required
rain_light: required
rain_heavy: optional
snow: optional
fog_light: required
fog_heavy: out_of_scope
lighting:
daylight: required
twilight: required
night_streetlight: required
night_no_light: optional
temperature:
range: [-40, 50] # Celsius
note: "Affects sensor performance"
road_conditions:
type:
- highway
- highway_ramp
- expressway
lanes:
min: 2
max: 6
markings: required
surface:
- asphalt
- concrete
dynamic_objects:
vehicles:
types: [car, truck, motorcycle, bus]
speed_range: [0, 200] # km/h
distance_range: [5, 250] # meters
pedestrians:
in_scope: false # Highway ODD
note: "Pedestrians not expected on highway"
sensor_requirements:
camera:
resolution: [1920, 1080]
frame_rate: 30
field_of_view: 120 # degrees
radar:
range: 250 # meters
update_rate: 20 # Hz
out_of_scope:
- "Construction zones with altered lanes"
- "Emergency vehicles with sirens"
- "Extreme weather (hurricane, tornado)"
- "Off-road driving"
degradation_handling:
sensor_occlusion:
response: "Request driver takeover"
timeout: 10 # seconds
out_of_odd:
response: "Alert driver, reduce assistance"
Requirement Example
---
ID: MLE-ADAS-001
Title: Vehicle Detection Performance
Type: Functional ML Requirement
Priority: Critical
ASIL: B
Status: Approved
---
## Description
The ML perception model shall detect vehicles in the forward path
with the following performance characteristics:
- Recall: ≥ 99.9% for vehicles within 100m
- Precision: ≥ 98% (false positive rate < 2%)
- Latency: ≤ 50ms inference time on target ECU
## Operational Domain
Within the defined ODD (MLE-ODD-001):
- Highway driving, 2-6 lanes
- Speed: 60-130 km/h
- Weather: clear, light rain, light fog
- Lighting: day, twilight, night with streetlights
## Acceptance Criteria
1. Validated on held-out test set (n ≥ 10,000 frames)
2. Validated on ODD-specific scenarios
3. Validated under target hardware constraints
## Safety Considerations
- False negative (missed detection) → Potential collision
- False positive → Unnecessary braking
- Safety mechanism: Radar fusion confirmation
## Data Requirements
- Training data from MLE-DATA-001
- Test data from MLE-DATA-002 (geographically diverse)
- Edge case data from MLE-DATA-003
## Traceability
- Derived from: SYS-ADAS-050 (System vehicle detection)
- Allocates to: MLE-MODEL-001 (Detection model)
- Verified by: MLE-TEST-001 (Validation test)
AI-Assisted Requirements Analysis
L1: Performance Criteria Selection
AI Analysis for Vehicle Detection Requirements:
───────────────────────────────────────────────
System Requirement: SYS-ADAS-050
"ADAS shall detect all vehicles in forward path"
AI Suggested ML Performance Metrics:
────────────────────────────────────
Primary Metrics:
1. Recall (Sensitivity)
- Recommended threshold: ≥ 99.9%
- Rationale: Safety-critical, must not miss vehicles
- Industry benchmark: 99.5% - 99.99%
2. Precision
- Recommended threshold: ≥ 98%
- Rationale: Balance false positives with safety
- Industry benchmark: 95% - 99%
3. Mean Average Precision (mAP)
- Recommended threshold: ≥ 0.85 @ IoU 0.5
- Rationale: Standard object detection metric
- Industry benchmark: 0.80 - 0.95
Secondary Metrics:
4. Latency
- Recommended: ≤ 50ms end-to-end
- Rationale: Real-time response requirement
- Hardware constraint: Target ECU capability
5. Localization Accuracy
- Bounding box IoU: ≥ 0.7
- Distance estimation error: ≤ 5%
Human Review Required:
• Confirm safety-driven recall threshold
• Validate latency requirement against system timing budget
• Review ODD coverage in test data
L2: Data Completeness Analysis
AI Data Coverage Analysis:
──────────────────────────
Analyzing dataset against ODD requirements...
ODD Coverage Matrix:
───────────────────
Weather Conditions:
[OK] Clear: 450,000 samples (45%)
[OK] Light rain: 150,000 samples (15%)
[GAP] Heavy rain: 20,000 samples (2%) - Below 5% target
[OK] Light fog: 80,000 samples (8%)
[OK] Snow: 50,000 samples (5%)
Lighting Conditions:
[OK] Daylight: 500,000 samples (50%)
[OK] Twilight: 100,000 samples (10%)
[OK] Night + streetlight: 150,000 samples (15%)
[GAP] Night no light: 30,000 samples (3%) - Below 5% target
Geographic Distribution:
[OK] Europe: 400,000 samples (40%)
[OK] North America: 350,000 samples (35%)
[GAP] Asia: 100,000 samples (10%) - Below 15% target
[--] Other: 150,000 samples (15%)
Gap Analysis:
─────────────
1. Heavy rain scenarios underrepresented
Recommendation: Collect 30,000+ additional samples
2. Night without streetlight underrepresented
Recommendation: Collect 20,000+ additional samples
3. Asian road scenarios underrepresented
Recommendation: Partner with Asian OEM for data collection
Human Action Required:
□ Approve data collection plan for gaps
□ Review alternative: synthetic data augmentation
□ Decide on ODD restriction if gaps cannot be filled
Work Products
| WP ID | Work Product | AI Role |
|---|---|---|
| 17-62 | ML requirements specification | Metric suggestion |
| 17-63 | Data requirements specification | Coverage analysis |
| 04-10 | ODD specification | Domain analysis |
| 17-11 | Traceability record | Link generation |
Summary
MLE.1 ML Requirements Analysis:
- AI Level: L1 (AI assists, human decides)
- Primary AI Value: Metric selection, data coverage analysis
- Human Essential: Performance thresholds, ODD boundaries
- Key Outputs: ML requirements, ODD specification, data requirements
- Safety Focus: Clear safety-related metrics (recall > precision)