3.2: Reviewing AI Output

Critical Review of AI-Generated Code

Why Review is Mandatory

AI Makes Mistakes:

  • Hallucinations (invents nonexistent APIs, functions)
  • Logic errors (off-by-one, boundary conditions)
  • Standard violations (MISRA C, CERT C)
  • Missing edge cases (null pointers, overflow, timeouts)
  • Security vulnerabilities (buffer overflow, injection)

ASPICE/ISO 26262 Requirement: Human review mandatory for safety-critical code (ASIL-B+)

Review Effectiveness: Catches 70–90% of AI errors before testing (based on industry studies of code review effectiveness — Fagan, 1976; Shull et al., 2002)

AI Error Taxonomy: Common AI error categories: (1) Hallucinations (invented APIs/functions), (2) Logic errors (off-by-one, boundary conditions), (3) Type errors (implicit conversions), (4) Standard violations (MISRA), (5) Missing error handling, (6) Security vulnerabilities. Track error frequency to identify training needs.

Review Checklist for AI Code

1. Correctness

Does it meet requirements?

/* Requirement: [SWE-045-11] Calculate safe distance (v × 2 seconds) */

/* AI Output: */
float ACC_CalculateSafeDistance(float speed_kmh) {
    return speed_kmh * 2.0;  /* [FAIL] ERROR: Returns km·h (dimensionally incorrect - should be meters) */
}

/* Correct: */
float ACC_CalculateSafeDistance(float speed_kmh) {
    const float KMH_TO_MS = 1.0F / 3.6F;
    float speed_ms = speed_kmh * KMH_TO_MS;
    return speed_ms * 2.0F;  /* [PASS] Correct: Returns meters */
}

Review Question: Does the math/logic match the requirement exactly?

Dimensional Analysis Technique: For calculations, verify units at each step: Input (km/h) x Conversion (h/s) x Time (s) = Output (m). AI often misses unit conversions. Apply dimensional analysis to catch physics-defying errors.

2. Edge Cases

Are boundary and error cases handled?

/* AI often generates nominal case only */

/* AI Output: */
float Divide(float a, float b) {
    return a / b;  /* [FAIL] MISSING: Division by zero check */
}

/* Correct: */
float Divide(float a, float b) {
    if (b == 0.0F) {
        Log_Error(ERROR_DIV_BY_ZERO);
        return 0.0F;  /* [PASS] Defensive: Handle division by zero */
    }
    return a / b;
}

Review Checklist:

  • Null pointer checks (if parameters are pointers)
  • Division by zero
  • Buffer overflow (array bounds)
  • Integer overflow/underflow
  • Timeout handling (CAN, Ethernet)
  • Sensor failure (invalid/out-of-range data)

3. MISRA C:2012 Compliance

Check for violations

/* AI Output: */
uint16_t x = 10;
int16_t y = -5;
int16_t z = x + y;  /* [FAIL] VIOLATION: MISRA 10.4 - Unsigned + signed (implicit conversion) */

/* Correct: */
uint16_t x = 10;
int16_t y = -5;
int16_t z = (int16_t)x + y;  /* [PASS] Explicit cast */

Common AI MISRA Violations:

  • Rule 10.4: Implicit type conversions
  • Rule 11.5: Cast from pointer to pointer
  • Rule 12.1: Operator precedence (missing parentheses)
  • Rule 14.4: For loop condition (not boolean)
  • Rule 21.3: Use of stdlib functions (malloc, free)

Action: Run cppcheck --addon=misra.py on AI-generated code. Note: the MISRA addon requires a separate license.

4. Error Handling

Is error handling defensive?

/* AI Output: Assumes success */
int ReadCANMessage(uint8_t* buffer) {
    CAN_Read(0x200, buffer);  /* [FAIL] MISSING: Return code check */
    return 0;
}

/* Correct: Defensive */
int ReadCANMessage(uint8_t* buffer) {
    int status = CAN_Read(0x200, buffer);
    if (status != CAN_OK) {
        Log_Error(ERROR_CAN_TIMEOUT, 0x200);
        return -1;  /* [PASS] Propagate error */
    }
    return 0;
}

5. Traceability

Are @implements tags present?

/* AI Output: Often missing traceability */
float CalculateDistance(float speed) {
    return speed * 2.0;
}

/* Correct: Add traceability */
/**
 * @brief Calculate safe following distance
 * @implements [SWE-045-11] Safe Following Distance  [PASS]
 */
float ACC_CalculateSafeDistance(float speed_kmh) {
    /* ... */
}

6. Test Coverage

Are tests sufficient?

/* AI often generates only happy path tests */

/* AI Output: */
TEST(ACC, CalculateSafeDistance_50kmh) {
    EXPECT_NEAR(ACC_CalculateSafeDistance(50.0), 27.78, 0.1);
}

/* Missing tests: */
TEST(ACC, CalculateSafeDistance_ZeroSpeed) { /* Boundary */ }
TEST(ACC, CalculateSafeDistance_NegativeSpeed) { /* Error */ }
TEST(ACC, CalculateSafeDistance_MaxSpeed) { /* Boundary */ }

Action: Add tests for boundaries, errors, equivalence partitions

Common AI Errors by Category

1. Hallucinations

Problem: AI invents APIs that don't exist

/* AI Output: */
CAN_ReadMessageWithRetry(0x200, buffer, 3);  /* [FAIL] ERROR: Function doesn't exist! */

/* Correct: */
int CAN_ReadMessageWithRetry(uint16_t msg_id, uint8_t* buffer, int retries) {
    /* [PASS] Implement this function yourself */
}

Detection: Code doesn't compile (undefined function)

Fix: Implement the function or use existing API

2. Off-By-One Errors

Problem: Loop bounds incorrect

/* AI Output: */
for (int i = 0; i <= array_size; i++) {  /* [FAIL] ERROR: Off-by-one (should be < not <=) */
    array[i] = 0;
}

/* Correct: */
for (int i = 0; i < array_size; i++) {  /* [PASS] Correct bounds */
    array[i] = 0;
}

Detection: Buffer overflow in testing

Fix: Review all loop conditions carefully

3. Type Mismatches

Problem: Incompatible types

/* AI Output: */
float speed_kmh = 50.5;
int speed_int = speed_kmh;  /* [FAIL] VIOLATION: Implicit float → int (data loss) */

/* Correct: */
float speed_kmh = 50.5F;
int speed_int = (int)speed_kmh;  /* [PASS] Explicit cast */

4. Missing Constants

Problem: Magic numbers

/* AI Output: */
if (speed > 150) {  /* [FAIL] MISSING: Named constant instead of magic number */
    speed = 150;
}

/* Correct: */
const int MAX_SPEED_KMH = 150;  /* ISO 26262 requirement */
if (speed > MAX_SPEED_KMH) {  /* [PASS] Named constant */
    speed = MAX_SPEED_KMH;
}

Review Process

Step-by-Step Review

1. Read Requirements First

  • Understand what code should do before reviewing
  • Check AI's interpretation matches requirement

2. Compile and Run Tests

  • Does it compile without warnings?
  • Do existing tests pass?
  • What is test coverage? (run gcov)

3. Static Analysis

Prerequisites:

  • cppcheck with MISRA addon (requires premium license)
  • radon (install: pip install radon)
  • clang-format (part of LLVM toolchain)
# MISRA C check
cppcheck --addon=misra.py --enable=all src/ai_generated.c

# Complexity check
radon cc src/ai_generated.c -a

# Format check
clang-format --dry-run --Werror src/ai_generated.c

4. Manual Code Review

  • Read line-by-line
  • Check edge cases, error handling, MISRA compliance
  • Verify traceability tags

5. Add Missing Tests

  • Identify missing test cases
  • Generate additional tests (or ask AI to generate)

6. Document Issues

## AI Code Review Report

**File**: acc_controller.c
**Generated**: 2025-12-18
**Reviewer**: John Smith

**Issues Found**:
1. Line 45: Missing null pointer check (Critical)
2. Line 67: MISRA 10.4 violation (Major)
3. Line 89: Magic number 150 (Minor)
4. Missing: @implements tag (Major)

**Action**: Author to fix all issues before merge

AI-Generated Requirements Review

Requirements Quality Check

AI Output (often vague):

The system shall respond quickly to obstacles.

[FAIL] Problems: "Quickly" is vague, not testable

Corrected:

[SWE-045-14] The system shall detect obstacles within 50ms (95th percentile).
Verification: HIL test with simulated radar, measure latency over 1000 trials.

[PASS] Fix: Quantify, add verification method

Review Checklist:

  • Unambiguous (no vague terms)
  • Quantified (numeric values with units)
  • Testable (pass/fail criteria)
  • Traceable (links to system requirement)
  • Complete (covers nominal + failure cases)
  • Consistent (no conflicts with other requirements)

Summary

Review is Mandatory: AI makes mistakes (hallucinations, logic errors, MISRA violations)

Review Checklist: Correctness, edge cases, MISRA compliance, error handling, traceability, test coverage

Common AI Errors: Hallucinations (nonexistent APIs), off-by-one, type mismatches, magic numbers

Review Process: Read requirements → Compile/test → Static analysis → Manual review → Add tests → Document issues

AI-Generated Requirements: Check for ambiguity, quantification, testability, traceability

Key Principle: Never trust AI blindly — always review critically before merging.

Next: HITL Decision Making (35.03) — When to trust AI, when to override, and when to escalate

Previous
Effective Prompting
Next
HITL Decision Making