regression-bisect

Automatically identify the commit that introduced a regression using git bisect.

Triggers

•"find regression commit"
•"bisect the regression"
•"when did this break"
•"which commit broke [feature]"
•"git bisect [test/issue]"
•"identify breaking commit"

Purpose

This skill automates regression root cause analysis by:

•Using git bisect to binary search commit history
•Running automated tests at each bisect point
•Identifying the exact commit that introduced a failure
•Providing blame information and diff context
•Suggesting potential fixes based on the breaking change

Behavior

When triggered, this skill:

•
Validates prerequisites:
- •Confirms reproducible failure exists
- •Verifies test automation is available
- •Ensures clean working directory
- •Identifies good and bad commits
•
Executes git bisect:
- •Starts bisect with known good/bad commits
- •Runs test suite at each bisect point
- •Marks commits as good/bad based on test results
- •Continues until breaking commit is found
•
Analyzes breaking commit:
- •Shows commit message and metadata
- •Displays full diff of the breaking change
- •Identifies changed files and functions
- •Maps to requirements/issues if available
•
Provides context:
- •Shows blame information for affected code
- •Lists related commits in the area
- •Identifies potential reviewers (commit authors)
- •Suggests rollback or fix strategies
•
Documents findings:
- •Creates regression analysis report
- •Links to issue tracker if available
- •Updates regression baseline with findings
- •Generates timeline of when regression was introduced

Bisect Configuration

Manual Bisect

yaml

bisect_manual:
  mode: interactive
  good_commit: v1.2.0  # Known good state
  bad_commit: HEAD     # Known bad state
  test_command: "npm test -- auth.test.ts"

  workflow:
    - git bisect start
    - git bisect bad HEAD
    - git bisect good v1.2.0
    - git bisect run npm test -- auth.test.ts

Automated Bisect

yaml

bisect_automated:
  mode: automated
  detect_good: auto  # Use last passing CI build
  detect_bad: auto   # Use current failing state
  test_command: "npm test"
  timeout: 600       # 10 minutes per commit test

  auto_detection:
    - check_ci_history
    - find_last_green_build
    - set_as_good_commit
    - use_current_as_bad

Bisect with Custom Test

yaml

bisect_custom:
  mode: custom
  test_script: scripts/regression-test.sh
  success_exit_code: 0
  failure_exit_code: 1

  test_script_template: |
    #!/bin/bash
    # Custom regression test
    npm run build || exit 1
    npm test -- specific-test.ts || exit 1
    ./validate-output.sh || exit 1
    exit 0

Bisect Report Format

markdown

# Regression Bisect Report

**Date**: 2026-01-28
**Issue**: Authentication fails on login
**Analyzer**: regression-bisect skill

## Executive Summary

**Breaking Commit**: abc123def456
**Author**: John Developer <john@example.com>
**Date**: 2026-01-15 14:32:00
**Bisect Duration**: 8 commits tested, 12 minutes

**Root Cause**: Token validation logic changed to require `iss` claim

## Bisect Results

### Commit Range

- **Good**: v1.2.0 (2026-01-10) - 50 commits ago
- **Bad**: HEAD (2026-01-28) - Current state
- **Breaking**: abc123def456 (2026-01-15) - 25 commits into range

### Bisect Log

| Commit | Status | Test Result | Duration |
|--------|--------|-------------|----------|
| abc123 | ❌ Bad | auth.test.ts failed | 45s |
| def789 | ❌ Bad | auth.test.ts failed | 42s |
| ghi012 | ✅ Good | all tests passed | 48s |
| jkl345 | ✅ Good | all tests passed | 46s |
| mno678 | ❌ Bad | auth.test.ts failed | 44s |
| **pqr901** | **❌ Breaking** | **First failure** | **43s** |
| stu234 | ✅ Good | all tests passed | 47s |
| vwx567 | ✅ Good | all tests passed | 45s |

## Breaking Commit Analysis

### Commit Details

commit pqr901xyz Author: John Developer john@example.com Date: Mon Jan 15 14:32:00 2026 +0000

code

feat(auth): add JWT issuer validation

- Require `iss` claim in JWT tokens
- Validate issuer against whitelist
- Update token generation to include issuer

Closes #456

code


### Files Changed

| File | Changes | Impact |
|------|---------|--------|
| src/auth/validate-token.ts | +15/-5 | High - Core validation logic |
| src/auth/generate-token.ts | +8/-2 | Medium - Token generation |
| test/auth/token.test.ts | +25/-0 | Low - Test updates |

### Diff Analysis

```diff
diff --git a/src/auth/validate-token.ts b/src/auth/validate-token.ts
index abc123..def456 100644
--- a/src/auth/validate-token.ts
+++ b/src/auth/validate-token.ts
@@ -15,6 +15,11 @@ export function validateToken(token: string): TokenPayload {
     throw new Error('Invalid token signature');
   }

+  // NEW: Require issuer claim
+  if (!payload.iss) {
+    throw new Error('Token missing issuer claim');
+  }
+
   return payload;
 }

Breaking Change: The new validation requires iss claim, but existing tokens don't include it.

Root Cause

The commit added mandatory iss (issuer) claim validation without:

•Updating existing token generation (added in same commit but missed migration)
•Providing backward compatibility for existing tokens
•Coordinating with deployment to invalidate old sessions

Impact: All existing user sessions became invalid immediately.

Affected Components

•Authentication: Token validation fails for all existing sessions
•API Gateway: 401 errors on all authenticated endpoints
•User Experience: Force logout of all active users

Related Information

Traceability

•Issue: #456 - "Add JWT issuer validation for security"
•Requirement: @.aiwg/requirements/use-cases/UC-AUTH-003-jwt-security.md
•Review: PR #789 - Merged without integration testing

Similar Issues

Git history shows related changes:

•commit xyz789 (2026-01-10): Initial JWT implementation
•commit uvw456 (2026-01-12): Add JWT expiration validation
•commit pqr901 (2026-01-15): Add JWT issuer validation ← BREAKING

Code Ownership

code

src/auth/validate-token.ts:
  85% John Developer <john@example.com>
  10% Jane Reviewer <jane@example.com>
   5% Bob Maintainer <bob@example.com>

Recommendations

Immediate (Hotfix)

• Option A: Revert commit pqr901 to restore service
• Option B: Make iss validation optional with feature flag
• Option C: Deploy token refresh that includes iss claim

Recommended: Option B - Feature flag allows gradual rollout

Short-term (This Sprint)

• Add integration tests for auth changes
• Add backward compatibility tests for token validation
• Update deployment runbook for auth changes
• Add session migration logic for breaking changes

Ongoing

• Require integration tests for auth PRs
• Add "breaking change" checklist to PR template
• Schedule auth regression suite in CI
• Document token format in API docs

Fix Strategy

Proposed Fix

typescript

// src/auth/validate-token.ts
export function validateToken(token: string): TokenPayload {
  const payload = jwt.verify(token, SECRET_KEY);

  if (!payload.sub) {
    throw new Error('Invalid token signature');
  }

  // FIXED: Make issuer validation optional during migration
  if (featureFlags.requireIssuer && !payload.iss) {
    throw new Error('Token missing issuer claim');
  }

  return payload;
}

Migration Path

•Deploy fix with requireIssuer: false
•Update token generation to include iss
•Wait for all old tokens to expire (24 hours)
•Enable requireIssuer: true
•Remove feature flag after validation

Timeline

code

2026-01-10: v1.2.0 released (working)
2026-01-15: Commit pqr901 merged (breaking change introduced)
2026-01-15: No CI failure (tests didn't catch integration issue)
2026-01-16: Deployed to production
2026-01-16: User reports started (authentication failures)
2026-01-28: Bisect identified root cause

Time to Detection: 13 days Time to Root Cause: Immediate (via bisect)

Prevention

This regression could have been prevented by:

•Integration test covering token validation end-to-end
•Staged rollout with feature flag
•Backward compatibility review in PR checklist
•Session invalidation notice to users

References

•Bisect documentation: docs/debugging/git-bisect-guide.md
•Breaking change policy: docs/development/breaking-changes.md
•Auth architecture: @.aiwg/architecture/auth-architecture.md

code


## Usage Examples

### Basic Regression Bisect

User: "Find which commit broke authentication"

Skill executes:

•Identify current failure: auth tests failing
•Find last known good: v1.2.0 (CI history)
•Run git bisect with test: npm test -- auth.test.ts
•Test 8 commits in 12 minutes
•Identify breaking commit: pqr901

Output: "Regression Bisect Complete

Breaking Commit: pqr901xyz Author: John Developer Date: 2026-01-15 14:32:00 Message: feat(auth): add JWT issuer validation

Root Cause: Required iss claim without backward compatibility

Files Changed:

•src/auth/validate-token.ts (+15/-5)
•src/auth/generate-token.ts (+8/-2)

Recommendation: Make validation optional with feature flag

Full report: .aiwg/testing/regression-bisect-auth.md"

code


### Bisect with Custom Test

User: "Bisect the payment processing regression"

Skill asks: "What is a good commit (when it worked)?" User: "v2.1.0"

Skill executes:

•git bisect start
•git bisect bad HEAD
•git bisect good v2.1.0
•git bisect run scripts/payment-test.sh

Output: "Breaking Commit: def456 Broke payment refund logic Changed: src/payments/refund.ts

Bisected 45 commits in 32 minutes See .aiwg/testing/regression-bisect-payment.md"

code


### Automatic Good/Bad Detection

User: "When did the API start returning 500 errors?"

Skill analyzes:

•Current state: 500 errors on /api/users
•CI history: Last green build was 2 days ago
•Auto-detect good: commit xyz789 (last green)
•Auto-detect bad: HEAD (current failing)

Runs bisect automatically.

Output: "Regression introduced in commit abc123 2 days ago by Jane Developer Changed database query logic

Breaking change: Added JOIN without NULL check See full analysis in .aiwg/testing/regression-api-500.md"

code


## Integration

This skill uses:
- `regression-baseline`: Load known good states
- `project-awareness`: Detect test commands and CI configuration
- `test-coverage`: Identify relevant test suites
- Git tools: For bisect operations and history analysis

## Agent Orchestration

```yaml
agents:
  analysis:
    agent: debugger
    focus: Root cause analysis and blame

  testing:
    agent: test-engineer
    focus: Test execution and validation

  documentation:
    agent: technical-writer
    focus: Report generation

Configuration

Bisect Presets

yaml

bisect_presets:
  auth_regression:
    test_command: npm test -- auth.test.ts
    timeout: 300
    auto_detect_good: true

  api_regression:
    test_command: npm run test:api
    timeout: 600
    auto_detect_good: true

  performance_regression:
    test_command: npm run benchmark
    threshold: "response_time < 1000ms"
    timeout: 900

Timeout Strategy

yaml

timeout_config:
  per_commit_test: 600     # 10 minutes
  total_bisect: 3600       # 1 hour max
  on_timeout: skip_commit  # or fail_bisect

Error Handling

Unstable Tests

If test results are inconsistent:

•Run test 3 times per commit
•Require 2/3 passes to mark as good
•Flag flaky tests in report
•Recommend test stabilization

Build Failures

If commit doesn't build:

•Skip commit and mark as bad
•Continue bisect with next commit
•Flag build issues in report
•Note potential false positive

Timeout Exceeded

If bisect takes too long:

•Stop bisect at timeout
•Report progress so far
•Suggest narrowing commit range
•Offer to resume later

Output Locations

•Bisect reports: .aiwg/testing/regression-bisect-{issue}.md
•Bisect logs: .aiwg/testing/bisect-logs/
•Fix recommendations: .aiwg/working/regression-fixes/

References

•@agentic/code/frameworks/sdlc-complete/schemas/test/regression-schema.yaml
•@agentic/code/frameworks/sdlc-complete/commands/regression-test.md
•@.claude/agents/debugger.md