QCon AI New York 2025: AI Works, PRs Don't – Addressing SDLC Disruption

AI Works, PRs Don’t: How AI Is Breaking the SDLC and What To Do About It

Michael Webster, Principal Engineer at CircleCI, detailed the disruptive impact of AI on the Software Development Lifecycle (SDLC) at QCon AI New York 2025. His presentation, “AI Works, Pull Requests Don’t: How AI Is Breaking the SDLC and What To Do About It,” highlighted a common pattern: initial gains in velocity followed by increased technical debt.

Why This Matters

The promise of AI-assisted coding is significant, but current implementations often prioritize speed over maintainability, creating a divergence from ideal software engineering practices. Without proper safeguards, increased code complexity and static analysis warnings can lead to long-term instability and escalating costs; a study of 7500 tests at CircleCI showed Test Impact Analysis reduced testing time from 30 minutes to 1.5 minutes, demonstrating the cost savings of optimized processes.

Key Insights

• AI Velocity Dip: AI-assisted coding provides a 3-5x development velocity increase for approximately one month before technical debt accumulation, according to CircleCI’s internal observations.
• Cursor Study: A Carnegie Mellon University study (He et al., ongoing) found that while Cursor increased initial development velocity and software quality, these gains were temporary, diminishing after about a month.
• Little’s Law: A queuing theory principle (Little, 1961) defining the relationship between average items in a system (L), arrival rate (λ), and average wait time (W) – L = λW – helps explain bottlenecks in CI/CD pipelines.

Working Example

// Example: Test Impact Analysis (TIA) mapping in TypeScript
interface EndpointToTestMap {
  [endpoint: string]: string[]; // Maps endpoint to relevant test file(s)
}

const testMap: EndpointToTestMap = {
  '/users': ['users.test.ts', 'userAuthentication.test.ts'],
  '/products': ['products.test.ts', 'productInventory.test.ts'],
  // ... more endpoints and tests
};

function getTestsForEndpoint(endpoint: string): string[] {
  return testMap[endpoint] || []; // Return tests for endpoint, or empty array if not found
}

// Example usage:
const endpoint = '/users';
const testsToRun = getTestsForEndpoint(endpoint);
console.log(`Running tests for ${endpoint}:`, testsToRun);

Practical Applications

• CircleCI: Uses Test Impact Analysis (TIA) to reduce CI/CD throughput and costs by only running tests relevant to code changes.
• Pitfall: Over-reliance on AI-generated code without thorough review can lead to a rapid increase in technical debt and system instability, negating initial velocity gains.

References:

• https://www.infoq.com/news/2025/12/ai-in-sdlc-webster/
• Little, J. D. C. (1961). A proof for the queueing formula L = λW. Operations Research, 9(3), 322–326.