Avoiding the Year Two Test Suite Collapse: Technical Debt in QA Automation

Why your test suite becomes unmaintainable in year two

Tudor Brad of BetterQA describes inheriting a client repository featuring a 3,147-line Page Object file and a CI pipeline with a five-attempt retry policy. This level of instability meant the team trusted the suite enough to skip manual checks but not enough to believe a failure was real.

Why This Matters

In theory, the Page Object Model simplifies UI testing, but in technical reality, scaling a single file to thousands of lines creates an unmanageable monolith where updating one selector becomes archaeology. When flaky tests lead teams to ignore red builds or implement excessive retries, the maintenance cost exceeds the automation’s value, effectively making the suite worse than having no tests at all.

Key Insights

• A single 3,147-line Page Object file at a BetterQA client led to unmaintainable code where four candidates existed for every selector update, hiding genuine bugs.
• Refactoring massive files into URL-specific component objects allowed BetterQA to split 3,147 lines of code into 23 manageable files with the same coverage.
• Playwright’s getByRole and getByText locators provide higher resilience against design changes compared to deep DOM XPaths like //div[1]/div[2]/div[3]/span[4].
• BetterQA tracks a ‘Flake Rate’ metric; if more than 2% of tests require a rerun to pass, new feature development stops in favor of test stabilization.
• Flows is a self-healing test recorder used by BetterQA to automatically find alternative elements when primary selectors break during UI iterations.

Working Examples

Example of refactoring a monolithic Page Object into smaller, composable component objects using stable data-testid attributes.

// Refactoring a 3,000-line MainPage.js into components
// Instead of one massive class, use Component Objects
class Sidebar {
  constructor() {
    this.userProfile = '[data-testid="sidebar-profile"]';
  }
}

class DashboardPage {
  constructor() {
    this.sidebar = new Sidebar();
    this.statsWidget = '[data-testid="stats-container"]';
  }
}

// Using stable data-testid hooks instead of brittle XPaths
// Avoid: //div[1]/div[2]/div[3]/span[4]
const submitButton = page.locator('[data-testid="checkout-submit"]');

Practical Applications

• Use Case: BetterQA refactored a 3,147-line file into 23 component objects to surface bugs previously lost in ‘retry noise.’
• Pitfall: Migrating frameworks (e.g., Selenium to Cypress) to fix flakiness without addressing underlying environment state issues, which often wastes budget without improving reliability.
• Use Case: Implementing data-testid attributes through developer collaboration to eliminate the need for XPath archaeology when UI layouts change.
• Pitfall: Running data-driven tests with over 20 rows of data, which can break an entire report and lead teams to disable the test rather than fix it.

References:

• https://dev.to/tudorsss-betterqa/why-your-test-suite-becomes-unmaintainable-in-year-two-1nm0