Green CI Checks Don't Mean Working Software

Green CI working software

Developer Nazarii Ahapevych encountered two consecutive no-op releases despite green pipeline checks using python-semantic-release. Both jobs exited successfully without pushing a version tag or including build artifacts.

Why This Matters

A green checkmark only indicates that a command exited with code zero—it does not verify that the intended side effects happened. Automated pipelines often encode assumptions about which branches execute and what outputs are staged. When those assumptions go untested until production conditions align, the pipeline reports success while silently omitting critical steps like version bumps or artifact commits. This gap between “passed” and “works” erodes trust in automation, making engineers chase phantom failures after deployments rather than catching them during verification.

Key Insights

• • python-semantic-release short-circuits on docs: / ci: / chore: commits — job exits zero without triggering push logic (Nazarii Ahapevych, July 2026).
• • The build_command option runs your shell command but does not automatically stage its output; antifacts remain unstaged unless explicitly added before commit (Ahapevych, July 2026).
• • A single-job view hides cross‑stage bugs: the missing artifacts were invisible until someone reviewed the whole branch diff (Ahapevych, July 2026).
• • Asserting effect over exit code: downstream steps should confirm tags exist or diffs contain expected files (Ahapevych recommends this practice).

Practical Applications

• • Automated semantic releases via conventional commits: trusting green check alone may mask skipped releases when recent commits lack feat: / fix: patterns. Pitfall: relying solely on exit codes instead of asserting tag creation.
• • Build‑then‑release pipelines: a successful build step does not guarantee its outputs will be committed unless staging is explicit. Pitfall assuming build artifacts propagate automatically into subsequent Git operations.
• • Protected‑branch pushes requiring tokens: token permissions remain untested until an actual push attempt occurs. Pitfall deferring integration verification until production scenarios arise.

References:

• https://dev.to/nazarii-nazarevich/green-ci-working-sofware-Scim