Accelerating GitLab CI: Reducing Build Times by 59% with Persistent Runners

We Cut Our GitLab Build Time by 59% With One Change

GitLab shared runners are ephemeral by design, forcing jobs to re-download dependencies and Docker layers on every push. By moving to a dedicated runner, one team reduced their build duration from 3 minutes 42 seconds down to just 1 minute 31 seconds.

Why This Matters

Ephemeral runners prioritize isolation but sacrifice performance by destroying the environment after every job, causing a bottleneck where multi-hundred megabyte archives must be round-tripped through S3. In reality, modern containerized builds rely heavily on local disk state; the lack of a persistent Docker daemon creates a scenario where downloading base layers often takes significantly longer than executing the actual build logic.

Key Insights

• Docker layer caching works natively on persistent machines, eliminating the need for complex registry-based workarounds or BuildKit inline caching.
• Shared runners often suffer from high queue times, whereas dedicated runners pick up jobs immediately because they are not shared with other GitLab.com users.
• The GitLab CI cache keyword can be inefficient for large projects as uploading and downloading 500MB archives to object storage introduces its own latency.
• Dedicated runners maintain a local /cache volume mounted as a Docker volume, allowing dependency managers like npm to read directly from disk.
• Docker-in-Docker (DinD) builds see the most significant gains because the persistent Docker daemon retains base images and unchanged layers across jobs.

Working Examples

Typical Node.js setup where npm ci benefits from local volume persistence on dedicated runners.

build:
  stage: build
  image: node:20
  cache:
    key: ${CI_COMMIT_REF_SLUG}
    paths:
      - node_modules/
  script:
    - npm ci
    - npm run build

Docker-in-Docker build configuration that avoids re-pulling base layers when using a persistent runner.

build-image:
  stage: build
  image: docker:24.0.5
  services:
    - docker:24.0.5-dind
  script:
    - docker build -t myapp:latest .

Practical Applications

• Use Case: Monorepos with large dependency trees utilize local volumes to speed up installation phases across 20+ daily pipeline runs.
• Pitfall: Relying on S3-backed caches for large archives results in high network overhead and frequent, silent cache misses.
• Use Case: Teams requiring instant job execution use dedicated VMs to bypass the shared runner queues on GitLab.com.
• Pitfall: Self-hosting runners without automated disk management can lead to build failures when /var/lib/docker fills with accumulated layers.

References:

• https://dev.to/sepcy/we-cut-our-gitlab-build-time-by-59-with-one-change-lle