Nomad vs. Kubernetes: Why We Switched Our SaaS to HashiCorp Nomad

The Complexity Tax: Why Kubernetes Isn’t Always the Answer

Many small-to-medium SaaS teams face a significant complexity tax when adopting Kubernetes as their default orchestrator. Ameer Hamza reports that his team spent 30% of their engineering time over two years solely maintaining their production K8s cluster instead of shipping features.

Why This Matters

The technical reality of Kubernetes often clashes with the ideal of automated scaling, as teams frequently struggle with debugging CNI plugins, managing complex RBAC, and wrestling with verbose Helm charts. HashiCorp Nomad addresses this by offering a single-binary architecture that simplifies operations while maintaining the ability to manage both containerized and non-containerized workloads.

Key Insights

• Nomad operates as a single binary, consolidating components like the API server and scheduler that are loosely coupled in Kubernetes.
• The HashiStack modular approach delegates service discovery to Consul and secrets management to Vault rather than bundling them into the orchestrator.
• HashiCorp Configuration Language (HCL) provides a more readable and less verbose alternative to standard Kubernetes YAML for job definitions.
• Nomad supports diverse task drivers including Docker, standalone binaries, Java, and virtual machines, offering greater flexibility than container-centric platforms.
• Resource usage in Nomad is significantly lower than Kubernetes, making it more efficient for smaller infrastructure footprints.

Working Examples

A basic Nomad agent configuration for high-availability setup.

data_dir = "/opt/nomad/data" bind_addr = "0.0.0.0" server { enabled = true bootstrap_expect = 3 } client { enabled = true }

A production-ready Nomad job file for a Next.js frontend.

job "webapp" { datacenters = ["dc1"] type = "service" group "frontend" { count = 3 network { port "http" { to = 3000 } } service { name = "webapp-frontend" port = "http" check { type = "http" path = "/api/health" interval = "10s" timeout = "2s" } } task "nextjs" { driver = "docker" config { image = "my-registry/webapp:latest" ports = ["http"] } resources { cpu = 500 memory = 256 } env { NODE_ENV = "production" } } } }

Injecting Vault secrets directly into Nomad environment variables.

template { data = <<EOH DATABASE_URL="{{with secret "database/creds/readonly"}}{{.Data.url}}{{end}}" EOH destination = "secrets/file.env" env = true }

Practical Applications

• System behavior: Using Consul and Nomad together allows for automatic service registration and dynamic load balancer updates via Fabio or Traefik.
• Pitfall: Relying on Nomad’s default host network stack can lead to port collisions; use bridge networking with CNI plugins for container isolation.
• System behavior: Native Vault integration allows for secure secret injection as files or environment variables without base64 encoding risks.
• Pitfall: Managing stateful databases via Nomad CSI can be complex; use host_volume for high IOPS or transition to managed services like RDS.

References:

• https://dev.to/ameer-pk/nomad-vs-kubernetes-why-we-switched-our-saas-to-hashicorp-nomad-5fhb