Building and Testing Production-Grade Kubernetes RBAC via ServiceAccount Tokens

How I Built a Production-Grade Kubernetes RBAC Setup — And Broke It On Purpose

Adil Khan implemented a production-grade Kubernetes RBAC system using a dedicated ServiceAccount in an isolated namespace. The system explicitly grants get and list verbs while denying secrets access to limit the blast radius of potential compromises.

Why This Matters

While many developers rely on local kubeconfig for testing, production workloads authenticate via ServiceAccount tokens mounted inside pods, creating a gap between local validation and runtime reality. Relying on default ServiceAccounts or ClusterRoles leads to privilege creep and excessive blast radii, whereas granular namespace-scoped Roles ensure that a single compromised workload cannot move laterally or escalate to cluster-admin privileges.

Key Insights

• Dedicated ServiceAccounts prevent privilege creep by ensuring workloads do not share a single identity, a common issue with the default account (Source: Adil Khan, 2026).
• Subresources such as pods/log are treated as separate targets by the API server and are not inherited from pod-level permissions.
• Kubernetes RBAC requires explicit declaration of every verb; the system defaults to denying all actions not specifically permitted in the Role.
• HTTP 403 Forbidden errors indicate that the identity is recognized and connectivity is established, but the specific action is unauthorized in the Role definition.
• Blast radius limitation is achieved by using namespace-scoped Roles and RoleBindings instead of Cluster-wide configurations.

Working Examples

The structural manifest layout for the isolated observability RBAC setup.

observability (namespace) |── log-reader-sa (Dedicated ServiceAccount) |── log-reader-role (Namespace-scoped Role) |── log-reader-binding (Binds SA to Role) └── testing (Deployment for validation)

Practical Applications

• Use Case: Restricting monitoring tools to an observability namespace using local Roles to prevent cluster-wide metadata exposure. Pitfall: Granting ClusterRole access by default, which allows lateral movement across all namespaces.
• Use Case: Explicitly listing pods/log in RBAC rules to enable log streaming for external tools. Pitfall: Assuming pod-level ‘get’ permissions include log access, resulting in runtime 403 errors.
• Use Case: Validating ServiceAccount tokens by executing API calls from inside a running container. Pitfall: Testing only with ‘kubectl auth can-i’, which uses administrative kubeconfig rather than pod-mounted tokens.

References:

• https://dev.to/adil-khan-723/how-i-built-a-production-grade-kubernetes-rbac-setup-and-broke-it-on-purpose-98a
• https://github.com/adil-khan-723/K8s-RBAC