RBAC patterns for production cloud environments

I've watched more incident calls than I'd like get stuck on one question: who actually has access to touch this account, and why. Not "who's supposed to," what the IAM console actually says right now. In shared cloud estates with multiple teams, that gap between assumed and actual access is where RBAC either holds or falls apart.

Most of the time it's not a monitoring gap. Nobody sat down and designed the access model. It grew, one exception at a time, until nobody could explain it in a sentence.

Here's the shape that's held up across a few different orgs: humans get read access by default, machines get scoped write access through federation, and anything elevated leaves a trail. Exceptions exist, but they're either written down somewhere or they're a liability waiting to be found during an audit.

graph LR
    subgraph "Human Access"
        H[Engineer] -->|"SSO federation\nshort-lived token"| IC[IAM Identity Center]
        IC -->|"ReadOnly\nPermission Set"| PROD[Production Account]
        IC -->|"Break-glass\n1-hour session"| PROD
    end

    subgraph "Machine Access"
        R[Repository / CI Pipeline] -->|"OIDC trust\nno static keys"| OP[OIDC Provider]
        OP -->|"Scoped deployer role\nbranch-locked"| PROD
    end

Figure 1. The two identity paths into production. They never share credentials.

When this applies

I'd reach for this model once a few things are true at the same time: multiple application teams deploying into shared AWS accounts, Azure subscriptions, or GCP projects, production changes frequent enough that informal access has become the norm, and compliance or customer reviews asking for attributable changes per engineer or pipeline.

There's also a quieter signal worth watching for: emergency access that's outlived the emergency that created it. That one tends to slip past people because nobody's watching for it directly.

A single account with two engineers probably doesn't need any of this yet. Five teams deploying independently with CI pipelines running AdministratorAccess? That's already overdue.

What usually goes wrong

IAM tends to get treated as a landing zone task you do once and move on from. That's usually where things start drifting.

• "PowerUser" creep. Someone needs to inspect a DynamoDB table mid-incident, so they get PowerUserAccess. Half a year later, fifteen engineers are carrying that policy into production permanently, because nobody owns the job of taking it back.
• CI/CD pipelines running AdministratorAccess. A deploy role hits a permission error during setup, so someone widens it to unblock the team, and it never gets narrowed again. A compromised pipeline or a malicious commit now has the blast radius of the entire account.
• Static, long-lived access keys. Service accounts authenticating with keys sitting in .env files, CI secrets, or some third-party SaaS tool. They leak eventually, rotation rarely happens on schedule, and there's no real audit trail when it matters. OIDC federation solves this for free.
• Break-glass that never breaks back. The role created for one incident stays attached to whoever used it. Eventually it becomes the default path for anyone who finds their normal access too limiting.
• Access that outlives the org chart. Someone changes teams or leaves, and their cloud permissions stick around because access was managed separately from the IdP. An audit six months later turns up a former contractor with production write access, and nobody can say exactly when that should have been revoked.

Human versus machine access

A lot of IAM drift traces back to one mistake: treating human and machine access as the same category of problem.

They're not, really. Different trust models, different expiry needs, and wildly different consequences when something goes wrong.

A human engineer debugging a customer issue needs to read production logs. That's it. If a write change is actually needed, the path is: commit the code, let the pipeline deploy it. The moment someone can write directly to production outside that pipeline, you've lost the audit trail and the rollback point in the same instant.

A CI/CD pipeline's job is narrower than people often design for. It should touch exactly the resources its own service owns, nothing from another team's IAM policies, network foundations, or secrets store. And the role it assumes should be branch-locked, so a feature branch can't quietly deploy to production.

The rule I keep coming back to: if a human is making a write change to production outside an incident, something upstream in the deployment model has already failed.

Reference architecture

Nothing clever here on purpose. A working directory structure looks like:

infra/iam/
  terraform/
    modules/human_access/
      sso_permission_sets.tf  # ReadOnly, AppDeployer, BreakGlass permission sets
      sso_assignments.tf      # Maps IdP groups to accounts and permission sets
    modules/machine_access/
      oidc_providers.tf       # GitHub Actions / GitLab trust relationships
      ci_roles.tf             # One scoped deployer role per service
  scripts/
    audit-stale-roles.sh      # Weekly: flag roles unused for >30 days

The folders matter less than who owns what:

Concern	Owner	Artifact
Access policy (what must be true)	Security lead / platform lead	Written standard + automated assertions
Identity federation	Platform engineering	OIDC providers, SSO configuration
Pipeline roles	Application teams	Scoped IAM policies within platform-defined boundaries
Break-glass review	Security operations	Post-incident CloudTrail review

When those owners blur together, the IAM setup turns into either a dumping ground or a document nobody follows. That table isn't filler, it's arguably the part of this whole system doing the actual work.

CI/CD roles with OIDC

Skip IAM users for pipelines. Use OpenID Connect federation so the pipeline runtime gets short-lived credentials directly, no key to store, nothing sitting around to leak, nothing anyone forgets to rotate.

# terraform/modules/machine_access/ci_roles.tf (illustrative)
resource "aws_iam_role" "app_deployer" {
  name = "ci-deployer-payment-service"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = {
          Federated = aws_iam_openid_connect_provider.github.arn
        }
        Action = "sts:AssumeRoleWithWebIdentity"
        Condition = {
          StringEquals = {
            # Validate the audience so the token can't be one issued for a different service.
            "token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
            # Lock to a specific repository and branch. A fork can't assume this.
            "token.actions.githubusercontent.com:sub": "repo:org/payment-service:ref:refs/heads/main"
          }
        }
      }
    ]
  })
}

# The attached policy should list exactly the actions this pipeline needs,
# not reach for AdministratorAccess to get past a setup error.
resource "aws_iam_role_policy_attachment" "deployer_policy" {
  role       = aws_iam_role.app_deployer.name
  policy_arn = aws_iam_policy.payment_service_deploy_boundary.arn
}

That sub claim check is doing the real work. A fork, a pull request workflow, a different repo entirely, none of them produce a token that matches, so none of them can assume the role. I'd only reach for StringLike if I had a specific reason and understood exactly what it opened up.

Worth flagging: that ref-based subject format is the default, but not universal. Repos using GitHub Environments produce something like repo:org/name:environment:<name> instead, and a custom sub_claim_prefix template changes the shape again. Before locking in the condition, I'd check the actual token via gh api /repos/{org}/{repo}/actions/oidc/customization/sub, or just look at a workflow run's OIDC debug output.

Human access via SSO

Engineers authenticate through whatever IdP you're running, Okta, Azure AD, Google Workspace, and their directory groups map directly to cloud permission sets. Someone leaves or changes teams, you pull them from the IdP group, and their cloud access is gone instantly without touching a single cloud console.

# terraform/modules/human_access/sso_assignments.tf (illustrative)
resource "aws_ssoadmin_permission_set" "read_only" {
  name             = "ProductionReadOnly"
  instance_arn     = tolist(data.aws_ssoadmin_instances.this.arns)
  session_duration = "PT8H"
}

resource "aws_ssoadmin_managed_policy_attachment" "read_only" {
  instance_arn       = tolist(data.aws_ssoadmin_instances.this.arns)
  managed_policy_arn = "arn:aws:iam::aws:policy/ReadOnlyAccess"
  permission_set_arn = aws_ssoadmin_permission_set.read_only.arn
}

# New engineer joins the team? This assignment is the only change needed.
resource "aws_ssoadmin_account_assignment" "platform_engineers" {
  instance_arn       = tolist(data.aws_ssoadmin_instances.this.arns)
  target_id          = var.production_account_id
  target_type        = "AWS_ACCOUNT"
  principal_type     = "GROUP"
  principal_id       = data.aws_identitystore_group.platform_engineers.group_id
  permission_set_arn = aws_ssoadmin_permission_set.read_only.arn
}

I picked eight hours for session duration somewhat deliberately: long enough to cover a real working day, short enough that a session doesn't just sit open through the weekend because someone forgot to close their terminal.

One thing Terraform won't do for you: IAM Identity Center itself has to be switched on manually in the console first. Terraform picks up permission sets and assignments after that instance exists, not before.

Break-glass without permanent privilege creep

Incidents need fast access, and that's fine. What actually matters is that every use of elevated access is visible, time-boxed, and gets looked at afterward.

sequenceDiagram
    participant E as Engineer
    participant IdP as Identity Provider
    participant AWS as Production Account
    participant SecOps as Security Operations

    E->>IdP: Request break-glass permission set
    IdP-->>E: Short-lived token (1 hour max)
    E->>AWS: Assume BreakGlass role
    AWS->>SecOps: EventBridge alert fires immediately
    Note over SecOps: PagerDuty / Slack notification with session details
    E->>AWS: Resolve incident
    Note over AWS: Token expires automatically. No manual revocation needed.
    SecOps->>SecOps: Post-incident CloudTrail review

Figure 2. Break-glass is fast, visible, and automatically bounded. It's not a special permission so much as a permission set with much more aggressive monitoring wrapped around it.

A few things make this actually work in practice:

Each environment gets its own break-glass permission set, scoped there and nowhere else. Session duration is capped at one hour, and that cap only moves if someone changes the code and someone else reviews it. The EventBridge rule fires the second the role gets assumed, so SecOps knows before the engineer's even opened their first console tab.

The session name has to carry the incident ticket reference, and I'd enforce that at the API level rather than trust people to remember:

{
  "Effect": "Allow",
  "Action": "sts:AssumeRole",
  "Resource": "arn:aws:iam::ACCOUNT_ID:role/BreakGlass-Production",
  "Condition": {
    "StringLike": {
      "sts:RoleSessionName": "incident-*"
    }
  }
}

This sits on the IAM identity or permission set allowed to assume the break-glass role. Without the incident- prefix, the assume call just fails, it's not a policy someone can forget to follow, it's a condition the API enforces.

Every one of these sessions gets a CloudTrail review within 24 hours of the incident closing.

Here's the metric I actually pay attention to: not whether break-glass exists, but how often it gets used. Once a month without a follow-up process fix, and I'd say the underlying RBAC model has a real gap somewhere.

Verification

I think of RBAC as working when I can answer yes to these without having to go digging first:

• Can every team deploy their service without standing administrator rights?
• Is every production change attributable to a named person or a named pipeline identity?
• Do access reviews happen on a fixed schedule, not just after an audit forces the question?
• When an engineer left last month, was their access gone within 24 hours?
• Is break-glass rare enough that each use is genuinely notable?
• Does a new service inherit a default role pattern, or does someone end up inventing permissions from scratch?

Any no in that list points to where the actual gap is. I'd fix it there rather than bolt on more monitoring to compensate.

What to avoid

Shared root or admin accounts kept around for everyday convenience. Static access keys on anything automated, when OIDC exists and costs nothing extra. Emergency access that quietly outlives the emergency. Roles broad enough that people default to them instead of asking for the right permission, and roles so narrow that teams route around the whole model with local secrets instead. Treating identity admin rights as a shortcut to dodge pipeline rework. And access reviews that only happen because a compliance deadline is forcing the issue.

If I'm honest, most of the RBAC failures I've seen trace back to one of these, not to some exotic attack. The basics just weren't held consistently.