Using Serverless Framework with permission boundaries.

Not authorized to CreateRole. A short post mortem.

Some time ago, we’ve changed the way we authenticate into our Amazon accounts. We wanted to streamline how to add or revoke developer access and make our accounts more secure.

The unseen consequence of this was an error that kept popping up whenever we tried to deploy a fresh app stack using Serverless Framework.

An error occurred: IamRoleCustomResourcesLambdaExecution - API: iam:CreateRole User: <deployer-arn> is not authorized to perform: iam:CreateRole on resource: <arn>-IamRoleCustomResourcesLa-XYZ

This caused the deployment to rollback and made it impossible for us to deploy fresh environments. The error didn’t happen when deploying to existing apps, which is why the misconfiguration went unnoticed.

After consulting, I’ve learned that this is due to our Serverless trying to create a role without something called a „permission boundary”. Permission boundaries limit what permissions a thing can give when it tries to create other things. It is used as a compliance and security feature. Additionally, we can deny any creations by entities without permission boundary. And this explicit deny is what causes the error.

In this example the user making the deploy (deployer-arn) was trying to create a role, but couldn’t because it didn’t have a boundary set and was being explicitly denied.

The solution to this problem is to add the permission boundary ARN in the CloudFormation generated by Serverless.

resources:
  extensions:
    IamRoleCustomResourcesLambdaExecution:
      Properties:
        PermissionsBoundary: <permission-boundary-policy-arn>
    IamRoleLambdaExecution:
      Properties:
        PermissionsBoundary: <permission-boundary-policy-arn>
    AppIamRoleLambdaExecution:
      Properties:
        PermissionsBoundary: <permission-boundary-policy-arn>

This will extend those resources with the permission boundary. This will bypass the „explicitly deny whenever things don’t have boundary” policy. The first two roles (IamRoleCustomResourcesLambdaExecution and IamRoleLambdaExecution) were being created by the Serverless Framework, while AppIamRoleLambdaExecution was made by one of the plugins we’ve used.