Using clusterawsadm to fulfill prerequisites
Requirements
IAM resources
With clusterawsadm
Get the latest clusterawsadm and place it in your path.
Cluster API Provider AWS ships with clusterawsadm, a utility to help you manage IAM objects for this project.
In order to use clusterawsadm you must have an administrative user in an AWS account. Once you have that administrator user you need to set your environment variables:
AWS_REGION
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_SESSION_TOKEN
(if you are using Multi-factor authentication)
After these are set run this command to get you up and running:
clusterawsadm bootstrap iam create-cloudformation-stack
Additional policies can be added by creating a configuration file
apiVersion: bootstrap.aws.infrastructure.cluster.x-k8s.io/v1beta1
kind: AWSIAMConfiguration
spec:
controlPlane:
extraPolicyAttachments:
- arn:aws:iam::<AWS_ACCOUNT>:policy/my-policy
- arn:aws:iam::aws:policy/AmazonEC2FullAccess
nodes:
extraPolicyAttachments:
- arn:aws:iam::<AWS_ACCOUNT>:policy/my-other-policy
and passing it to clusterawsadm as follows
clusterawsadm bootstrap iam create-cloudformation-stack --config bootstrap-config.yaml
These will be added to the control plane and node roles respectively when they are created.
Note: If you used the now deprecated
clusterawsadm alpha bootstrap
0.5.4 or earlier to create IAM objects for the Cluster API Provider for AWS, usingclusterawsadm bootstrap iam
0.5.5 or later will, by default, remove the bootstrap user and group. Anything using those credentials to authenticate will start experiencing authentication failures. If you rely on the bootstrap user and group credentials, specifybootstrapUser.enable = true
in the configuration file, like this:apiVersion: bootstrap.aws.infrastructure.cluster.x-k8s.io/v1beta1 kind: AWSIAMConfiguration spec: bootstrapUser: enable: true
With EKS Support
The pre-requisities for EKS are enabled by default. However, if you want to use some of the optional features of EKS (see here for more information on what these are) then you will need to enable these features via the configuration file. For example:
apiVersion: bootstrap.aws.infrastructure.cluster.x-k8s.io/v1beta1
kind: AWSIAMConfiguration
spec:
eks:
iamRoleCreation: false # Set to true if you plan to use the EKSEnableIAM feature flag to enable automatic creation of IAM roles
managedMachinePool:
disable: false # Set to false to enable creation of the default node role for managed machine pools
fargate:
disable: false # Set to false to enable creation of the default role for the fargate profiles
and then use that configuration file:
clusterawsadm bootstrap iam create-cloudformation-stack --config bootstrap-config.yaml
Enabling EventBridge Events
To enable EventBridge instance state events, additional permissions must be granted along with enabling the feature-flag. Additional permissions for events and queue management can be enabled through the configuration file as follows:
apiVersion: bootstrap.aws.infrastructure.cluster.x-k8s.io/v1beta1
kind: AWSIAMConfiguration
spec:
...
eventBridge:
enable: true
...
Cross Account Role Assumption
CAPA, by default, does not provide the necessary permissions to allow cross-account role assumption, which can be used to manage clusters in other environments. This is documented here. The ‘sts:AssumeRole’ permissions can be added via the following configuration on the manager account configuration:
apiVersion: bootstrap.aws.infrastructure.cluster.x-k8s.io/v1beta1
kind: AWSIAMConfiguration
spec:
...
allowAssumeRole: true
...
The above will give the controller to have the necessary permissions needed in order for it to manage clusters in other accounts using the AWSClusterRoleIdentity. Please note, the above should only be applied to the account where CAPA is running. To allow CAPA to assume the roles in the managed/target accounts, the following configuration needs to be used:
apiVersion: bootstrap.aws.infrastructure.cluster.x-k8s.io/v1beta1
kind: AWSIAMConfiguration
spec:
...
clusterAPIControllers:
disabled: false
trustStatements:
- Action:
- "sts:AssumeRole"
Effect: "Allow"
Principal:
AWS:
- "arn:aws:iam::<manager account>:role/controllers.cluster-api-provider-aws.sigs.k8s.io"
...
Without clusterawsadm
This is not a recommended route as the policies are very specific and will change with new features.
If you do not wish to use the clusteradwsadm
tool then you will need to
understand exactly which IAM policies and groups we are expecting. There are
several policies, roles and users that need to be created. Please see our
controller policy file to understand the permissions that are necessary.
You can use clusteradwadm
to print out the needed IAM policies, e.g.
clusterawsadm bootstrap iam print-policy --document AWSIAMManagedPolicyControllers --config bootstrap-config.yaml
SSH Key pair
If you plan to use SSH to access the instances created by Cluster API Provider AWS then you will need to specify the name of an existing SSH key pair within the region you plan on using. If you don’t have one yet, a new one needs to be created.
Create a new key pair
# Save the output to a secure location
aws ec2 create-key-pair --key-name default --output json | jq .KeyMaterial -r
-----BEGIN RSA PRIVATE KEY-----
[... contents omitted ...]
-----END RSA PRIVATE KEY-----
If you want to save the private key directly into AWS Systems Manager Parameter Store with KMS encryption for security, you can use the following command:
aws ssm put-parameter --name "/sigs.k8s.io/cluster-api-provider-aws/ssh-key" \
--type SecureString \
--value "$(aws ec2 create-key-pair --key-name default --output json | jq .KeyMaterial -r)"
Adding an existing public key to AWS
# Replace with your own public key
aws ec2 import-key-pair \
--key-name default \
--public-key-material "$(cat ~/.ssh/id_rsa.pub)"
NB: Only RSA keys are supported by AWS.
Setting up the environment
The current iteration of the Cluster API Provider AWS relies on credentials being present in your environment. These then get written into the cluster manifests for use by the controllers.
E.g.
export AWS_REGION=us-east-1 # This is used to help encode your environment variables
export AWS_ACCESS_KEY_ID=<your-access-key>
export AWS_SECRET_ACCESS_KEY=<your-secret-access-key>
export AWS_SESSION_TOKEN=<session-token> # If you are using Multi-Factor Auth.
Note: The credentials used must have the appropriate permissions for use by the controllers. You can get the required policy statement by using the following command:
clusterawsadm bootstrap iam print-policy --document AWSIAMManagedPolicyControllers --config bootstrap-config.yaml
To save credentials securely in your environment, aws-vault uses the OS keystore as permanent storage, and offers shell features to securely expose and setup local AWS environments.