**Problem** * While adding e2e tests for AssumeRole functionality, it was noticed that some of the static credential e2e tests were flaky. * The tests would sometimes fail with an AuthFailure. **Solution** * It was discovered that this error happened sometimes after a static credential rotation and that AuthFailure occurs if EC2 is invoked with the wrong credentials. * Due to IAM's eventual consistency model, it can take some time (up to several seconds) for changes, including rotated credentials, to get propagated to all AWS services, including EC2. * The tests were intermittently failing because sometimes the credentials would be propagated in time before invoking EC2 and other times they were not propagated in time. * The fix is to add a retry mechanism when calling DescribeInstances. A 30 second window is defined in which the retries can take place. * The retries happen only for AuthFailures. Any other failure will not trigger a retry. **Testing** * After making this change, the intermittent failures were no longer seen for the e2e tests * The e2e tests cover updating/creating catalogs and sets as well us updating the host list. All pass. * New unit tests were added to test the retry mechanism. Support had to be added to mock a series of DescribeInstances calls vs a single call * New and existing unit tests passed
Getting Started
Refer to Attributes and Secrets for more detail on configuration options for host catalogs and sets.
To create a host catalog (using default scope created by boundary dev):
boundary host-catalogs create plugin \
-scope-id p_1234567890 \
-name "Example Plugin-Based Host Catalog" \
-description "Description for plugin-based host catalog" \
-plugin-name aws \
-attr region=REGION \
-secret access_key_id='KEY' \
-secret secret_access_key='SECRET'
To create a host set, filtering the host set based on tag keys foo or bar
(either tag can be present), ensuring that any targets set to this host set only
connect to external addresses in the 54.0.0.0/8 class A subnet:
boundary host-sets create plugin \
-host-catalog-id HOST_CATALOG_ID \
-name "Example Plugin-Based Host Set" \
-description "Description for plugin-based host set" \
-attr filters=tag-key=foo,bar \
-preferred-endpoint "cidr:54.0.0.0/8"
As above, but instances must have both tags (both foo and bar must be
present):
boundary host-sets create plugin \
-host-catalog-id HOST_CATALOG_ID \
-name "Example Plugin-Based Host Set" \
-description "Description for plugin-based host set" \
-attr filters=tag-key=foo \
-attr filters=tag-key=bar \
-preferred-endpoint "cidr:54.0.0.0/8"
As above, but matching on tag key named "development" with value "prod" or "dev" and launch date:
boundary host-sets create plugin \
-host-catalog-id HOST_CATALOG_ID \
-name "Example Plugin-Based Host Set" \
-description "Description for plugin-based host set" \
-attr filters=tag:development=prod,dev \
-attr filters=launch-time=2022-01-04T* \
-preferred-endpoint "cidr:54.0.0.0/8"
Required IAM Privileges
The following IAM privileges, at the very least, are required to be attached to a configured IAM user for this provider:
ec2:DescribeInstances, configured to*(DescribeInstancescannot be scoped to a resource ARN). Example policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"ec2:DescribeInstances"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
iam:GetUser,iam:CreateAccessKey, andiam:DeleteAccessKey, configured to the IAM user to allow credential rotation. Example policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"iam:DeleteAccessKey",
"iam:GetUser",
"iam:CreateAccessKey"
],
"Effect": "Allow",
"Resource": "arn:aws:iam::123456789012:user/JohnDoe"
}
]
}
Attributes and Secrets
Host Catalog
The following attributes are valid on an AWS host catalog resource:
disable_credential_rotation(bool): Iftrue, credential rotation will not be performed. See the Credential Rotation readme for more information.region(string): The region to configure the host catalog for. All host sets in this catalog will be configured for this region.role_arn(string): The AWS Role ARN used forAssumeRoleauthentication. If provided,disable_credential_rotationmust be also set totrue.role_external_id(string): The external ID configured for the principalAssumeRoleprovider.role_session_name(string): The session name configured for the principalAssumeRoleprovider.role_tags(object): The key-value pair tags configured for the principalAssumeRoleprovider.target_role_arn(string): The AWS role ARN of the target IAM role to assume for cross-account discovery. Required if any othertarget_*attributes are configured.target_region(string): The region where the target hosts reside. Defaults to the principal accountregionif omitted.target_role_external_id(string): The external ID configured for the targetAssumeRoleprovider.target_role_session_name(string): The session name configured for the targetAssumeRoleprovider.target_role_tags(object): The key-value pair tags configured for the targetAssumeRoleprovider.
The following secrets are required on an AWS host catalog resource:
access_key_id(string): The access key ID for the IAM user to use with this host catalog.secret_access_key(string): The secret access key for the IAM user to use with this host catalog.
See the Credential Rotation readme for more information.
Host Set
The following attributes are valid on an AWS host Set resource:
filters(array of strings): An array of string filters in the formatkey=val1,val2. The key corresponds to a filter option, and the value(s) are a comma-separated list. For a list of filter options, check out describe-instances in the AWS CLI reference.