Compare commits

...
Author SHA1 Message Date
Matt Romanelli f81151b58c bug(go) AuthFailure when performing static credential rotation
**Problem**
* While adding e2e tests for AssumeRole functionality, it was noticed that some of the static credential e2e tests were flaky.
* The tests would sometimes fail with an AuthFailure.

**Solution**
* It was discovered that this error happened sometimes after a static credential rotation and that AuthFailure occurs if EC2 is invoked with the wrong credentials.
* Due to IAM's eventual consistency model, it can take some time (up to several seconds) for changes, including rotated credentials, to get propagated to all AWS services, including EC2.
* The tests were intermittently failing because sometimes the credentials would be propagated in time before invoking EC2 and other times they were not propagated in time.
* The fix is to add a retry mechanism when calling DescribeInstances. A 30 second window is defined in which the retries can take place.
* The retries happen only for AuthFailures. Any other failure will not trigger a retry.

**Testing**
* After making this change, the intermittent failures were no longer seen for the e2e tests
* The e2e tests cover updating/creating catalogs and sets as well us updating the host list. All pass.
* New unit tests were added to test the retry mechanism. Support had to be added to mock a series of DescribeInstances calls vs a single call
* New and existing unit tests passed
2026-09-22 12:27:00 -05:00
Matt Romanelli e484c09479 Merge pull request #83 from hashicorp/readme-updates
Updated the READMEs for cross-account discovery
2026-09-21 15:50:44 -05:00
Ryan Flynn 4126a0dbfc feat(cli) Implement infra to support cross account AssumeRole for DHC on ListHosts (#86) 2026-09-18 14:33:31 -04:00
Ryan Flynn 09dd7f8a4d feat(cli) Implement parsing of target-prefixed attrs for cross account dynamic host catalogs (#84) 2026-09-18 12:17:24 -04:00
Matt Romanelli 4c4a901d33 Updated the READMEs for cross-account discovery 2026-09-16 16:42:17 -05:00
Susmitha Girumala 163ec69771 Update github test workflow permissions to read-only (#76)
Set GITHUB_TOKEN to read-only permission by default
2026-07-14 15:31:58 -04:00
Piotr KazmierczakandHugo e8a0b0bf10 feat(storage): support multipart uploads (#72)
This changeset implements multipart uploads. AWS caps PutObject calls at 5
GiB, effectively causing the plugin to error on any upload larger than that.
Multipart limit is 8 TiB, which should be plenty for most use cases.

Multipart implementation is size-gated: we only switch to multipart uploads once
the file size is greater than multipartThreshold, which is currently set to 8
MiB, and we set the part size to 8 MiB as well (this follows AWS CLI and boto3).
That gives us ~78 GiB max object before we auto-scale, due to 10,000 part cap on
AWS.

---------

Co-authored-by: Hugo <10965479+hugoghx@users.noreply.github.com>
2026-06-26 16:34:32 +02:00
Damian Debkowski 4c8e95fcd8 fix: improve http transport (#73) 2026-06-24 07:12:41 -07:00
Louis Ruch 01f8da94e4 feat(host): add catalog attribute to only sync instance primary addresses (#71)
* feat(host): add catalog attribute to only sync instance primary
addresses
2026-05-27 13:12:34 -07:00
21 changed files with 1969 additions and 55 deletions

No files matched your search

+2
View File
@@ -4,6 +4,8 @@ on:
- workflow_dispatch - workflow_dispatch
- push - push
permissions: read-all
jobs: jobs:
test: test:
runs-on: ubuntu-latest runs-on: ubuntu-latest
+2
View File
@@ -25,6 +25,8 @@ Credential rotation can be turned off by setting the
This plugin will attempt to assume a role when a `role_arn` is supplied through the This plugin will attempt to assume a role when a `role_arn` is supplied through the
`attributes` object. More information about assume an AWS role can be found [here](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html). This feature only works when the plugin is running on a self managed Boundary [worker](https://developer.hashicorp.com/boundary/tutorials/hcp-administration/hcp-manage-workers). `attributes` object. More information about assume an AWS role can be found [here](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html). This feature only works when the plugin is running on a self managed Boundary [worker](https://developer.hashicorp.com/boundary/tutorials/hcp-administration/hcp-manage-workers).
To discover hosts across multiple AWS accounts, supply a `target_role_arn` in addition to `role_arn`. The plugin uses the principal credentials to assume the target role via a two-hop `AssumeRole` chain, enabling cross-account host discovery from a single worker. See [Attributes and Secrets](plugin/service/host/README.md#attributes-and-secrets) for the full list of `target_*` attributes.
### Environment Credentials ### Environment Credentials
This plugin will attempt to retrieve credentials from environment variables. More This plugin will attempt to retrieve credentials from environment variables. More
+1 -1
View File
@@ -4,6 +4,7 @@ go 1.25.7
require ( require (
github.com/aws/aws-sdk-go-v2 v1.41.7 github.com/aws/aws-sdk-go-v2 v1.41.7
github.com/aws/aws-sdk-go-v2/credentials v1.17.46
github.com/aws/aws-sdk-go-v2/service/ec2 v1.254.0 github.com/aws/aws-sdk-go-v2/service/ec2 v1.254.0
github.com/aws/aws-sdk-go-v2/service/iam v1.47.6 github.com/aws/aws-sdk-go-v2/service/iam v1.47.6
github.com/aws/aws-sdk-go-v2/service/s3 v1.101.0 github.com/aws/aws-sdk-go-v2/service/s3 v1.101.0
@@ -26,7 +27,6 @@ require (
github.com/apparentlymart/go-textseg/v15 v15.0.0 // indirect github.com/apparentlymart/go-textseg/v15 v15.0.0 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.10 // indirect github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.10 // indirect
github.com/aws/aws-sdk-go-v2/config v1.28.5 // indirect github.com/aws/aws-sdk-go-v2/config v1.28.5 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.17.46 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.20 // indirect github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.20 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.23 // indirect github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.23 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.23 // indirect github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.23 // indirect
+60
View File
@@ -186,3 +186,63 @@ func GetCredentialAttributes(in *structpb.Struct) (*CredentialAttributes, error)
RoleTags: roleTags, RoleTags: roleTags,
}, nil }, nil
} }
// GetTargetCredentialAttributes parses values out of a protobuf struct input and returns a CredentialAttributes used
// for configuring a second AWS session. target_role_arn is required when parsing target credentials.
// A status error is returned with an InvalidArgument code when target account attributes are missing
// required values or have invalid value types.
func GetTargetCredentialAttributes(in *structpb.Struct) (*CredentialAttributes, error) {
badFields := make(map[string]string)
// Parse the role ARN of the target account to AssumeRole into.
targetRoleArn, err := values.GetStringValue(in, ConstTargetRoleArn, true)
if err != nil {
badFields[fmt.Sprintf("attributes.%s", ConstTargetRoleArn)] = err.Error()
}
// try parsing target_region, fallback to region, adds target_region to bad fields only when malformed since not required
targetRegion, err := values.GetStringValue(in, ConstTargetRegion, false)
if err != nil {
badFields[fmt.Sprintf("attributes.%s", ConstTargetRegion)] = err.Error()
}
if targetRegion == "" {
targetRegion, err = values.GetStringValue(in, ConstRegion, true) // region is required for processing
if err != nil {
badFields[fmt.Sprintf("attributes.%s", ConstRegion)] = err.Error()
}
}
// the remaining fields are optional and only raise errors when malformed
disableCredentialRotation, err := values.GetBoolValue(in, ConstDisableCredentialRotation, false)
if err != nil {
badFields[fmt.Sprintf("attributes.%s", ConstDisableCredentialRotation)] = err.Error()
}
targetRoleExternalId, err := values.GetStringValue(in, ConstTargetRoleExternalId, false)
if err != nil {
badFields[fmt.Sprintf("attributes.%s", ConstTargetRoleExternalId)] = err.Error()
}
targetRoleSessionName, err := values.GetStringValue(in, ConstTargetRoleSessionName, false)
if err != nil {
badFields[fmt.Sprintf("attributes.%s", ConstTargetRoleSessionName)] = err.Error()
}
targetRoleTags, err := values.GetMapStringString(in, ConstTargetRoleTags, false)
if err != nil {
badFields[fmt.Sprintf("attributes.%s", ConstTargetRoleTags)] = err.Error()
}
if len(badFields) > 0 {
return nil, errors.InvalidArgumentError("Error in the attributes provided", badFields)
}
return &CredentialAttributes{
Region: targetRegion,
DisableCredentialRotation: disableCredentialRotation,
RoleArn: targetRoleArn,
RoleExternalId: targetRoleExternalId,
RoleSessionName: targetRoleSessionName,
RoleTags: targetRoleTags,
}, nil
}
+226
View File
@@ -101,6 +101,232 @@ func TestGetCredentialAttributes(t *testing.T) {
} }
} }
func TestGetTargetCredentialAttributes(t *testing.T) {
const targetRoleArn = "arn:aws:iam::222222222222:role/Target"
cases := []struct {
name string
in map[string]any
expected *CredentialAttributes
expectedErrContains string
}{
{
name: "missing target_role_arn",
in: map[string]any{
ConstRegion: "us-west-2",
},
expectedErrContains: "attributes.target_role_arn: missing required value \"target_role_arn\"",
},
{
name: "missing target_role_arn and region",
in: map[string]any{},
expectedErrContains: "attributes.target_role_arn: missing required value \"target_role_arn\"",
},
{
name: "empty target_role_arn",
in: map[string]any{
ConstRegion: "us-west-2",
ConstTargetRoleArn: "",
},
expectedErrContains: "attributes.target_role_arn: value \"target_role_arn\" cannot be empty",
},
{
name: "wrong type target_role_arn",
in: map[string]any{
ConstRegion: "us-west-2",
ConstTargetRoleArn: true,
},
expectedErrContains: "attributes.target_role_arn: unexpected type for value \"target_role_arn\": want string, got bool",
},
{
name: "target_region without target_role_arn",
in: map[string]any{
ConstRegion: "us-west-2",
ConstTargetRegion: "eu-west-1",
},
expectedErrContains: "attributes.target_role_arn: missing required value \"target_role_arn\"",
},
{
name: "other target fields without target_role_arn",
in: map[string]any{
ConstRegion: "us-west-2",
ConstTargetRoleExternalId: "ext",
ConstTargetRoleSessionName: "sess",
ConstTargetRoleTags: map[string]any{"k": "v"},
},
expectedErrContains: "attributes.target_role_arn: missing required value \"target_role_arn\"",
},
{
name: "target_role_arn uses region when target_region omitted",
in: map[string]any{
ConstRegion: "us-west-2",
ConstTargetRoleArn: targetRoleArn,
},
expected: &CredentialAttributes{
Region: "us-west-2",
RoleArn: targetRoleArn,
},
},
{
name: "target_role_arn without target_region or region",
in: map[string]any{
ConstTargetRoleArn: targetRoleArn,
},
expectedErrContains: "missing required value \"region\"",
},
{
name: "target_region overrides region",
in: map[string]any{
ConstRegion: "us-west-2",
ConstTargetRegion: "eu-west-1",
ConstTargetRoleArn: targetRoleArn,
},
expected: &CredentialAttributes{
Region: "eu-west-1",
RoleArn: targetRoleArn,
},
},
{
name: "empty target_region falls back to region",
in: map[string]any{
ConstRegion: "us-west-2",
ConstTargetRegion: "",
ConstTargetRoleArn: targetRoleArn,
},
expected: &CredentialAttributes{
Region: "us-west-2",
RoleArn: targetRoleArn,
},
},
{
name: "all optional target fields",
in: map[string]any{
ConstRegion: "us-west-2",
ConstDisableCredentialRotation: true,
ConstTargetRoleArn: targetRoleArn,
ConstTargetRegion: "eu-west-1",
ConstTargetRoleExternalId: "target-ext",
ConstTargetRoleSessionName: "target-sess",
ConstTargetRoleTags: map[string]any{"env": "target"},
},
expected: &CredentialAttributes{
Region: "eu-west-1",
DisableCredentialRotation: true,
RoleArn: targetRoleArn,
RoleExternalId: "target-ext",
RoleSessionName: "target-sess",
RoleTags: map[string]string{"env": "target"},
},
},
{
name: "optional target fields omitted do not copy principal role fields",
in: map[string]any{
ConstRegion: "us-west-2",
ConstRoleArn: "arn:aws:iam::111111111111:role/Principal",
ConstRoleExternalId: "principal-ext",
ConstRoleSessionName: "principal-sess",
ConstRoleTags: map[string]any{"env": "principal"},
ConstTargetRoleArn: targetRoleArn,
},
expected: &CredentialAttributes{
Region: "us-west-2",
RoleArn: targetRoleArn,
},
},
{
name: "disable_credential_rotation false is copied",
in: map[string]any{
ConstRegion: "us-west-2",
ConstDisableCredentialRotation: false,
ConstTargetRoleArn: targetRoleArn,
},
expected: &CredentialAttributes{
Region: "us-west-2",
DisableCredentialRotation: false,
RoleArn: targetRoleArn,
},
},
{
name: "wrong type target_region",
in: map[string]any{
ConstRegion: "us-west-2",
ConstTargetRoleArn: targetRoleArn,
ConstTargetRegion: true,
},
expectedErrContains: "attributes.target_region: unexpected type for value \"target_region\": want string, got bool",
},
{
name: "wrong type target_role_external_id",
in: map[string]any{
ConstRegion: "us-west-2",
ConstTargetRoleArn: targetRoleArn,
ConstTargetRoleExternalId: true,
},
expectedErrContains: "attributes.target_role_external_id: unexpected type for value \"target_role_external_id\": want string, got bool",
},
{
name: "wrong type target_role_session_name",
in: map[string]any{
ConstRegion: "us-west-2",
ConstTargetRoleArn: targetRoleArn,
ConstTargetRoleSessionName: true,
},
expectedErrContains: "attributes.target_role_session_name: unexpected type for value \"target_role_session_name\": want string, got bool",
},
{
name: "wrong type target_role_tags",
in: map[string]any{
ConstRegion: "us-west-2",
ConstTargetRoleArn: targetRoleArn,
ConstTargetRoleTags: true,
},
expectedErrContains: "attributes.target_role_tags: unexpected type for value \"target_role_tags\": want map[string]string, got bool",
},
{
name: "wrong type target_role_tag value",
in: map[string]any{
ConstRegion: "us-west-2",
ConstTargetRoleArn: targetRoleArn,
ConstTargetRoleTags: map[string]any{
"env": true,
},
},
expectedErrContains: `unexpected type for value in map["env"]: want string, got bool`,
},
{
name: "wrong type disable_credential_rotation",
in: map[string]any{
ConstRegion: "us-west-2",
ConstTargetRoleArn: targetRoleArn,
ConstDisableCredentialRotation: "sure",
},
expectedErrContains: "attributes.disable_credential_rotation: unexpected type for value \"disable_credential_rotation\": want bool, got string",
},
}
for _, tc := range cases {
tc := tc
t.Run(tc.name, func(t *testing.T) {
require := require.New(t)
input, err := structpb.NewStruct(tc.in)
require.NoError(err)
actual, err := GetTargetCredentialAttributes(input)
if tc.expectedErrContains != "" {
require.Error(err)
require.Nil(actual)
require.Contains(err.Error(), tc.expectedErrContains)
require.Equal(codes.InvalidArgument, status.Code(err))
return
}
require.NoError(err)
require.Equal(tc.expected, actual)
})
}
}
func TestGetCredentialsConfig(t *testing.T) { func TestGetCredentialsConfig(t *testing.T) {
cases := []struct { cases := []struct {
name string name string
+17 -2
View File
@@ -4,10 +4,10 @@
package credential package credential
const ( const (
// ConstAccessKey is the key for the region in the aws credentials. // ConstRegion is the key for the region in the aws credentials.
ConstRegion = "region" ConstRegion = "region"
// ConstAccessKey is the key for the access key id in the aws credentials. // ConstAccessKeyId is the key for the access key id in the aws credentials.
ConstAccessKeyId = "access_key_id" ConstAccessKeyId = "access_key_id"
// ConstSecretAccessKey is the key for the secret access key in the aws credentials. // ConstSecretAccessKey is the key for the secret access key in the aws credentials.
@@ -30,4 +30,19 @@ const (
// ConstRoleTags is the key for the tags used for assuming a IAM role. // ConstRoleTags is the key for the tags used for assuming a IAM role.
ConstRoleTags = "role_tags" ConstRoleTags = "role_tags"
// ConstTargetRegion is the key for the region of the target account in the aws credentials.
ConstTargetRegion = "target_region"
// ConstTargetRoleArn is the key for assuming a IAM role for the target account.
ConstTargetRoleArn = "target_role_arn"
// ConstTargetRoleExternalId is the key for the external id used for assuming a IAM role for the target account.
ConstTargetRoleExternalId = "target_role_external_id"
// ConstTargetRoleSessionName is the key for the session name used for assuming a IAM role of the target account.
ConstTargetRoleSessionName = "target_role_session_name"
// ConstTargetRoleTags is the key for the tags used for assuming a IAM role of the target account.
ConstTargetRoleTags = "target_role_tags"
) )
+5
View File
@@ -23,6 +23,11 @@ import (
) )
const ( const (
// AwsErrorAuthFailure is returned by EC2 when credentials exist in IAM
// but have not yet propagated to EC2. This is a transient eventual
// consistency issue that can resolve itself by retrying the request.
AwsErrorAuthFailure = "AuthFailure"
// awsErrorAccessDenied is returned when the credentials do not have // awsErrorAccessDenied is returned when the credentials do not have
// the required permissions to do a specific action on a resource. // the required permissions to do a specific action on a resource.
// This error will persist until the IAM Role Policy is updated by // This error will persist until the IAM Role Policy is updated by
+14 -4
View File
@@ -109,13 +109,23 @@ The following `attributes` are valid on an AWS host catalog resource:
- `region` (string): The region to configure the host catalog for. All host sets - `region` (string): The region to configure the host catalog for. All host sets
in this catalog will be configured for this region. in this catalog will be configured for this region.
- `role_arn` (string): The AWS Role ARN used for `AssumeRole` authentication. If - `role_arn` (string): The AWS Role ARN used for `AssumeRole` authentication. If
provided, `disable_credential_rotation` must be also set to `true. provided, `disable_credential_rotation` must be also set to `true`.
- `role_external_id` (string): The external id configured for the `AssumeRole` - `role_external_id` (string): The external ID configured for the principal `AssumeRole`
provider. provider.
- `role_session_name` (string): The session name configured for the `AssumeRole` - `role_session_name` (string): The session name configured for the principal `AssumeRole`
provider. provider.
- `role_tags` (object): The key-value pair tags configured for the `AssumeRole` - `role_tags` (object): The key-value pair tags configured for the principal `AssumeRole`
provider. provider.
- `target_role_arn` (string): The AWS role ARN of the target IAM role to assume for
cross-account discovery. Required if any other `target_*` attributes are configured.
- `target_region` (string): The region where the target hosts reside. Defaults to the
principal account `region` if omitted.
- `target_role_external_id` (string): The external ID configured for the target
`AssumeRole` provider.
- `target_role_session_name` (string): The session name configured for the target
`AssumeRole` provider.
- `target_role_tags` (object): The key-value pair tags configured for the target
`AssumeRole` provider.
The following `secrets` are required on an AWS host catalog resource: The following `secrets` are required on an AWS host catalog resource:
+52 -2
View File
@@ -20,8 +20,16 @@ import (
type CatalogAttributes struct { type CatalogAttributes struct {
*cred.CredentialAttributes *cred.CredentialAttributes
// TargetAccount optionally holds the credentials used to AssumeRole into a second, affiliated account when provided.
TargetAccount *cred.CredentialAttributes
// DualStack is used for configuring how the aws client will resolve requests. // DualStack is used for configuring how the aws client will resolve requests.
DualStack bool DualStack bool
// InstanceAddressesOnly specifies whether only the instance addresses (private IPv4/DNS,
// public IPv4/DNS, and IPv6) are synced, or all addresses, including those from
// secondary ENIs, are synced.
InstanceAddressesOnly bool
} }
func getCatalogAttributes(in *structpb.Struct) (*CatalogAttributes, error) { func getCatalogAttributes(in *structpb.Struct) (*CatalogAttributes, error) {
@@ -34,12 +42,26 @@ func getCatalogAttributes(in *structpb.Struct) (*CatalogAttributes, error) {
return nil, err return nil, err
} }
var targetCredAttributes *cred.CredentialAttributes
if hasTargetAccountAttributes(in) {
targetCredAttributes, err = cred.GetTargetCredentialAttributes(in)
if err != nil {
return nil, err
}
}
dualStack, err := values.GetBoolValue(in, ConstAwsDualStack, false) dualStack, err := values.GetBoolValue(in, ConstAwsDualStack, false)
if err != nil { if err != nil {
badFields[fmt.Sprintf("attributes.%s", ConstAwsDualStack)] = err.Error() badFields[fmt.Sprintf("attributes.%s", ConstAwsDualStack)] = err.Error()
} }
delete(unknownFields, ConstAwsDualStack) delete(unknownFields, ConstAwsDualStack)
instanceAddressesOnly, err := values.GetBoolValue(in, ConstInstanceAddressesOnly, false)
if err != nil {
badFields[fmt.Sprintf("attributes.%s", ConstInstanceAddressesOnly)] = err.Error()
}
delete(unknownFields, ConstInstanceAddressesOnly)
for s := range unknownFields { for s := range unknownFields {
switch s { switch s {
// Ignore knownFields from CredentialAttributes // Ignore knownFields from CredentialAttributes
@@ -55,6 +77,16 @@ func getCatalogAttributes(in *structpb.Struct) (*CatalogAttributes, error) {
continue continue
case cred.ConstRoleTags: case cred.ConstRoleTags:
continue continue
case cred.ConstTargetRegion:
continue
case cred.ConstTargetRoleArn:
continue
case cred.ConstTargetRoleExternalId:
continue
case cred.ConstTargetRoleSessionName:
continue
case cred.ConstTargetRoleTags:
continue
default: default:
badFields[fmt.Sprintf("attributes.%s", s)] = "unrecognized field" badFields[fmt.Sprintf("attributes.%s", s)] = "unrecognized field"
} }
@@ -65,8 +97,10 @@ func getCatalogAttributes(in *structpb.Struct) (*CatalogAttributes, error) {
} }
return &CatalogAttributes{ return &CatalogAttributes{
CredentialAttributes: credAttributes, CredentialAttributes: credAttributes,
DualStack: dualStack, TargetAccount: targetCredAttributes,
DualStack: dualStack,
InstanceAddressesOnly: instanceAddressesOnly,
}, nil }, nil
} }
@@ -109,3 +143,19 @@ func getSetAttributes(in *structpb.Struct) (*SetAttributes, error) {
return &setAttrs, nil return &setAttrs, nil
} }
func hasTargetAccountAttributes(in *structpb.Struct) bool {
fields := in.GetFields()
_, hasTargetRegion := fields[cred.ConstTargetRegion]
_, hasTargetRoleArn := fields[cred.ConstTargetRoleArn]
_, hasTargetRoleExternalID := fields[cred.ConstTargetRoleExternalId]
_, hasTargetRoleSessionName := fields[cred.ConstTargetRoleSessionName]
_, hasTargetRoleTags := fields[cred.ConstTargetRoleTags]
return hasTargetRegion ||
hasTargetRoleArn ||
hasTargetRoleExternalID ||
hasTargetRoleSessionName ||
hasTargetRoleTags
}
+180
View File
@@ -59,6 +59,22 @@ func TestGetCatalogAttributes(t *testing.T) {
DualStack: true, DualStack: true,
}, },
}, },
{
name: "withInstanceAddressesOnly",
in: &structpb.Struct{
Fields: map[string]*structpb.Value{
"region": structpb.NewStringValue("us-west-2"),
"instance_addresses_only": structpb.NewBoolValue(true),
},
},
expected: &CatalogAttributes{
CredentialAttributes: &credential.CredentialAttributes{
Region: "us-west-2",
DisableCredentialRotation: false,
},
InstanceAddressesOnly: true,
},
},
{ {
name: "default", name: "default",
in: &structpb.Struct{ in: &structpb.Struct{
@@ -102,6 +118,170 @@ func TestGetCatalogAttributes(t *testing.T) {
}, },
}, },
}, },
{
name: "target_region without target_role_arn",
in: &structpb.Struct{
Fields: map[string]*structpb.Value{
"region": structpb.NewStringValue("us-west-2"),
"target_region": structpb.NewStringValue("eu-west-1"),
},
},
expectedErrContains: "attributes.target_role_arn: missing required value \"target_role_arn\"",
},
{
name: "target role field without target_role_arn",
in: &structpb.Struct{
Fields: map[string]*structpb.Value{
"region": structpb.NewStringValue("us-west-2"),
"target_role_external_id": structpb.NewStringValue("target-ext"),
},
},
expectedErrContains: "attributes.target_role_arn: missing required value \"target_role_arn\"",
},
{
name: "other target role fields without target_role_arn",
in: &structpb.Struct{
Fields: map[string]*structpb.Value{
"region": structpb.NewStringValue("us-west-2"),
"target_role_session_name": structpb.NewStringValue("target-session"),
"target_role_tags": structpb.NewStructValue(&structpb.Struct{
Fields: map[string]*structpb.Value{
"env": structpb.NewStringValue("target"),
},
}),
},
},
expectedErrContains: "attributes.target_role_arn: missing required value \"target_role_arn\"",
},
{
name: "empty target_role_arn with target_region",
in: &structpb.Struct{
Fields: map[string]*structpb.Value{
"region": structpb.NewStringValue("us-west-2"),
"target_role_arn": structpb.NewStringValue(""),
"target_region": structpb.NewStringValue("eu-west-1"),
},
},
expectedErrContains: "attributes.target_role_arn: value \"target_role_arn\" cannot be empty",
},
{
name: "target_role_arn without target_region uses region; principal sts fields are not copied",
in: &structpb.Struct{
Fields: map[string]*structpb.Value{
"region": structpb.NewStringValue("us-west-2"),
"disable_credential_rotation": structpb.NewBoolValue(true),
"role_arn": structpb.NewStringValue("arn:aws:iam::111111111111:role/Principal"),
"role_external_id": structpb.NewStringValue("principal-ext"),
"role_session_name": structpb.NewStringValue("principal-sess"),
"role_tags": structpb.NewStructValue(&structpb.Struct{
Fields: map[string]*structpb.Value{
"env": structpb.NewStringValue("principal"),
},
}),
"target_role_arn": structpb.NewStringValue("arn:aws:iam::222222222222:role/Target"),
},
},
expected: &CatalogAttributes{
CredentialAttributes: &cred.CredentialAttributes{
Region: "us-west-2",
DisableCredentialRotation: true,
RoleArn: "arn:aws:iam::111111111111:role/Principal",
RoleExternalId: "principal-ext",
RoleSessionName: "principal-sess",
RoleTags: map[string]string{
"env": "principal",
},
},
TargetAccount: &cred.CredentialAttributes{
Region: "us-west-2",
DisableCredentialRotation: true,
RoleArn: "arn:aws:iam::222222222222:role/Target",
},
},
},
{
name: "target attributes are independent of principal",
in: &structpb.Struct{
Fields: map[string]*structpb.Value{
"region": structpb.NewStringValue("us-west-2"),
"disable_credential_rotation": structpb.NewBoolValue(true),
"role_arn": structpb.NewStringValue("arn:aws:iam::111111111111:role/Principal"),
"role_external_id": structpb.NewStringValue("principal-ext"),
"role_session_name": structpb.NewStringValue("principal-sess"),
"role_tags": structpb.NewStructValue(&structpb.Struct{
Fields: map[string]*structpb.Value{
"env": structpb.NewStringValue("principal"),
},
}),
"target_role_arn": structpb.NewStringValue("arn:aws:iam::222222222222:role/Target"),
"target_region": structpb.NewStringValue("eu-west-1"),
"target_role_external_id": structpb.NewStringValue("target-ext"),
"target_role_session_name": structpb.NewStringValue("target-sess"),
"target_role_tags": structpb.NewStructValue(&structpb.Struct{
Fields: map[string]*structpb.Value{
"env": structpb.NewStringValue("target"),
},
}),
},
},
expected: &CatalogAttributes{
CredentialAttributes: &cred.CredentialAttributes{
Region: "us-west-2",
DisableCredentialRotation: true,
RoleArn: "arn:aws:iam::111111111111:role/Principal",
RoleExternalId: "principal-ext",
RoleSessionName: "principal-sess",
RoleTags: map[string]string{
"env": "principal",
},
},
TargetAccount: &cred.CredentialAttributes{
Region: "eu-west-1",
DisableCredentialRotation: true,
RoleArn: "arn:aws:iam::222222222222:role/Target",
RoleExternalId: "target-ext",
RoleSessionName: "target-sess",
RoleTags: map[string]string{
"env": "target",
},
},
},
},
{
name: "catalog dual_stack and instance_addresses_only with target",
in: &structpb.Struct{
Fields: map[string]*structpb.Value{
"region": structpb.NewStringValue("us-west-2"),
"dual_stack": structpb.NewBoolValue(true),
"instance_addresses_only": structpb.NewBoolValue(true),
"target_role_arn": structpb.NewStringValue("arn:aws:iam::222222222222:role/Target"),
},
},
expected: &CatalogAttributes{
CredentialAttributes: &credential.CredentialAttributes{
Region: "us-west-2",
DisableCredentialRotation: false,
},
TargetAccount: &cred.CredentialAttributes{
Region: "us-west-2",
DisableCredentialRotation: false,
RoleArn: "arn:aws:iam::222222222222:role/Target",
},
DualStack: true,
InstanceAddressesOnly: true,
},
},
{
name: "unknown field with target_role_arn",
in: &structpb.Struct{
Fields: map[string]*structpb.Value{
"region": structpb.NewStringValue("us-west-2"),
"target_role_arn": structpb.NewStringValue("arn:aws:iam::222222222222:role/Target"),
"foo": structpb.NewBoolValue(true),
},
},
expectedErrContains: "attributes.foo: unrecognized field",
},
} }
for _, tc := range cases { for _, tc := range cases {
+5
View File
@@ -13,4 +13,9 @@ const (
// ConstAwsDualStack is the key for the dualstack flag in the aws s3 client. // ConstAwsDualStack is the key for the dualstack flag in the aws s3 client.
ConstAwsDualStack = "dual_stack" ConstAwsDualStack = "dual_stack"
// ConstInstanceAddressesOnly is the key (instance_addresses_only) that specifies whether
// only the instance addresses (private IPv4/DNS, public IPv4/DNS, and IPv6) are synced,
// or all addresses, including those from secondary ENIs, are synced.
ConstInstanceAddressesOnly = "instance_addresses_only"
) )
+183 -22
View File
@@ -5,12 +5,18 @@ package host
import ( import (
"context" "context"
goerrors "errors"
"fmt" "fmt"
"strings" "strings"
"time"
"github.com/aws/aws-sdk-go-v2/aws" "github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/credentials/stscreds"
"github.com/aws/aws-sdk-go-v2/service/ec2" "github.com/aws/aws-sdk-go-v2/service/ec2"
"github.com/aws/aws-sdk-go-v2/service/ec2/types" "github.com/aws/aws-sdk-go-v2/service/ec2/types"
"github.com/aws/aws-sdk-go-v2/service/sts"
stsTypes "github.com/aws/aws-sdk-go-v2/service/sts/types"
"github.com/aws/smithy-go"
"github.com/hashicorp/boundary-plugin-aws/internal/credential" "github.com/hashicorp/boundary-plugin-aws/internal/credential"
"github.com/hashicorp/boundary-plugin-aws/internal/errors" "github.com/hashicorp/boundary-plugin-aws/internal/errors"
pb "github.com/hashicorp/boundary/sdk/pbs/plugin" pb "github.com/hashicorp/boundary/sdk/pbs/plugin"
@@ -32,6 +38,16 @@ type HostPlugin struct {
testCatalogStateOpts []awsCatalogPersistedStateOption testCatalogStateOpts []awsCatalogPersistedStateOption
} }
type hostSetQuery struct {
Id string
Input *ec2.DescribeInstancesInput
Output *ec2.DescribeInstancesOutput
OutputHosts []*pb.ListHostsResponseHost
}
const defaultSessionName = "boundary-default"
const dryRunOperationErrorCode = "DryRunOperation"
// Ensure that we are implementing HostPluginServiceServer // Ensure that we are implementing HostPluginServiceServer
var _ pb.HostPluginServiceServer = (*HostPlugin)(nil) var _ pb.HostPluginServiceServer = (*HostPlugin)(nil)
@@ -495,13 +511,6 @@ func (p *HostPlugin) ListHosts(ctx context.Context, req *pb.ListHostsRequest) (*
} }
// Build all the queries in advance. // Build all the queries in advance.
type hostSetQuery struct {
Id string
Input *ec2.DescribeInstancesInput
Output *ec2.DescribeInstancesOutput
OutputHosts []*pb.ListHostsResponseHost
}
queries := make([]hostSetQuery, len(sets)) queries := make([]hostSetQuery, len(sets))
for i, set := range sets { for i, set := range sets {
// Validate Id since we use it in output // Validate Id since we use it in output
@@ -531,26 +540,28 @@ func (p *HostPlugin) ListHosts(ctx context.Context, req *pb.ListHostsRequest) (*
if catalogAttributes.DualStack { if catalogAttributes.DualStack {
opts = append(opts, WithDualStack(catalogAttributes.DualStack)) opts = append(opts, WithDualStack(catalogAttributes.DualStack))
} }
ec2Client, err := catalogState.EC2Client(ctx, opts...)
if err != nil {
return nil, errors.BadRequestStatusf("error getting EC2 client: %s", err)
}
// Run all queries now and assemble output. // Run all queries now and assemble output.
var maxLen int var maxLen int
for i, query := range queries { for i, query := range queries {
output, err := ec2Client.DescribeInstances(ctx, query.Input) output, err := checkHosts(ctx, catalogState, query.Input, opts, catalogAttributes.TargetAccount)
if err != nil { if err != nil {
return nil, errors.BadRequestStatusf("error running DescribeInstances for host set id %q: %s", query.Id, err) if catalogAttributes.TargetAccount != nil {
return nil, errors.BadRequestStatusf("unable to list hosts for host set %q: %s", query.Id, err)
}
return nil, errors.BadRequestStatusf("error retrieving host results for host set id %q: %s", query.Id, err)
} }
queries[i].Output = output queries[i].Output = output
if output == nil {
continue
}
// Process the output here, we will normalize this into a single // Process the output here, we will normalize this into a single
// set of hosts afterwards (possibly removing duplicates). // set of hosts afterwards (possibly removing duplicates).
for _, reservation := range output.Reservations { for _, reservation := range output.Reservations {
for _, instance := range reservation.Instances { for _, instance := range reservation.Instances {
host, err := awsInstanceToHost(instance) host, err := awsInstanceToHost(instance, catalogAttributes)
if err != nil { if err != nil {
return nil, errors.BadRequestStatusf("error processing host results for host set id %q: %s", query.Id, err) return nil, errors.BadRequestStatusf("error processing host results for host set id %q: %s", query.Id, err)
} }
@@ -648,10 +659,13 @@ func buildDescribeInstancesInput(attrs *SetAttributes, dryRun bool) (*ec2.Descri
// awsInstanceToHost processes data from an ec2.Instance and returns // awsInstanceToHost processes data from an ec2.Instance and returns
// a ListHostsResponseHost with the ID and network addresses // a ListHostsResponseHost with the ID and network addresses
// populated. // populated.
func awsInstanceToHost(instance types.Instance) (*pb.ListHostsResponseHost, error) { func awsInstanceToHost(instance types.Instance, catalogAttributes *CatalogAttributes) (*pb.ListHostsResponseHost, error) {
if aws.ToString(instance.InstanceId) == "" { if aws.ToString(instance.InstanceId) == "" {
return nil, fmt.Errorf("response integrity error: missing instance id") return nil, fmt.Errorf("response integrity error: missing instance id")
} }
if catalogAttributes == nil {
return nil, fmt.Errorf("response integrity error: missing catalog attributes")
}
result := new(pb.ListHostsResponseHost) result := new(pb.ListHostsResponseHost)
@@ -666,10 +680,16 @@ func awsInstanceToHost(instance types.Instance) (*pb.ListHostsResponseHost, erro
} }
} }
result.IpAddresses = appendDistinct(result.IpAddresses, instance.PrivateIpAddress, instance.PublicIpAddress) // First let's get the instance's Private IPv4/DNS, Public IPv4/DNS, and IPv6 addresses
result.IpAddresses = appendDistinct(result.IpAddresses, instance.PrivateIpAddress, instance.PublicIpAddress, instance.Ipv6Address)
result.DnsNames = appendDistinct(result.DnsNames, instance.PrivateDnsName, instance.PublicDnsName) result.DnsNames = appendDistinct(result.DnsNames, instance.PrivateDnsName, instance.PublicDnsName)
// Now go through all of the interfaces and log the IP address of if catalogAttributes.InstanceAddressesOnly {
// return early as we already have the instance addresses
return result, nil
}
// Now go through all of the interfaces and sync the IP address of
// every interface. // every interface.
for _, iface := range instance.NetworkInterfaces { for _, iface := range instance.NetworkInterfaces {
// Populate default IP addresses/DNS name similar to how we do // Populate default IP addresses/DNS name similar to how we do
@@ -698,10 +718,129 @@ func awsInstanceToHost(instance types.Instance) (*pb.ListHostsResponseHost, erro
return result, nil return result, nil
} }
func checkHosts(ctx context.Context,
principalCatalogState *awsCatalogPersistedState,
input *ec2.DescribeInstancesInput,
opts []ec2Option,
target *credential.CredentialAttributes,
) (*ec2.DescribeInstancesOutput, error) {
if principalCatalogState == nil {
return nil, fmt.Errorf("unable to create EC2 client from catalog state")
}
var ec2Client EC2API
var err error
if target == nil {
// use the principal account to execute the DescribeInstances command
ec2Client, err = principalCatalogState.EC2Client(ctx, opts...)
if err != nil {
return nil, err
}
} else {
// use the target account to execute the DescribeInstances command
ec2Client, err = ec2ClientForTarget(ctx, principalCatalogState, target, opts)
if err != nil {
return nil, fmt.Errorf("EC2 client setup failed for target account: %w", err)
}
}
output, err := ec2Client.DescribeInstances(ctx, input)
if err != nil {
var awsErr smithy.APIError
ok := goerrors.As(err, &awsErr)
if ok && awsErr.ErrorCode() == dryRunOperationErrorCode {
// successful DryRun validation, no response necessary, no error thrown
return nil, nil
}
if target != nil {
return nil, fmt.Errorf("EC2 DescribeInstances failed for target account: %w", err)
}
return nil, err
}
return output, nil
}
// ec2ClientForTarget builds an EC2 client to query the target account.
// The hop is a continuation of the principal session: STS AssumeRole is called
// as the principal to get to the target, then EC2 uses the resulting temporary credentials.
// This is not a second worker-level credential chain on the target attributes.
func ec2ClientForTarget(
ctx context.Context,
principalCatalogState *awsCatalogPersistedState,
target *credential.CredentialAttributes,
opts []ec2Option) (EC2API, error) {
// Reuse the principal SDK config (HTTP client, retries, logger, and the
// principal identity).
principalCfg, err := principalCatalogState.GenerateCredentialChain(ctx)
if err != nil {
return nil, fmt.Errorf("principal AWS configuration could not be loaded: %w", err)
}
if principalCfg == nil {
return nil, fmt.Errorf("nil principal AWS configuration while configuring EC2 client for target account")
}
// STS must run as the principal so the target role's trust policy sees that
// identity, not the worker's. Session name, external ID, and tags are
// forwarded from attrs when set so the AssumeRole matches the target role's configuration.
stsClient := sts.NewFromConfig(*principalCfg)
provider := stscreds.NewAssumeRoleProvider(stsClient, target.RoleArn, func(o *stscreds.AssumeRoleOptions) {
if target.RoleSessionName != "" {
o.RoleSessionName = target.RoleSessionName
} else {
// STS may reject the AssumeRole request if session name is not provided.
o.RoleSessionName = defaultSessionName
}
if target.RoleExternalId != "" {
o.ExternalID = &target.RoleExternalId
}
for k, v := range target.RoleTags {
o.Tags = append(o.Tags, stsTypes.Tag{
Key: aws.String(k),
Value: aws.String(v),
})
}
})
// Same worker SDK environment as the principal; only identity and region change.
targetCfg := principalCfg.Copy()
targetCfg.Credentials = aws.NewCredentialsCache(provider)
if target.Region != "" {
targetCfg.Region = target.Region
}
parsed, err := getOpts(opts...)
if err != nil {
return nil, fmt.Errorf("EC2 client options are invalid: %w", err)
}
// Dual-stack is the worker's path to AWS endpoints, not per-account. Apply
// it on the hopped client the same way EC2Client does for the principal.
var ec2Opts []func(*ec2.Options)
if parsed.withDualStack {
ec2Opts = append(ec2Opts, ec2.WithEndpointResolverV2(&endpointResolver{
dualStack: parsed.withDualStack,
}))
}
if principalCatalogState.testEC2APIFunc != nil {
return principalCatalogState.testEC2APIFunc(targetCfg)
}
return ec2.NewFromConfig(targetCfg, ec2Opts...), nil
}
// dryRunValidationTimeout is the maximum time to wait for credentials to
// propagate across AWS services after rotation before giving up.
const dryRunValidationTimeout = 30 * time.Second
// dryRunValidation performs an AWS DescribeInstances call to verify the state's // dryRunValidation performs an AWS DescribeInstances call to verify the state's
// credentials, the host listing functionality as well as the filters, if any // credentials, the host listing functionality as well as the filters, if any
// are passed in. This function can therefore be used for both host catalog and // are passed in. This function can therefore be used for both host catalog and
// host set validation. // host set validation.
//
// After credential rotation, IAM keys may be valid for STS (GetCallerIdentity)
// but not yet propagated to EC2. This function retries on AuthFailure to
// tolerate that eventual consistency window.
func dryRunValidation(ctx context.Context, state *awsCatalogPersistedState, ec2Opts []ec2Option, filters ...types.Filter) *status.Status { func dryRunValidation(ctx context.Context, state *awsCatalogPersistedState, ec2Opts []ec2Option, filters ...types.Filter) *status.Status {
if state == nil { if state == nil {
return status.New(codes.InvalidArgument, "persisted state is required") return status.New(codes.InvalidArgument, "persisted state is required")
@@ -712,12 +851,34 @@ func dryRunValidation(ctx context.Context, state *awsCatalogPersistedState, ec2O
return status.New(codes.InvalidArgument, fmt.Sprintf("error getting EC2 client: %s", err)) return status.New(codes.InvalidArgument, fmt.Sprintf("error getting EC2 client: %s", err))
} }
_, err = ec2Client.DescribeInstances(ctx, &ec2.DescribeInstancesInput{Filters: filters}) deadline := time.Now().Add(dryRunValidationTimeout)
if err != nil { delay := time.Second
return status.New(codes.FailedPrecondition, fmt.Sprintf("aws describe instances failed: %s", err)) for {
} _, err = ec2Client.DescribeInstances(ctx, &ec2.DescribeInstancesInput{Filters: filters})
if err == nil {
return nil
}
return nil // AuthFailure indicates the credentials exist but haven't propagated
// to EC2 yet — a transient eventual consistency issue after rotation.
// Retry until the deadline.
if strings.Contains(err.Error(), errors.AwsErrorAuthFailure) && time.Now().Before(deadline) {
select {
case <-time.After(delay):
// Wait before retrying to give credentials time to propagate.
case <-ctx.Done():
// The caller cancelled — break out of the select.
break
}
if ctx.Err() != nil {
break
}
continue
}
break
}
return status.New(codes.FailedPrecondition, fmt.Sprintf("aws describe instances failed: %s", err))
} }
// appendDistinct will append the elements to the slice // appendDistinct will append the elements to the slice
+403 -14
View File
@@ -18,6 +18,7 @@ import (
iamTypes "github.com/aws/aws-sdk-go-v2/service/iam/types" iamTypes "github.com/aws/aws-sdk-go-v2/service/iam/types"
"github.com/google/go-cmp/cmp" "github.com/google/go-cmp/cmp"
"github.com/hashicorp/boundary-plugin-aws/internal/credential" "github.com/hashicorp/boundary-plugin-aws/internal/credential"
pluginErrors "github.com/hashicorp/boundary-plugin-aws/internal/errors"
"github.com/hashicorp/boundary/sdk/pbs/controller/api/resources/hostcatalogs" "github.com/hashicorp/boundary/sdk/pbs/controller/api/resources/hostcatalogs"
"github.com/hashicorp/boundary/sdk/pbs/controller/api/resources/hostsets" "github.com/hashicorp/boundary/sdk/pbs/controller/api/resources/hostsets"
pb "github.com/hashicorp/boundary/sdk/pbs/plugin" pb "github.com/hashicorp/boundary/sdk/pbs/plugin"
@@ -638,7 +639,8 @@ func TestPluginOnCreateCatalogErr(t *testing.T) {
Fields: map[string]*structpb.Value{ Fields: map[string]*structpb.Value{
credential.ConstAccessKeyId: structpb.NewStringValue("AKIA_foobar"), credential.ConstAccessKeyId: structpb.NewStringValue("AKIA_foobar"),
credential.ConstSecretAccessKey: structpb.NewStringValue("bazqux"), credential.ConstSecretAccessKey: structpb.NewStringValue("bazqux"),
}}, },
},
Attrs: &hostcatalogs.HostCatalog_Attributes{ Attrs: &hostcatalogs.HostCatalog_Attributes{
Attributes: &structpb.Struct{ Attributes: &structpb.Struct{
Fields: map[string]*structpb.Value{ Fields: map[string]*structpb.Value{
@@ -664,7 +666,8 @@ func TestPluginOnCreateCatalogErr(t *testing.T) {
Fields: map[string]*structpb.Value{ Fields: map[string]*structpb.Value{
credential.ConstAccessKeyId: structpb.NewStringValue("AKIA_foobar"), credential.ConstAccessKeyId: structpb.NewStringValue("AKIA_foobar"),
credential.ConstSecretAccessKey: structpb.NewStringValue("bazqux"), credential.ConstSecretAccessKey: structpb.NewStringValue("bazqux"),
}}, },
},
Attrs: &hostcatalogs.HostCatalog_Attributes{ Attrs: &hostcatalogs.HostCatalog_Attributes{
Attributes: &structpb.Struct{ Attributes: &structpb.Struct{
Fields: map[string]*structpb.Value{ Fields: map[string]*structpb.Value{
@@ -694,7 +697,8 @@ func TestPluginOnCreateCatalogErr(t *testing.T) {
Fields: map[string]*structpb.Value{ Fields: map[string]*structpb.Value{
credential.ConstAccessKeyId: structpb.NewStringValue("AKIA_foobar"), credential.ConstAccessKeyId: structpb.NewStringValue("AKIA_foobar"),
credential.ConstSecretAccessKey: structpb.NewStringValue("bazqux"), credential.ConstSecretAccessKey: structpb.NewStringValue("bazqux"),
}}, },
},
Attrs: &hostcatalogs.HostCatalog_Attributes{ Attrs: &hostcatalogs.HostCatalog_Attributes{
Attributes: &structpb.Struct{ Attributes: &structpb.Struct{
Fields: map[string]*structpb.Value{ Fields: map[string]*structpb.Value{
@@ -721,7 +725,8 @@ func TestPluginOnCreateCatalogErr(t *testing.T) {
Fields: map[string]*structpb.Value{ Fields: map[string]*structpb.Value{
credential.ConstAccessKeyId: structpb.NewStringValue("AKIA_foobar"), credential.ConstAccessKeyId: structpb.NewStringValue("AKIA_foobar"),
credential.ConstSecretAccessKey: structpb.NewStringValue("bazqux"), credential.ConstSecretAccessKey: structpb.NewStringValue("bazqux"),
}}, },
},
Attrs: &hostcatalogs.HostCatalog_Attributes{ Attrs: &hostcatalogs.HostCatalog_Attributes{
Attributes: &structpb.Struct{ Attributes: &structpb.Struct{
Fields: map[string]*structpb.Value{ Fields: map[string]*structpb.Value{
@@ -1845,6 +1850,162 @@ func TestPluginOnUpdateSetErr(t *testing.T) {
} }
} }
func testListHostsSet(id, filter string) *hostsets.HostSet {
return &hostsets.HostSet{
Id: id,
Attrs: &hostsets.HostSet_Attributes{
Attributes: &structpb.Struct{
Fields: map[string]*structpb.Value{
ConstDescribeInstancesFilters: structpb.NewListValue(
&structpb.ListValue{
Values: []*structpb.Value{
structpb.NewStringValue(filter),
},
},
),
},
},
},
}
}
func testListHostsPersisted() *pb.HostCatalogPersisted {
return &pb.HostCatalogPersisted{
Secrets: &structpb.Struct{
Fields: map[string]*structpb.Value{
credential.ConstAccessKeyId: structpb.NewStringValue("AKIA_foobar"),
credential.ConstSecretAccessKey: structpb.NewStringValue("bazqux"),
},
},
}
}
func testExpectedDescribeInput(filterKey, filterValue string) *ec2.DescribeInstancesInput {
return &ec2.DescribeInstancesInput{
DryRun: aws.Bool(false),
Filters: []types.Filter{
{
Name: aws.String(filterKey),
Values: []string{filterValue},
},
{
Name: aws.String("instance-state-name"),
Values: []string{string(types.InstanceStateNameRunning)},
},
},
}
}
func TestPluginListHosts(t *testing.T) {
describeOutput := &ec2.DescribeInstancesOutput{
Reservations: []types.Reservation{
{
Instances: []types.Instance{
{
InstanceId: aws.String("i-shared"),
PrivateIpAddress: aws.String("10.0.0.1"),
},
},
},
},
}
cases := []struct {
name string
catalogFields map[string]*structpb.Value
sets []*hostsets.HostSet
expectedRegion string
expectedHosts []*pb.ListHostsResponseHost
expectedInputs []*ec2.DescribeInstancesInput
expectedDescribe int
}{
{
name: "principal account lists hosts",
catalogFields: map[string]*structpb.Value{
credential.ConstRegion: structpb.NewStringValue("us-west-2"),
},
sets: []*hostsets.HostSet{
testListHostsSet("set-1", "tag-key=foo"),
},
expectedRegion: "us-west-2",
expectedHosts: []*pb.ListHostsResponseHost{
{
ExternalId: "i-shared",
SetIds: []string{"set-1"},
},
},
expectedInputs: []*ec2.DescribeInstancesInput{
testExpectedDescribeInput("tag-key", "foo"),
},
expectedDescribe: 1,
},
{
name: "target account lists hosts without querying principal",
catalogFields: map[string]*structpb.Value{
credential.ConstRegion: structpb.NewStringValue("us-west-2"),
credential.ConstTargetRoleArn: structpb.NewStringValue("arn:aws:iam::222222222222:role/Target"),
credential.ConstTargetRegion: structpb.NewStringValue("eu-west-1"),
},
sets: []*hostsets.HostSet{
testListHostsSet("set-1", "tag-key=foo"),
testListHostsSet("set-2", "tag-key=bar"),
},
expectedRegion: "eu-west-1",
expectedHosts: []*pb.ListHostsResponseHost{
{
ExternalId: "i-shared",
SetIds: []string{"set-1", "set-2"},
},
},
expectedInputs: []*ec2.DescribeInstancesInput{
testExpectedDescribeInput("tag-key", "foo"),
testExpectedDescribeInput("tag-key", "bar"),
},
expectedDescribe: 2,
},
}
for _, tc := range cases {
tc := tc
t.Run(tc.name, func(t *testing.T) {
require := require.New(t)
state := &testMockEC2State{}
p := &HostPlugin{
testCatalogStateOpts: []awsCatalogPersistedStateOption{
withTestEC2APIFunc(newTestMockEC2(
state,
testMockEC2WithDescribeInstancesOutput(describeOutput),
)),
},
}
resp, err := p.ListHosts(context.Background(), &pb.ListHostsRequest{
Catalog: &hostcatalogs.HostCatalog{
Attrs: &hostcatalogs.HostCatalog_Attributes{
Attributes: &structpb.Struct{Fields: tc.catalogFields},
},
},
Persisted: testListHostsPersisted(),
Sets: tc.sets,
})
require.NoError(err)
require.NotNil(resp)
require.Len(resp.GetHosts(), len(tc.expectedHosts))
for i, expected := range tc.expectedHosts {
require.Equal(expected.ExternalId, resp.Hosts[i].ExternalId)
require.Equal(expected.SetIds, resp.Hosts[i].SetIds)
}
require.Equal(tc.expectedDescribe, state.DescribeInstancesCallCount)
require.Len(state.ClientRegions, tc.expectedDescribe)
for _, region := range state.ClientRegions {
require.Equal(tc.expectedRegion, region)
}
require.Equal(tc.expectedInputs, state.DescribeInstancesInputs)
})
}
}
func TestPluginListHostsErr(t *testing.T) { func TestPluginListHostsErr(t *testing.T) {
cases := []struct { cases := []struct {
name string name string
@@ -2126,7 +2287,101 @@ func TestPluginListHostsErr(t *testing.T) {
testMockEC2WithDescribeInstancesError(errors.New(testDescribeInstancesError)), testMockEC2WithDescribeInstancesError(errors.New(testDescribeInstancesError)),
)), )),
}, },
expectedErrContains: fmt.Sprintf("error running DescribeInstances for host set id \"foobar\": %s", testDescribeInstancesError), expectedErrContains: fmt.Sprintf("error retrieving host results for host set id \"foobar\": %s", testDescribeInstancesError),
expectedErrCode: codes.InvalidArgument,
},
{
name: "target attributes missing target_role_arn",
req: &pb.ListHostsRequest{
Catalog: &hostcatalogs.HostCatalog{
Attrs: &hostcatalogs.HostCatalog_Attributes{
Attributes: &structpb.Struct{
Fields: map[string]*structpb.Value{
credential.ConstRegion: structpb.NewStringValue("us-west-2"),
credential.ConstTargetRegion: structpb.NewStringValue("eu-west-1"),
},
},
},
},
},
expectedErrContains: "attributes.target_role_arn: missing required value \"target_role_arn\"",
expectedErrCode: codes.InvalidArgument,
},
{
name: "target EC2 client error",
req: &pb.ListHostsRequest{
Catalog: &hostcatalogs.HostCatalog{
Attrs: &hostcatalogs.HostCatalog_Attributes{
Attributes: &structpb.Struct{
Fields: map[string]*structpb.Value{
credential.ConstRegion: structpb.NewStringValue("us-west-2"),
credential.ConstTargetRoleArn: structpb.NewStringValue("arn:aws:iam::222222222222:role/Target"),
},
},
},
},
Persisted: &pb.HostCatalogPersisted{
Secrets: &structpb.Struct{
Fields: map[string]*structpb.Value{
credential.ConstAccessKeyId: structpb.NewStringValue("AKIA_foobar"),
credential.ConstSecretAccessKey: structpb.NewStringValue("bazqux"),
credential.ConstCredsLastRotatedTime: structpb.NewStringValue("2006-01-02T15:04:05+07:00"),
},
},
},
Sets: []*hostsets.HostSet{
{
Id: "foobar",
Attrs: &hostsets.HostSet_Attributes{
Attributes: &structpb.Struct{
Fields: map[string]*structpb.Value{
ConstDescribeInstancesFilters: structpb.NewListValue(
&structpb.ListValue{
Values: []*structpb.Value{
structpb.NewStringValue("tag-key=foo"),
},
},
),
},
},
},
},
},
},
catalogOpts: []awsCatalogPersistedStateOption{
withTestEC2APIFunc(func(...aws.Config) (EC2API, error) {
return nil, errors.New("test EC2 client error")
}),
},
expectedErrContains: "unable to list hosts for host set \"foobar\": EC2 client setup failed for target account: test EC2 client error",
expectedErrCode: codes.InvalidArgument,
},
{
name: "target DescribeInstances error",
req: &pb.ListHostsRequest{
Catalog: &hostcatalogs.HostCatalog{
Attrs: &hostcatalogs.HostCatalog_Attributes{
Attributes: &structpb.Struct{
Fields: map[string]*structpb.Value{
credential.ConstRegion: structpb.NewStringValue("us-west-2"),
credential.ConstTargetRoleArn: structpb.NewStringValue("arn:aws:iam::222222222222:role/Target"),
credential.ConstTargetRegion: structpb.NewStringValue("eu-west-1"),
},
},
},
},
Persisted: testListHostsPersisted(),
Sets: []*hostsets.HostSet{
testListHostsSet("foobar", "tag-key=foo"),
},
},
catalogOpts: []awsCatalogPersistedStateOption{
withTestEC2APIFunc(newTestMockEC2(
nil,
testMockEC2WithDescribeInstancesError(errors.New(testDescribeInstancesError)),
)),
},
expectedErrContains: "unable to list hosts for host set \"foobar\": EC2 DescribeInstances failed for target account: DescribeInstances error",
expectedErrCode: codes.InvalidArgument, expectedErrCode: codes.InvalidArgument,
}, },
{ {
@@ -2383,15 +2638,25 @@ func TestBuildDescribeInstancesInput(t *testing.T) {
func TestAwsInstanceToHost(t *testing.T) { func TestAwsInstanceToHost(t *testing.T) {
cases := []struct { cases := []struct {
name string name string
instance types.Instance instance types.Instance
expected *pb.ListHostsResponseHost catalogAttrs *CatalogAttributes
expectedErr string expected *pb.ListHostsResponseHost
expectedErr string
}{ }{
{ {
name: "missing instance id", name: "missing instance id",
instance: types.Instance{}, instance: types.Instance{},
expectedErr: "response integrity error: missing instance id", catalogAttrs: &CatalogAttributes{},
expectedErr: "response integrity error: missing instance id",
},
{
name: "missing catalog attributes",
instance: types.Instance{
InstanceId: aws.String("foobar"),
},
catalogAttrs: nil,
expectedErr: "response integrity error: missing catalog attributes",
}, },
{ {
name: "good, single IP w/public addr", name: "good, single IP w/public addr",
@@ -2418,6 +2683,7 @@ func TestAwsInstanceToHost(t *testing.T) {
}, },
}, },
}, },
catalogAttrs: &CatalogAttributes{},
expected: &pb.ListHostsResponseHost{ expected: &pb.ListHostsResponseHost{
ExternalId: "foobar", ExternalId: "foobar",
IpAddresses: []string{"10.0.0.1", "1.1.1.1"}, IpAddresses: []string{"10.0.0.1", "1.1.1.1"},
@@ -2443,6 +2709,7 @@ func TestAwsInstanceToHost(t *testing.T) {
}, },
}, },
}, },
catalogAttrs: &CatalogAttributes{},
expected: &pb.ListHostsResponseHost{ expected: &pb.ListHostsResponseHost{
ExternalId: "foobar", ExternalId: "foobar",
IpAddresses: []string{"10.0.0.1"}, IpAddresses: []string{"10.0.0.1"},
@@ -2484,6 +2751,7 @@ func TestAwsInstanceToHost(t *testing.T) {
}, },
}, },
}, },
catalogAttrs: &CatalogAttributes{},
expected: &pb.ListHostsResponseHost{ expected: &pb.ListHostsResponseHost{
ExternalId: "foobar", ExternalId: "foobar",
IpAddresses: []string{"10.0.0.1", "1.1.1.1", "10.0.0.2"}, IpAddresses: []string{"10.0.0.1", "1.1.1.1", "10.0.0.2"},
@@ -2529,6 +2797,7 @@ func TestAwsInstanceToHost(t *testing.T) {
}, },
}, },
}, },
catalogAttrs: &CatalogAttributes{},
expected: &pb.ListHostsResponseHost{ expected: &pb.ListHostsResponseHost{
ExternalId: "foobar", ExternalId: "foobar",
IpAddresses: []string{"10.0.0.1", "1.1.1.1", "10.0.0.2", "1.1.1.2"}, IpAddresses: []string{"10.0.0.1", "1.1.1.1", "10.0.0.2", "1.1.1.2"},
@@ -2564,6 +2833,7 @@ func TestAwsInstanceToHost(t *testing.T) {
}, },
}, },
}, },
catalogAttrs: &CatalogAttributes{},
expected: &pb.ListHostsResponseHost{ expected: &pb.ListHostsResponseHost{
ExternalId: "foobar", ExternalId: "foobar",
IpAddresses: []string{"10.0.0.1", "1.1.1.1", "10.0.0.2"}, IpAddresses: []string{"10.0.0.1", "1.1.1.1", "10.0.0.2"},
@@ -2578,6 +2848,7 @@ func TestAwsInstanceToHost(t *testing.T) {
PrivateDnsName: aws.String("test.example.internal"), PrivateDnsName: aws.String("test.example.internal"),
PublicIpAddress: aws.String("1.1.1.1"), PublicIpAddress: aws.String("1.1.1.1"),
PublicDnsName: aws.String("test.example.com"), PublicDnsName: aws.String("test.example.com"),
Ipv6Address: aws.String("instance::ipv6::address"),
NetworkInterfaces: []types.InstanceNetworkInterface{ NetworkInterfaces: []types.InstanceNetworkInterface{
{ {
PrivateIpAddress: aws.String("10.0.0.1"), PrivateIpAddress: aws.String("10.0.0.1"),
@@ -2596,9 +2867,63 @@ func TestAwsInstanceToHost(t *testing.T) {
}, },
}, },
}, },
catalogAttrs: &CatalogAttributes{},
expected: &pb.ListHostsResponseHost{ expected: &pb.ListHostsResponseHost{
ExternalId: "foobar", ExternalId: "foobar",
IpAddresses: []string{"10.0.0.1", "1.1.1.1", "some::fake::address"}, IpAddresses: []string{"10.0.0.1", "1.1.1.1", "instance::ipv6::address", "some::fake::address"},
DnsNames: []string{"test.example.internal", "test.example.com"},
},
},
{
name: "good, instance addresses only excludes interface secondary addresses",
instance: types.Instance{
InstanceId: aws.String("foobar"),
PrivateIpAddress: aws.String("10.0.0.1"),
PrivateDnsName: aws.String("test.example.internal"),
PublicIpAddress: aws.String("1.1.1.1"),
PublicDnsName: aws.String("test.example.com"),
Ipv6Address: aws.String("instance::ipv6::address"),
NetworkInterfaces: []types.InstanceNetworkInterface{
{
PrivateIpAddress: aws.String("10.0.0.1"),
PrivateDnsName: aws.String("test.example.internal"),
PrivateIpAddresses: []types.InstancePrivateIpAddress{
{
Association: &types.InstanceNetworkInterfaceAssociation{
PublicIp: aws.String("1.1.1.1"),
PublicDnsName: aws.String("test.example.com"),
},
PrivateIpAddress: aws.String("10.0.0.1"),
PrivateDnsName: aws.String("test.example.internal"),
},
{
PrivateIpAddress: aws.String("10.0.0.2"),
PrivateDnsName: aws.String("test2.example.internal"),
},
},
Ipv6Addresses: []types.InstanceIpv6Address{{Ipv6Address: aws.String("iface::ipv6::address")}},
},
{
PrivateIpAddress: aws.String("10.0.1.1"),
PrivateDnsName: aws.String("secondary.example.internal"),
PrivateIpAddresses: []types.InstancePrivateIpAddress{
{
Association: &types.InstanceNetworkInterfaceAssociation{
PublicIp: aws.String("2.2.2.2"),
PublicDnsName: aws.String("secondary.example.com"),
},
PrivateIpAddress: aws.String("10.0.1.1"),
PrivateDnsName: aws.String("secondary.example.internal"),
},
},
Ipv6Addresses: []types.InstanceIpv6Address{{Ipv6Address: aws.String("secondary::ipv6::address")}},
},
},
},
catalogAttrs: &CatalogAttributes{InstanceAddressesOnly: true},
expected: &pb.ListHostsResponseHost{
ExternalId: "foobar",
IpAddresses: []string{"10.0.0.1", "1.1.1.1", "instance::ipv6::address"},
DnsNames: []string{"test.example.internal", "test.example.com"}, DnsNames: []string{"test.example.internal", "test.example.com"},
}, },
}, },
@@ -2632,6 +2957,7 @@ func TestAwsInstanceToHost(t *testing.T) {
}, },
}, },
}, },
catalogAttrs: &CatalogAttributes{},
expected: &pb.ListHostsResponseHost{ expected: &pb.ListHostsResponseHost{
ExternalId: "foobar", ExternalId: "foobar",
ExternalName: "test-instance-actual-name", ExternalName: "test-instance-actual-name",
@@ -2669,6 +2995,7 @@ func TestAwsInstanceToHost(t *testing.T) {
}, },
}, },
}, },
catalogAttrs: &CatalogAttributes{},
expected: &pb.ListHostsResponseHost{ expected: &pb.ListHostsResponseHost{
ExternalId: "foobar", ExternalId: "foobar",
ExternalName: "test-instance-actual-name", ExternalName: "test-instance-actual-name",
@@ -2682,7 +3009,7 @@ func TestAwsInstanceToHost(t *testing.T) {
tc := tc tc := tc
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
require := require.New(t) require := require.New(t)
actual, err := awsInstanceToHost(tc.instance) actual, err := awsInstanceToHost(tc.instance, tc.catalogAttrs)
if tc.expectedErr != "" { if tc.expectedErr != "" {
require.EqualError(err, tc.expectedErr) require.EqualError(err, tc.expectedErr)
return return
@@ -2795,4 +3122,66 @@ func TestDryRunValidation(t *testing.T) {
}, []ec2Option{}) }, []ec2Option{})
require.Nil(t, st) require.Nil(t, st)
}) })
t.Run("authFailureThenSuccess", func(t *testing.T) {
// Simulates the IAM eventual consistency window: EC2 returns AuthFailure
// twice before the credentials propagate and the call succeeds.
state := &testMockEC2State{}
st := dryRunValidation(context.Background(), &awsCatalogPersistedState{
AwsCredentialPersistedState: &credential.AwsCredentialPersistedState{
CredentialsConfig: &awsutil.CredentialsConfig{
AccessKey: "AKIAfoo",
SecretKey: "baz",
},
},
testEC2APIFunc: newTestMockEC2(state,
testMockEC2WithDescribeInstancesErrorSequence(
fmt.Errorf("%s: credentials not yet propagated", pluginErrors.AwsErrorAuthFailure),
fmt.Errorf("%s: credentials not yet propagated", pluginErrors.AwsErrorAuthFailure),
),
testMockEC2WithDescribeInstancesOutput(&ec2.DescribeInstancesOutput{}),
),
}, []ec2Option{})
require.Nil(t, st)
require.Equal(t, 3, state.DescribeInstancesCallCount)
})
t.Run("authFailureContextCancelled", func(t *testing.T) {
// If the context is cancelled while waiting to retry, the function should
// return a FailedPrecondition error immediately rather than hanging.
ctx, cancel := context.WithCancel(context.Background())
cancel() // cancel immediately so the first retry select hits ctx.Done()
st := dryRunValidation(ctx, &awsCatalogPersistedState{
AwsCredentialPersistedState: &credential.AwsCredentialPersistedState{
CredentialsConfig: &awsutil.CredentialsConfig{
AccessKey: "AKIAfoo",
SecretKey: "baz",
},
},
testEC2APIFunc: newTestMockEC2(nil,
testMockEC2WithDescribeInstancesError(fmt.Errorf("%s: credentials not yet propagated", pluginErrors.AwsErrorAuthFailure)),
),
}, []ec2Option{})
require.NotNil(t, st)
require.Equal(t, codes.FailedPrecondition.String(), st.Code().String())
})
t.Run("nonAuthFailureNoRetry", func(t *testing.T) {
// A non-AuthFailure error should fail immediately without retrying.
state := &testMockEC2State{}
st := dryRunValidation(context.Background(), &awsCatalogPersistedState{
AwsCredentialPersistedState: &credential.AwsCredentialPersistedState{
CredentialsConfig: &awsutil.CredentialsConfig{
AccessKey: "AKIAfoo",
SecretKey: "baz",
},
},
testEC2APIFunc: newTestMockEC2(state,
testMockEC2WithDescribeInstancesError(fmt.Errorf("UnauthorizedOperation: access denied")),
),
}, []ec2Option{})
require.NotNil(t, st)
require.Equal(t, codes.FailedPrecondition.String(), st.Code().String())
require.Equal(t, 1, state.DescribeInstancesCallCount)
})
} }
+31
View File
@@ -22,11 +22,17 @@ const (
type testMockEC2State struct { type testMockEC2State struct {
DescribeInstancesCalled bool DescribeInstancesCalled bool
DescribeInstancesInputParams *ec2.DescribeInstancesInput DescribeInstancesInputParams *ec2.DescribeInstancesInput
DescribeInstancesCallCount int
DescribeInstancesInputs []*ec2.DescribeInstancesInput
ClientRegions []string
} }
func (s *testMockEC2State) Reset() { func (s *testMockEC2State) Reset() {
s.DescribeInstancesCalled = false s.DescribeInstancesCalled = false
s.DescribeInstancesInputParams = nil s.DescribeInstancesInputParams = nil
s.DescribeInstancesCallCount = 0
s.DescribeInstancesInputs = nil
s.ClientRegions = nil
} }
type testMockEC2 struct { type testMockEC2 struct {
@@ -37,6 +43,10 @@ type testMockEC2 struct {
DescribeInstancesOutput *ec2.DescribeInstancesOutput DescribeInstancesOutput *ec2.DescribeInstancesOutput
DescribeInstancesError error DescribeInstancesError error
// DescribeInstancesErrorSequence, if set, returns each error in order on
// successive calls. Once exhausted, DescribeInstancesOutput is returned.
DescribeInstancesErrorSequence []error
} }
type testMockEC2Option func(m *testMockEC2) error type testMockEC2Option func(m *testMockEC2) error
@@ -55,6 +65,16 @@ func testMockEC2WithDescribeInstancesError(e error) testMockEC2Option {
} }
} }
// testMockEC2WithDescribeInstancesErrorSequence sets a sequence of errors to
// return on successive DescribeInstances calls. Once the sequence is exhausted,
// the mock returns DescribeInstancesOutput (nil error).
func testMockEC2WithDescribeInstancesErrorSequence(errs ...error) testMockEC2Option {
return func(m *testMockEC2) error {
m.DescribeInstancesErrorSequence = errs
return nil
}
}
func newTestMockEC2(state *testMockEC2State, opts ...testMockEC2Option) ec2APIFunc { func newTestMockEC2(state *testMockEC2State, opts ...testMockEC2Option) ec2APIFunc {
return func(cfgs ...aws.Config) (EC2API, error) { return func(cfgs ...aws.Config) (EC2API, error) {
m := &testMockEC2{ m := &testMockEC2{
@@ -72,7 +92,10 @@ func newTestMockEC2(state *testMockEC2State, opts ...testMockEC2Option) ec2APIFu
if cfg.Region != "" { if cfg.Region != "" {
m.Region = cfg.Region m.Region = cfg.Region
} }
}
if state != nil {
state.ClientRegions = append(state.ClientRegions, m.Region)
} }
return m, nil return m, nil
@@ -83,6 +106,14 @@ func (m *testMockEC2) DescribeInstances(ctx context.Context, input *ec2.Describe
if m.State != nil { if m.State != nil {
m.State.DescribeInstancesCalled = true m.State.DescribeInstancesCalled = true
m.State.DescribeInstancesInputParams = input m.State.DescribeInstancesInputParams = input
m.State.DescribeInstancesCallCount++
m.State.DescribeInstancesInputs = append(m.State.DescribeInstancesInputs, input)
}
if len(m.DescribeInstancesErrorSequence) > 0 {
err := m.DescribeInstancesErrorSequence[0]
m.DescribeInstancesErrorSequence = m.DescribeInstancesErrorSequence[1:]
return nil, err
} }
if m.DescribeInstancesError != nil { if m.DescribeInstancesError != nil {
+1 -1
View File
@@ -11,5 +11,5 @@ const (
ConstAwsDualStack = "dual_stack" ConstAwsDualStack = "dual_stack"
// defaultStreamChunkSize is the recommened chunk size for sending data through a stream // defaultStreamChunkSize is the recommened chunk size for sending data through a stream
defaultStreamChunkSize = 65536 // 64 KiB defaultStreamChunkSize = 1 << 20 // 1 MiB
) )
+158 -4
View File
@@ -665,6 +665,23 @@ func (p *StoragePlugin) GetObject(req *pb.GetObjectRequest, stream pb.StoragePlu
return nil return nil
} }
const (
// multipartThreshold is the file size, in bytes, above which PutObject
// uploads an object using a multipart upload instead of a single PutObject
// request. 8 MiB matches the AWS CLI and boto3 defaults.
multipartThreshold = 8 * 1024 * 1024 // 8 MiB
// defaultMultipartPartSize is the default size, in bytes, of each part
// uploaded during a multipart upload. It is kept equal to
// multipartThreshold so a multipart upload always has at least two parts
// (S3 requires every part except the last to be at least 5 MiB).
defaultMultipartPartSize = 8 * 1024 * 1024 // 8 MiB
// maxMultipartParts is the maximum number of parts S3 permits for a single
// multipart upload.
maxMultipartParts = 10000
)
// PutObject is called when putting objects into an s3 bucket. // PutObject is called when putting objects into an s3 bucket.
func (p *StoragePlugin) PutObject(ctx context.Context, req *pb.PutObjectRequest) (*pb.PutObjectResponse, error) { func (p *StoragePlugin) PutObject(ctx context.Context, req *pb.PutObjectRequest) (*pb.PutObjectResponse, error) {
if req == nil { if req == nil {
@@ -751,6 +768,32 @@ func (p *StoragePlugin) PutObject(ctx context.Context, req *pb.PutObjectRequest)
} }
objectKey := path.Join(bucket.GetBucketPrefix(), req.GetKey()) objectKey := path.Join(bucket.GetBucketPrefix(), req.GetKey())
// decodedChecksum is the raw SHA-256 digest of the entire object. Boundary
// compares this byte-for-byte against the hash it computed locally before
// the upload, so it must always be the whole-object checksum regardless of
// whether a single PutObject or a multipart upload is used.
decodedChecksum, err := base64.StdEncoding.DecodeString(checksum)
if err != nil {
return nil, errors.UnknownStatus("failed to decode checksum value")
}
// for larger files, use a multipart upload instead of a single PutObject
// request.
if info.Size() > multipartThreshold {
partSize := int64(defaultMultipartPartSize)
// S3 allows at most maxMultipartParts parts per upload, so grow the
// part size for very large objects to stay within that limit.
if sz := info.Size(); sz/partSize >= maxMultipartParts {
partSize = (sz + maxMultipartParts - 1) / maxMultipartParts
}
if err := p.putObjectMultipart(ctx, req, storageState, bucket.GetBucketName(), objectKey, file, partSize, opts...); err != nil {
return nil, err
}
return &pb.PutObjectResponse{
ChecksumSha_256: decodedChecksum,
}, nil
}
putCall := func(s3Client S3API) (any, error) { putCall := func(s3Client S3API) (any, error) {
return s3Client.PutObject(ctx, &s3.PutObjectInput{ return s3Client.PutObject(ctx, &s3.PutObjectInput{
Bucket: aws.String(bucket.GetBucketName()), Bucket: aws.String(bucket.GetBucketName()),
@@ -772,15 +815,126 @@ func (p *StoragePlugin) PutObject(ctx context.Context, req *pb.PutObjectRequest)
if checksum != *resp.ChecksumSHA256 { if checksum != *resp.ChecksumSHA256 {
return nil, checksumMistmatchStatus() return nil, checksumMistmatchStatus()
} }
decodedChecksum, err := base64.StdEncoding.DecodeString(*resp.ChecksumSHA256)
if err != nil {
return nil, errors.UnknownStatus("failed to decode checksum value from aws")
}
return &pb.PutObjectResponse{ return &pb.PutObjectResponse{
ChecksumSha_256: decodedChecksum, ChecksumSha_256: decodedChecksum,
}, nil }, nil
} }
// putObjectMultipart uploads file to the given bucket and key using an S3
// multipart upload. The file is split into parts of partSize bytes (the final
// part may be smaller) and each part is uploaded with its own SHA-256 checksum
// so S3 can verify part integrity on the wire.
//
// If any step fails the upload is aborted on a best-effort basis to avoid
// leaving orphaned parts behind, which would otherwise continue to accrue
// storage charges.
func (p *StoragePlugin) putObjectMultipart(
ctx context.Context,
req *pb.PutObjectRequest,
storageState *awsStoragePersistedState,
bucketName, objectKey string,
file *os.File,
partSize int64,
opts ...s3Option,
) error {
bucketId := req.GetBucket().GetId()
createCall := func(s3Client S3API) (any, error) {
return s3Client.CreateMultipartUpload(ctx, &s3.CreateMultipartUploadInput{
Bucket: aws.String(bucketName),
Key: aws.String(objectKey),
ChecksumAlgorithm: types.ChecksumAlgorithmSha256,
})
}
createResp, err := p.call(ctx, createCall, bucketId, storageState, opts...)
if err != nil {
return parseS3Error("create multipart upload", err, req).Err()
}
uploadId := createResp.(*s3.CreateMultipartUploadOutput).UploadId
// once the upload has been created, any subsequent failure must abort it so
// uploaded parts are not orphaned.
abort := func() {
abortCall := func(s3Client S3API) (any, error) {
return s3Client.AbortMultipartUpload(ctx, &s3.AbortMultipartUploadInput{
Bucket: aws.String(bucketName),
Key: aws.String(objectKey),
UploadId: uploadId,
})
}
// best effort: the failure that triggered the abort is more useful to
// the caller than an error from the abort itself.
_, _ = p.call(ctx, abortCall, bucketId, storageState, opts...)
}
var completedParts []types.CompletedPart
buf := make([]byte, partSize)
var partNumber int32 = 1
for {
n, readErr := io.ReadFull(file, buf)
if readErr == io.EOF {
// previous iteration consumed the final bytes of the file
break
}
if readErr != nil && readErr != io.ErrUnexpectedEOF {
abort()
return errors.UnknownStatusf("failed to read file: %s", readErr)
}
partData := buf[:n]
sum := sha256.Sum256(partData)
partChecksum := base64.StdEncoding.EncodeToString(sum[:])
pn := partNumber
uploadCall := func(s3Client S3API) (any, error) {
return s3Client.UploadPart(ctx, &s3.UploadPartInput{
Bucket: aws.String(bucketName),
Key: aws.String(objectKey),
UploadId: uploadId,
PartNumber: aws.Int32(pn),
Body: bytes.NewReader(partData),
ContentLength: aws.Int64(int64(n)),
ChecksumAlgorithm: types.ChecksumAlgorithmSha256,
ChecksumSHA256: aws.String(partChecksum),
})
}
uploadResp, err := p.call(ctx, uploadCall, bucketId, storageState, opts...)
if err != nil {
abort()
return parseS3Error("upload part", err, req).Err()
}
part := uploadResp.(*s3.UploadPartOutput)
completedParts = append(completedParts, types.CompletedPart{
ETag: part.ETag,
ChecksumSHA256: part.ChecksumSHA256,
PartNumber: aws.Int32(pn),
})
partNumber++
if readErr == io.ErrUnexpectedEOF {
// the final, short part
break
}
}
completeCall := func(s3Client S3API) (any, error) {
return s3Client.CompleteMultipartUpload(ctx, &s3.CompleteMultipartUploadInput{
Bucket: aws.String(bucketName),
Key: aws.String(objectKey),
UploadId: uploadId,
MultipartUpload: &types.CompletedMultipartUpload{
Parts: completedParts,
},
})
}
if _, err := p.call(ctx, completeCall, bucketId, storageState, opts...); err != nil {
abort()
return parseS3Error("complete multipart upload", err, req).Err()
}
return nil
}
// DeleteObjects is used to delete one or many objects from an s3 bucket. // DeleteObjects is used to delete one or many objects from an s3 bucket.
func (p *StoragePlugin) DeleteObjects(ctx context.Context, req *pb.DeleteObjectsRequest) (*pb.DeleteObjectsResponse, error) { func (p *StoragePlugin) DeleteObjects(ctx context.Context, req *pb.DeleteObjectsRequest) (*pb.DeleteObjectsResponse, error) {
if req.GetKeyPrefix() == "" { if req.GetKeyPrefix() == "" {
+189
View File
@@ -3715,6 +3715,195 @@ func TestStoragePlugin_PutObject(t *testing.T) {
} }
} }
func TestStoragePlugin_PutObjectMultipart(t *testing.T) {
td := t.TempDir()
// genFile writes size bytes of deterministic content to a temp file and
// returns its path along with the raw SHA-256 digest of the whole file.
genFile := func(t *testing.T, name string, size int) (string, []byte) {
t.Helper()
data := make([]byte, size)
for i := range data {
data[i] = byte(i % 251)
}
fp := path.Join(td, name)
require.NoError(t, os.WriteFile(fp, data, 0o600))
sum := sha256.Sum256(data)
return fp, sum[:]
}
validRequest := func(p string) *pb.PutObjectRequest {
return &pb.PutObjectRequest{
Bucket: &storagebuckets.StorageBucket{
BucketName: "external-obj-store",
BucketPrefix: "prefix",
Secrets: credential.MockStaticCredentialSecrets(),
Attributes: &structpb.Struct{
Fields: map[string]*structpb.Value{
credential.ConstRegion: structpb.NewStringValue("us-west-2"),
},
},
},
Key: "mock-object",
Path: p,
}
}
t.Run("multipart success spanning multiple parts", func(t *testing.T) {
require, assert := require.New(t), assert.New(t)
// 20 MiB with 8 MiB parts -> 3 parts (8, 8, 4 MiB).
fp, wholeChecksum := genFile(t, "large-file", 20*1024*1024)
state := &testMockS3State{}
p := &StoragePlugin{
testCredStateOpts: validSTSMock(),
testStorageStateOpts: []awsStoragePersistedStateOption{
withTestS3APIFunc(newTestMockS3(state)),
},
}
resp, err := p.PutObject(context.Background(), validRequest(fp))
require.NoError(err)
require.NotNil(resp)
// Boundary must receive the whole-object SHA-256, not the multipart
// composite checksum.
assert.True(bytes.Equal(wholeChecksum, resp.GetChecksumSha_256()))
// The single-shot PutObject path must not have been used.
assert.False(state.PutObjectCalled)
assert.True(state.CreateMultipartUploadCalled)
assert.Equal(3, state.UploadPartCallCount)
assert.True(state.CompleteMultipartUploadCalled)
assert.False(state.AbortMultipartUploadCalled)
// The reassembled part bodies must equal the original file content.
orig, err := os.ReadFile(fp)
require.NoError(err)
assert.True(bytes.Equal(orig, state.UploadPartBody))
// The object key must include the configured bucket prefix.
require.NotNil(state.CreateMultipartUploadInputParams)
assert.Equal("prefix/mock-object", aws.ToString(state.CreateMultipartUploadInputParams.Key))
})
t.Run("file just above threshold uses two parts", func(t *testing.T) {
require, assert := require.New(t), assert.New(t)
// 8 MiB + 1 byte -> 2 parts (8 MiB, 1 byte).
fp, wholeChecksum := genFile(t, "just-over", 8*1024*1024+1)
state := &testMockS3State{}
p := &StoragePlugin{
testCredStateOpts: validSTSMock(),
testStorageStateOpts: []awsStoragePersistedStateOption{
withTestS3APIFunc(newTestMockS3(state)),
},
}
resp, err := p.PutObject(context.Background(), validRequest(fp))
require.NoError(err)
assert.True(bytes.Equal(wholeChecksum, resp.GetChecksumSha_256()))
assert.Equal(2, state.UploadPartCallCount)
assert.True(state.CompleteMultipartUploadCalled)
})
t.Run("file at threshold uses single PutObject", func(t *testing.T) {
require, assert := require.New(t), assert.New(t)
// Exactly 8 MiB is not above the threshold, so the single-shot path is used.
fp, wholeChecksum := genFile(t, "at-threshold", 8*1024*1024)
checksum := base64.StdEncoding.EncodeToString(wholeChecksum)
state := &testMockS3State{}
p := &StoragePlugin{
testCredStateOpts: validSTSMock(),
testStorageStateOpts: []awsStoragePersistedStateOption{
withTestS3APIFunc(newTestMockS3(
state,
testMockS3WithPutObjectOutput(&s3.PutObjectOutput{
ChecksumSHA256: aws.String(checksum),
}),
)),
},
}
resp, err := p.PutObject(context.Background(), validRequest(fp))
require.NoError(err)
assert.True(bytes.Equal(wholeChecksum, resp.GetChecksumSha_256()))
assert.True(state.PutObjectCalled)
assert.False(state.CreateMultipartUploadCalled)
})
t.Run("create multipart upload error", func(t *testing.T) {
require, assert := require.New(t), assert.New(t)
fp, _ := genFile(t, "create-err", 20*1024*1024)
state := &testMockS3State{}
p := &StoragePlugin{
testCredStateOpts: validSTSMock(),
testStorageStateOpts: []awsStoragePersistedStateOption{
withTestS3APIFunc(newTestMockS3(
state,
testMockS3WithCreateMultipartUploadError(TestAwsS3Error("AccessDenied", "CreateMultipartUpload", "not authorized")),
)),
},
}
resp, err := p.PutObject(context.Background(), validRequest(fp))
require.Error(err)
assert.Nil(resp)
assert.Equal(codes.PermissionDenied, status.Code(err))
// Nothing was created, so nothing should be aborted.
assert.False(state.AbortMultipartUploadCalled)
})
t.Run("upload part error aborts upload", func(t *testing.T) {
require, assert := require.New(t), assert.New(t)
fp, _ := genFile(t, "upload-err", 20*1024*1024)
state := &testMockS3State{}
p := &StoragePlugin{
testCredStateOpts: validSTSMock(),
testStorageStateOpts: []awsStoragePersistedStateOption{
withTestS3APIFunc(newTestMockS3(
state,
testMockS3WithUploadPartErrorOnPart(2, TestAwsS3Error("AccessDenied", "UploadPart", "not authorized")),
)),
},
}
resp, err := p.PutObject(context.Background(), validRequest(fp))
require.Error(err)
assert.Nil(resp)
assert.Equal(codes.PermissionDenied, status.Code(err))
assert.True(state.CreateMultipartUploadCalled)
assert.False(state.CompleteMultipartUploadCalled)
// The failed upload must be aborted to avoid orphaned parts.
assert.True(state.AbortMultipartUploadCalled)
})
t.Run("complete multipart upload error aborts upload", func(t *testing.T) {
require, assert := require.New(t), assert.New(t)
fp, _ := genFile(t, "complete-err", 20*1024*1024)
state := &testMockS3State{}
p := &StoragePlugin{
testCredStateOpts: validSTSMock(),
testStorageStateOpts: []awsStoragePersistedStateOption{
withTestS3APIFunc(newTestMockS3(
state,
testMockS3WithCompleteMultipartUploadError(TestAwsS3Error("InternalError", "CompleteMultipartUpload", "boom")),
)),
},
}
resp, err := p.PutObject(context.Background(), validRequest(fp))
require.Error(err)
assert.Nil(resp)
assert.Equal(3, state.UploadPartCallCount)
assert.True(state.AbortMultipartUploadCalled)
})
}
func TestStoragePlugin_DeleteObjects(t *testing.T) { func TestStoragePlugin_DeleteObjects(t *testing.T) {
validRequest := func() *pb.DeleteObjectsRequest { validRequest := func() *pb.DeleteObjectsRequest {
return &pb.DeleteObjectsRequest{ return &pb.DeleteObjectsRequest{
+39 -1
View File
@@ -7,9 +7,13 @@ import (
"context" "context"
"errors" "errors"
"fmt" "fmt"
"net/http"
"net/url" "net/url"
"time"
"github.com/aws/aws-sdk-go-v2/aws" "github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/aws/retry"
awshttp "github.com/aws/aws-sdk-go-v2/aws/transport/http"
"github.com/aws/aws-sdk-go-v2/service/s3" "github.com/aws/aws-sdk-go-v2/service/s3"
smithyEndpoints "github.com/aws/smithy-go/endpoints" smithyEndpoints "github.com/aws/smithy-go/endpoints"
"github.com/hashicorp/boundary-plugin-aws/internal/credential" "github.com/hashicorp/boundary-plugin-aws/internal/credential"
@@ -22,6 +26,10 @@ type S3API interface {
GetObject(ctx context.Context, params *s3.GetObjectInput, optFns ...func(*s3.Options)) (*s3.GetObjectOutput, error) GetObject(ctx context.Context, params *s3.GetObjectInput, optFns ...func(*s3.Options)) (*s3.GetObjectOutput, error)
HeadObject(ctx context.Context, params *s3.HeadObjectInput, optFns ...func(*s3.Options)) (*s3.HeadObjectOutput, error) HeadObject(ctx context.Context, params *s3.HeadObjectInput, optFns ...func(*s3.Options)) (*s3.HeadObjectOutput, error)
PutObject(ctx context.Context, params *s3.PutObjectInput, optFns ...func(*s3.Options)) (*s3.PutObjectOutput, error) PutObject(ctx context.Context, params *s3.PutObjectInput, optFns ...func(*s3.Options)) (*s3.PutObjectOutput, error)
CreateMultipartUpload(ctx context.Context, params *s3.CreateMultipartUploadInput, optFns ...func(*s3.Options)) (*s3.CreateMultipartUploadOutput, error)
UploadPart(ctx context.Context, params *s3.UploadPartInput, optFns ...func(*s3.Options)) (*s3.UploadPartOutput, error)
CompleteMultipartUpload(ctx context.Context, params *s3.CompleteMultipartUploadInput, optFns ...func(*s3.Options)) (*s3.CompleteMultipartUploadOutput, error)
AbortMultipartUpload(ctx context.Context, params *s3.AbortMultipartUploadInput, optFns ...func(*s3.Options)) (*s3.AbortMultipartUploadOutput, error)
DeleteObjects(ctx context.Context, params *s3.DeleteObjectsInput, optFns ...func(*s3.Options)) (*s3.DeleteObjectsOutput, error) DeleteObjects(ctx context.Context, params *s3.DeleteObjectsInput, optFns ...func(*s3.Options)) (*s3.DeleteObjectsOutput, error)
ListObjectsV2(ctx context.Context, params *s3.ListObjectsV2Input, optFns ...func(*s3.Options)) (*s3.ListObjectsV2Output, error) ListObjectsV2(ctx context.Context, params *s3.ListObjectsV2Input, optFns ...func(*s3.Options)) (*s3.ListObjectsV2Output, error)
Credentials() aws.Credentials Credentials() aws.Credentials
@@ -118,7 +126,10 @@ func (s *awsStoragePersistedState) s3Client(ctx context.Context, opt ...s3Option
if err != nil { if err != nil {
return nil, fmt.Errorf("failed to retrieve credentials: %v\n", err) return nil, fmt.Errorf("failed to retrieve credentials: %v\n", err)
} }
err = newTransport(awsCfg)
if err != nil {
return nil, fmt.Errorf("failed to configure new http transport: %v", err)
}
c := s3.NewFromConfig(*awsCfg, s3Opts...) c := s3.NewFromConfig(*awsCfg, s3Opts...)
return &s3Client{Client: c, creds: creds}, nil return &s3Client{Client: c, creds: creds}, nil
} }
@@ -202,3 +213,30 @@ func (e *endpointResolver) ResolveEndpoint(ctx context.Context, params s3.Endpoi
} }
return return
} }
// newTransport returns an http Transport object. The http transport object utilizes the
// default transport created by minio. The following transport fields are updated to provide
// more reliable connectivity when handling larger objects:
// MaxResponseHeaderBytes is set to 1MiB to prevent header reads from stalling.
// WriteBufferSize is set to 1MiB, matching the default buffer size used in Boundary storage.
// ReadBufferSize is set to 1MiB, matching the default buffer size used in Boundary storage.
// ForceAttemptHTTP2 is set to true, supporting more resilient multiplexing and avoids HOL blocking on retries.
// IdleConnTimeout is set to 2 minutes.
func newTransport(cfg *aws.Config) error {
if cfg == nil {
return fmt.Errorf("missing aws configuration")
}
cfg.HTTPClient = awshttp.NewBuildableClient().WithTransportOptions(func(transport *http.Transport) {
transport.MaxResponseHeaderBytes = 1 << 20 // 1 MiB prevents header read from stalling
transport.WriteBufferSize = 1 << 20 // 1 MiB write buffer (default is 4 KiB)
transport.ReadBufferSize = 1 << 20 // 1 MiB read buffer (default is 4 KiB)
transport.ForceAttemptHTTP2 = true // enables HTTP/2 where supported more resilient multiplexing, avoids HOL blocking on retries
transport.IdleConnTimeout = 2 * time.Minute // double the idle connection timeout, default is 1 minute
})
cfg.Retryer = func() aws.Retryer {
return retry.NewStandard(func(o *retry.StandardOptions) {
o.MaxAttempts = 20
})
}
return nil
}
+188
View File
@@ -6,6 +6,8 @@ package storage
import ( import (
"bytes" "bytes"
"context" "context"
"crypto/sha256"
"encoding/base64"
"fmt" "fmt"
"io" "io"
"testing" "testing"
@@ -39,6 +41,23 @@ type testMockS3State struct {
PutObjectInputParams *s3.PutObjectInput PutObjectInputParams *s3.PutObjectInput
PutObjectBody []byte PutObjectBody []byte
CreateMultipartUploadCalled bool
CreateMultipartUploadInputParams *s3.CreateMultipartUploadInput
UploadPartCalled bool
UploadPartCallCount int
UploadPartInputParams []*s3.UploadPartInput
// UploadPartBody is the concatenation, in part-number order, of every
// part body received by UploadPart. It can be compared against the full
// object that was expected to be uploaded.
UploadPartBody []byte
CompleteMultipartUploadCalled bool
CompleteMultipartUploadInputParams *s3.CompleteMultipartUploadInput
AbortMultipartUploadCalled bool
AbortMultipartUploadInputParams *s3.AbortMultipartUploadInput
HeadObjectCalled bool HeadObjectCalled bool
HeadObjectInputParams *s3.HeadObjectInput HeadObjectInputParams *s3.HeadObjectInput
@@ -57,6 +76,16 @@ func (s *testMockS3State) Reset() {
s.PutObjectCalled = false s.PutObjectCalled = false
s.PutObjectInputParams = nil s.PutObjectInputParams = nil
s.PutObjectBody = nil s.PutObjectBody = nil
s.CreateMultipartUploadCalled = false
s.CreateMultipartUploadInputParams = nil
s.UploadPartCalled = false
s.UploadPartCallCount = 0
s.UploadPartInputParams = nil
s.UploadPartBody = nil
s.CompleteMultipartUploadCalled = false
s.CompleteMultipartUploadInputParams = nil
s.AbortMultipartUploadCalled = false
s.AbortMultipartUploadInputParams = nil
s.HeadObjectCalled = false s.HeadObjectCalled = false
s.HeadObjectInputParams = nil s.HeadObjectInputParams = nil
s.ListObjectsV2Called = false s.ListObjectsV2Called = false
@@ -81,6 +110,19 @@ type testMockS3 struct {
PutObjectOutput *s3.PutObjectOutput PutObjectOutput *s3.PutObjectOutput
PutObjectErr error PutObjectErr error
// mocked responses for the multipart upload calls
CreateMultipartUploadOutput *s3.CreateMultipartUploadOutput
CreateMultipartUploadErr error
UploadPartOutput *s3.UploadPartOutput
UploadPartErr error
// UploadPartErrOnPart, when greater than zero, causes UploadPart to return
// UploadPartErr (or a generic error) only for the matching part number.
UploadPartErrOnPart int32
CompleteMultipartUploadOutput *s3.CompleteMultipartUploadOutput
CompleteMultipartUploadErr error
AbortMultipartUploadOutput *s3.AbortMultipartUploadOutput
AbortMultipartUploadErr error
// mocked responses for headObject // mocked responses for headObject
HeadObjectOutput *s3.HeadObjectOutput HeadObjectOutput *s3.HeadObjectOutput
HeadObjectErr error HeadObjectErr error
@@ -114,6 +156,63 @@ func testMockS3WithPutObjectError(e error) testMockS3Option {
} }
} }
func testMockS3WithCreateMultipartUploadOutput(o *s3.CreateMultipartUploadOutput) testMockS3Option {
return func(m *testMockS3) error {
m.CreateMultipartUploadOutput = o
return nil
}
}
func testMockS3WithCreateMultipartUploadError(e error) testMockS3Option {
return func(m *testMockS3) error {
m.CreateMultipartUploadErr = e
return nil
}
}
func testMockS3WithUploadPartOutput(o *s3.UploadPartOutput) testMockS3Option {
return func(m *testMockS3) error {
m.UploadPartOutput = o
return nil
}
}
func testMockS3WithUploadPartError(e error) testMockS3Option {
return func(m *testMockS3) error {
m.UploadPartErr = e
return nil
}
}
func testMockS3WithUploadPartErrorOnPart(partNumber int32, e error) testMockS3Option {
return func(m *testMockS3) error {
m.UploadPartErr = e
m.UploadPartErrOnPart = partNumber
return nil
}
}
func testMockS3WithCompleteMultipartUploadOutput(o *s3.CompleteMultipartUploadOutput) testMockS3Option {
return func(m *testMockS3) error {
m.CompleteMultipartUploadOutput = o
return nil
}
}
func testMockS3WithCompleteMultipartUploadError(e error) testMockS3Option {
return func(m *testMockS3) error {
m.CompleteMultipartUploadErr = e
return nil
}
}
func testMockS3WithAbortMultipartUploadError(e error) testMockS3Option {
return func(m *testMockS3) error {
m.AbortMultipartUploadErr = e
return nil
}
}
func testMockS3WithGetObjectOutput(o *s3.GetObjectOutput) testMockS3Option { func testMockS3WithGetObjectOutput(o *s3.GetObjectOutput) testMockS3Option {
return func(m *testMockS3) error { return func(m *testMockS3) error {
m.GetObjectOutput = o m.GetObjectOutput = o
@@ -255,6 +354,95 @@ func (m *testMockS3) PutObject(ctx context.Context, params *s3.PutObjectInput, o
return m.PutObjectOutput, nil return m.PutObjectOutput, nil
} }
func (m *testMockS3) CreateMultipartUpload(ctx context.Context, params *s3.CreateMultipartUploadInput, optFns ...func(*s3.Options)) (*s3.CreateMultipartUploadOutput, error) {
if m.State != nil {
m.State.CreateMultipartUploadCalled = true
m.State.CreateMultipartUploadInputParams = params
}
if m.CreateMultipartUploadErr != nil {
return nil, m.CreateMultipartUploadErr
}
if m.CreateMultipartUploadOutput != nil {
return m.CreateMultipartUploadOutput, nil
}
return &s3.CreateMultipartUploadOutput{
UploadId: aws.String("test-upload-id"),
}, nil
}
func (m *testMockS3) UploadPart(ctx context.Context, params *s3.UploadPartInput, optFns ...func(*s3.Options)) (*s3.UploadPartOutput, error) {
if m.State != nil {
m.State.UploadPartCalled = true
m.State.UploadPartCallCount++
m.State.UploadPartInputParams = append(m.State.UploadPartInputParams, params)
}
// always drain the body so the part data can be asserted on and so the
// reader is not left dangling.
var data []byte
if params.Body != nil {
var err error
data, err = io.ReadAll(params.Body)
if err != nil {
return nil, err
}
if m.State != nil {
m.State.UploadPartBody = append(m.State.UploadPartBody, data...)
}
}
if m.UploadPartErr != nil {
if m.UploadPartErrOnPart == 0 || (params.PartNumber != nil && *params.PartNumber == m.UploadPartErrOnPart) {
return nil, m.UploadPartErr
}
}
if m.UploadPartOutput != nil {
return m.UploadPartOutput, nil
}
// echo the supplied part checksum so CompleteMultipartUpload can be built
// the way the real service would allow.
sum := sha256.Sum256(data)
return &s3.UploadPartOutput{
ETag: aws.String(fmt.Sprintf("etag-%d", aws.ToInt32(params.PartNumber))),
ChecksumSHA256: aws.String(base64.StdEncoding.EncodeToString(sum[:])),
}, nil
}
func (m *testMockS3) CompleteMultipartUpload(ctx context.Context, params *s3.CompleteMultipartUploadInput, optFns ...func(*s3.Options)) (*s3.CompleteMultipartUploadOutput, error) {
if m.State != nil {
m.State.CompleteMultipartUploadCalled = true
m.State.CompleteMultipartUploadInputParams = params
}
if m.CompleteMultipartUploadErr != nil {
return nil, m.CompleteMultipartUploadErr
}
if m.CompleteMultipartUploadOutput != nil {
return m.CompleteMultipartUploadOutput, nil
}
return &s3.CompleteMultipartUploadOutput{}, nil
}
func (m *testMockS3) AbortMultipartUpload(ctx context.Context, params *s3.AbortMultipartUploadInput, optFns ...func(*s3.Options)) (*s3.AbortMultipartUploadOutput, error) {
if m.State != nil {
m.State.AbortMultipartUploadCalled = true
m.State.AbortMultipartUploadInputParams = params
}
if m.AbortMultipartUploadErr != nil {
return nil, m.AbortMultipartUploadErr
}
if m.AbortMultipartUploadOutput != nil {
return m.AbortMultipartUploadOutput, nil
}
return &s3.AbortMultipartUploadOutput{}, nil
}
func (m *testMockS3) ListObjectsV2(ctx context.Context, params *s3.ListObjectsV2Input, optFns ...func(*s3.Options)) (*s3.ListObjectsV2Output, error) { func (m *testMockS3) ListObjectsV2(ctx context.Context, params *s3.ListObjectsV2Input, optFns ...func(*s3.Options)) (*s3.ListObjectsV2Output, error) {
if m.State != nil { if m.State != nil {
m.State.ListObjectsV2Called = true m.State.ListObjectsV2Called = true
+92
View File
@@ -165,6 +165,9 @@ func TestHostPlugin(t *testing.T) {
testPluginListHosts(ctx, t, p, region, keyid, secret, tc, expectedTagInstancesMap) testPluginListHosts(ctx, t, p, region, keyid, secret, tc, expectedTagInstancesMap)
// TODO: add OnDeleteSet if it needs to be implemented // TODO: add OnDeleteSet if it needs to be implemented
} }
// finally lets test instance_addresses_only catalog attribute correctly syncs only instance addresses
testInstanceAddressesOnlyListHosts(ctx, t, p, tf, region, keyid, secret)
} }
func testPluginOnCreateCatalog(ctx context.Context, t *testing.T, p *host.HostPlugin, region, accessKeyId, secretAccessKey string, rotate bool) (string, string) { func testPluginOnCreateCatalog(ctx context.Context, t *testing.T, p *host.HostPlugin, region, accessKeyId, secretAccessKey string, rotate bool) (string, string) {
@@ -450,3 +453,92 @@ func testPluginListHosts(ctx context.Context, t *testing.T, p *host.HostPlugin,
// Success // Success
t.Logf("testing ListHosts: success (region=%s, tags=%v, expected/actual=(len=%d, ids=%s))", region, tags, len(actualInstances), actualInstances) t.Logf("testing ListHosts: success (region=%s, tags=%v, expected/actual=(len=%d, ids=%s))", region, tags, len(actualInstances), actualInstances)
} }
func testInstanceAddressesOnlyListHosts(ctx context.Context, t *testing.T, p *host.HostPlugin, tf *TestTerraformer, region, accessKeyId, secretAccessKey string) {
t.Helper()
t.Logf("testing testInstanceAddressesOnlyListHosts")
// Get the instance details from tf output
instanceID, err := tf.GetOutputString("multi_address_instance_id")
require.NoError(t, err)
primaryAddresses, err := tf.GetOutputMap("multi_address_primary_addresses")
require.NoError(t, err)
secondaryAddresses, err := tf.GetOutputMap("multi_address_secondary_addresses")
require.NoError(t, err)
var primary, allAddresses []string
for _, addr := range primaryAddresses {
primary = append(primary, addr.(string))
}
allAddresses = append(allAddresses, primary...)
for _, addr := range secondaryAddresses {
allAddresses = append(allAddresses, addr.(string))
}
cases := []struct {
name string
instanceAddressesOnly bool
expectedIPs []string
}{
{
name: "all addresses",
expectedIPs: allAddresses,
},
{
name: "instance addresses only",
instanceAddressesOnly: true,
expectedIPs: primary,
},
}
for _, tc := range cases {
tc := tc
t.Run(tc.name, func(t *testing.T) {
require := require.New(t)
catalogAttrMap := map[string]any{
credential.ConstRegion: region,
credential.ConstDisableCredentialRotation: true,
}
if tc.instanceAddressesOnly {
catalogAttrMap[host.ConstInstanceAddressesOnly] = true
}
catalogAttrs, err := structpb.NewStruct(catalogAttrMap)
require.NoError(err)
setAttrs, err := structpb.NewStruct(map[string]any{
host.ConstDescribeInstancesFilters: []any{fmt.Sprintf("instance-id=%s", instanceID)},
})
require.NoError(err)
reqPersistedSecrets, err := structpb.NewStruct(map[string]any{
credential.ConstAccessKeyId: accessKeyId,
credential.ConstSecretAccessKey: secretAccessKey,
credential.ConstCredsLastRotatedTime: (time.Time{}).Format(time.RFC3339Nano),
})
require.NoError(err)
request := &pb.ListHostsRequest{
Catalog: &hostcatalogs.HostCatalog{
Attrs: &hostcatalogs.HostCatalog_Attributes{
Attributes: catalogAttrs,
},
},
Sets: []*hostsets.HostSet{{
Id: "hostset-0",
Attrs: &hostsets.HostSet_Attributes{Attributes: setAttrs},
}},
Persisted: &pb.HostCatalogPersisted{
Secrets: reqPersistedSecrets,
},
}
response, err := p.ListHosts(ctx, request)
require.NoError(err)
require.Len(response.GetHosts(), 1)
hostResp := response.GetHosts()[0]
require.Equal(instanceID, hostResp.ExternalId)
require.ElementsMatch(hostResp.IpAddresses, tc.expectedIPs)
})
}
}
+121 -4
View File
@@ -20,13 +20,41 @@ data "aws_ami" "ubuntu" {
} }
resource "aws_vpc" "vpc" { resource "aws_vpc" "vpc" {
cidr_block = "10.0.0.0/16" cidr_block = "10.0.0.0/16"
assign_generated_ipv6_cidr_block = true
} }
resource "aws_subnet" "subnet" { resource "aws_subnet" "subnet" {
vpc_id = aws_vpc.vpc.id vpc_id = aws_vpc.vpc.id
cidr_block = aws_vpc.vpc.cidr_block cidr_block = aws_vpc.vpc.cidr_block
availability_zone = data.aws_availability_zones.azs.names[0] ipv6_cidr_block = cidrsubnet(aws_vpc.vpc.ipv6_cidr_block, 8, 0)
availability_zone = data.aws_availability_zones.azs.names[0]
map_public_ip_on_launch = true
assign_ipv6_address_on_creation = true
}
# Need to add a gateway for public IP addr assignments
resource "aws_internet_gateway" "igw" {
vpc_id = aws_vpc.vpc.id
}
resource "aws_route_table" "public" {
vpc_id = aws_vpc.vpc.id
route {
cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.igw.id
}
route {
ipv6_cidr_block = "::/0"
gateway_id = aws_internet_gateway.igw.id
}
}
resource "aws_route_table_association" "public" {
subnet_id = aws_subnet.subnet.id
route_table_id = aws_route_table.public.id
} }
resource "aws_instance" "instances" { resource "aws_instance" "instances" {
@@ -38,6 +66,73 @@ resource "aws_instance" "instances" {
tags = local.instance_tags[count.index] tags = local.instance_tags[count.index]
} }
resource "aws_instance" "multi_address_instance" {
ami = data.aws_ami.ubuntu.id
instance_type = "t3.nano"
subnet_id = aws_subnet.subnet.id
secondary_private_ips = ["10.0.10.10"]
ipv6_address_count = 1
associate_public_ip_address = true
tags = {
Name = "boundary-multi-address-instance"
"boundary-demo" = local.hashicorp_email
}
}
resource "aws_network_interface" "multi_address_secondary_eni" {
subnet_id = aws_subnet.subnet.id
private_ips = ["10.0.20.10"]
ipv6_addresses = [cidrhost(aws_subnet.subnet.ipv6_cidr_block, 32)]
tags = {
Name = "boundary-multi-address-secondary-eni"
"boundary-demo" = local.hashicorp_email
}
}
resource "aws_network_interface_attachment" "multi_address_secondary_eni" {
instance_id = aws_instance.multi_address_instance.id
network_interface_id = aws_network_interface.multi_address_secondary_eni.id
device_index = 1
}
resource "aws_eip" "multi_address_primary" {
domain = "vpc"
network_interface = aws_instance.multi_address_instance.primary_network_interface_id
associate_with_private_ip = aws_instance.multi_address_instance.private_ip
depends_on = [aws_internet_gateway.igw, aws_route_table_association.public]
tags = {
Name = "boundary-multi-address-primary-eip"
"boundary-demo" = local.hashicorp_email
}
}
resource "aws_eip" "multi_address_primary_secondary_private" {
domain = "vpc"
network_interface = aws_instance.multi_address_instance.primary_network_interface_id
associate_with_private_ip = "10.0.10.10"
depends_on = [aws_internet_gateway.igw, aws_route_table_association.public, aws_instance.multi_address_instance]
tags = {
Name = "boundary-multi-address-primary-secondary-eip"
"boundary-demo" = local.hashicorp_email
}
}
resource "aws_eip" "multi_address_secondary_eni" {
domain = "vpc"
network_interface = aws_network_interface.multi_address_secondary_eni.id
associate_with_private_ip = "10.0.20.10"
depends_on = [aws_internet_gateway.igw, aws_route_table_association.public, aws_network_interface_attachment.multi_address_secondary_eni]
tags = {
Name = "boundary-multi-address-secondary-eni-eip"
"boundary-demo" = local.hashicorp_email
}
}
output "instance_ids" { output "instance_ids" {
value = aws_instance.instances.*.id value = aws_instance.instances.*.id
} }
@@ -61,3 +156,25 @@ output "instance_tag_keys" {
random_id.baz_key.dec, random_id.baz_key.dec,
] ]
} }
output "multi_address_instance_id" {
value = aws_instance.multi_address_instance.id
}
output "multi_address_primary_addresses" {
value = {
private_ip = aws_instance.multi_address_instance.private_ip
public_ip = aws_eip.multi_address_primary.public_ip
ipv6 = tolist(aws_instance.multi_address_instance.ipv6_addresses)[0]
}
}
output "multi_address_secondary_addresses" {
value = {
primary_eni_private_ip = "10.0.10.10"
primary_eni_public_ip = aws_eip.multi_address_primary_secondary_private.public_ip
secondary_eni_private_ip = aws_network_interface.multi_address_secondary_eni.private_ip
secondary_eni_public_ip = aws_eip.multi_address_secondary_eni.public_ip
secondary_eni_ipv6 = tolist(aws_network_interface.multi_address_secondary_eni.ipv6_addresses)[0]
}
}