This runs the make targets:
- make test
- make test-api
- make test-sdk
The go tests in the main module are split by package and run in parallel
in a matrix build with GOMAXPROCS=1. This is done primarily to work
around an issue that is only seen on the github runners. When running
at a higher GOMAXPROCS tests that use vault instances in docker
containers start to intermitently fail with errors from vault, usually
either a 400, 403, or TLS error. It is not clear what the cause of this
is, since the same tests run fine on self hosted runners and on
CircleCI.
Co-authored-by: Sarah Thompson <sthompson@hashicorp.com>
Co-authored-by: Daniel Kimsey <daniel.kimsey@hashicorp.com>
Sometimes, the workflow can fail before enos is even installed, which causes this step to fail due enos not existing.
This change makes it such that this step only runs if the Retry step fails.
When the `Run Enos scenario` and `Retry Enos scenario` steps fail, the Destroy command does not get run since the `Retry` step marks the workflow as failed. This leaves some infrastructure hanging around.
* test(e2e): Add database migration test
* chore(enos): Add enos scenario for migration test
* chore(enos): Update go test command to use count option
* doc(e2e): Update readme with migration test details
* Fix: Typo
* CR: Use different library
* CR: Add additional context in comment
* CR: Move some code from test into functions
* doc(e2e): Add instructions to more easily retrieve environment variables
* refact(e2e): Create some more methods to increase test readability
* Add Enos CI bootstrapping scenarios
* Add GitHub Actions workflow for CI bootstrapping
* Add service quotas limit increase request to Enos CI bootstrapping
* Onboard boundary to use new .release/VERSION file and reproducible actions-go-build
* Added automatic patch version bumps to end of release pipeline
* Added back VersionMetadata to internal directory
* Moved VERSION file to version/
* Update to current version
* Using prepare workflow for pre-stable channel workflow
* Test prepare workflow
* PR feedback: allow builds to run builds locally without setting BIN_PATH
* PR feedback: revert Enos changes
* PR Feedback
* Remove current branch from ci pipeline
* Run make fmt
Rearranges the directory structure to align better with the enos scenarios. Tests will now be added in the `tests` directory, with a folder in the directory corresponding to an enos scenario.
This adds a github actions workflow to run the new e2e test suite
Co-authored-by: Michael Anthony <manthony@hashicorp.com>
Co-authored-by: Josh Brand <jbrand@hashicorp.com>
Several of these tests are being skipped in the enos
run. This seems to be due to differences in the setup that is performed.
When these tests were run manually, `boundary dev` was used which
performs some setup of resources automatically. However since enos uses
a non-dev controller, a similar setup is performed via the enos modules.
Until the discrepancies between the setup can be investigated, this adds
a separate CI job to run the tests against `boundary dev`. These tests
are all passing, without any being skipped. This test suite can still be
useful even if the enos version is fixed, since this test suite will run
much faster, and therefore identify failures sooner.
Disable the fail-fast feature for the matrix strategy. This prevents a
single failure to cascade to all jobs in the matrix. This prevents cases
where other jobs fail before they're able to clean up resources.
Signed-off-by: Ryan Cragun <me@ryan.ec>
Enos jobs are still a bit unstable, we should use them as
advisory for now. When they have gained more stability,
we can let them fail pipelines again.
* [QTI-275] Add infrastructure integration testing
On pull requests that originate from branches of the `hashicorp/boundary`
repo, we now conditionally dispatch infrastructure integration tests using
the Enos framework. The `integration` Enos scenario includes two `builder`
variants which allow developers or CI to deploy a Boundary cluster in AWS
using either a local `boundary` install bundle or that which is produced
in the `build` workflow for CRT. The `test` variants determine which type
of infrastructure integration test to run. At present those options are
`cli_ui` (Bats CLI tests) or `smoke` (creates target instances and a
target host catalog and verifies that `boundary` works).
In addition to the new workflow and quality scenarios, we simplified the
some `build` workflow steps and added Go module caching to speed up the
build pipeline.
See the [enos/README.md](https://github.com/hashicorp/boundary/blob/main/enos/README.md)
for a detailed explanation on how to configure and execute Enos scenarios
from your machine.
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Rebecca Willett <rwillett@hashicorp.com>
Co-authored-by: Josh Branch <jbrand@hashicorp.com>
Michael Li
Josh Brand
Mike Baum
Mark Collao
Ashlee M Boyer
Irena Rindos
Jeff Mitchell
Danielle
Mike Baum
emily neil
Michael Anthony
Ryan Cragun
Jaymala
Michele Degges
