chore(cli): Run bats tests against boundary dev in CI
Several of these tests are being skipped in the enos run. This seems to be due to differences in the setup that is performed. When these tests were run manually, `boundary dev` was used which performs some setup of resources automatically. However since enos uses a non-dev controller, a similar setup is performed via the enos modules. Until the discrepancies between the setup can be investigated, this adds a separate CI job to run the tests against `boundary dev`. These tests are all passing, without any being skipped. This test suite can still be useful even if the enos version is fixed, since this test suite will run much faster, and therefore identify failures sooner.
This commit is contained in:
5 files changed
+139
-4
No files matched your search
@@ -0,0 +1,5 @@
|
||||
all: test
|
||||
|
||||
.PHONY: test
|
||||
test:
|
||||
./test.sh
|
||||
@@ -1,9 +1,9 @@
|
||||
# Boundary CLI Tests
|
||||
|
||||
This directory contains [bats tests](https://github.com/bats-core/bats-core) for testing the Boundary CLI against arbitrary Boundary deployments.
|
||||
This directory contains [bats tests](https://github.com/bats-core/bats-core) for testing the Boundary CLI against arbitrary Boundary deployments.
|
||||
The tests are meant to mimic common workflows such as creating resources, and connecting to targets. Currently, the tests rely heavily on
|
||||
generated resources when running Boundary in `dev` mode. In the future, we hope to remove this dependency and generate all resources through
|
||||
the Boundary CLI from the outset.
|
||||
the Boundary CLI from the outset.
|
||||
|
||||
The tests are designed to be idempotent.
|
||||
|
||||
@@ -13,6 +13,7 @@ The tests are designed to be idempotent.
|
||||
|
||||
- [jq](https://stedolan.github.io/jq/)
|
||||
- [bats](https://github.com/bats-core/bats-core)
|
||||
- [netcat](https://nc110.sourceforge.io/)
|
||||
- [boundary](https://github.com/hashicorp/boundary)
|
||||
|
||||
#### Running Tests
|
||||
@@ -44,7 +45,7 @@ IS_VERSION=true bats -p boundary/
|
||||
You can run this suite against an arbitrary cluster by overriding the following env vars:
|
||||
|
||||
```bash
|
||||
DEFAULT_LOGIN=<my_login> \
|
||||
DEFAULT_LOGIN=<my_login> \
|
||||
DEFAULT_PASSWORD=<my_password> \
|
||||
BOUNDARY_ADDR=http://boundary-test-controller-salmon-b70f2710539143b4.elb.us-east-1.amazonaws.com:9200 \
|
||||
bats boundary/
|
||||
@@ -52,4 +53,4 @@ bats boundary/
|
||||
|
||||
~> Note that these tests currently expect generated resources to exist. Failure to run against a Boundary deployment that does
|
||||
not have generated resources will result in a lot of test failures. Future work includes breaking thses tests out in a way
|
||||
that makes them not dependant on generated resources.
|
||||
that makes them not dependent on generated resources.
|
||||
Executable
+38
@@ -0,0 +1,38 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
# TERM isn't set automatically in CI so we need to make sure it's always there.
|
||||
export TERM=${TERM:=dumb}
|
||||
|
||||
function die {
|
||||
echo $*
|
||||
exit -1
|
||||
}
|
||||
|
||||
# error out early if missing a command
|
||||
which boundary || die "missing boundary"
|
||||
which jq || die "missing jq"
|
||||
which bats || die "missing bats"
|
||||
which nc || die "missing nc"
|
||||
|
||||
echo "starting boundary dev in background"
|
||||
boundary dev &>/dev/null &
|
||||
boundary_pid=$!
|
||||
|
||||
function cleanup {
|
||||
rv=$?
|
||||
echo "stopping boundary dev"
|
||||
if [[ -n ${boundary_pid} ]]; then
|
||||
kill ${boundary_pid}
|
||||
fi
|
||||
exit $rv
|
||||
}
|
||||
|
||||
trap cleanup EXIT
|
||||
|
||||
until boundary scopes list; do
|
||||
echo 'waiting for boundary to be up'
|
||||
sleep 1
|
||||
done
|
||||
|
||||
echo "running bats tests"
|
||||
bats -p ./boundary
|
||||
Reference in new issue
Block a user