threshold-ui/e2e-tests

• Boundary UI E2E tests
  • Prerequisites:
    • Accesses
    • Software
  • Getting Started
    • Set up Boundary CLI
    • Setup Boundary UI
    • Setup AWS:
    • Setup Enos:
  • Run tests:
    • Launch Enos Scenario
    • Admin
    • Desktop
    • Destroy Enos Scenario
  • Developing Tests

Boundary UI E2E tests

This test suite tests the Boundary admin UI and desktop client in an end-to-end setting. It was designed to be run in a variety of environments as long as the appropriate environment variables are set. You can choose to spin up your own infrastructure or use Enos to generate it for you. More information about Enos available here.

The test suite uses the Playwright framework.

Prerequisites:

You will need Homebrew installed. To securely store your SSH keys and tokens we recommend using 1Password.

Accesses

• Doormat account: boundary_team_acctest_dev.
  • Request access through Doormat.

Software

• Vault: documentation how to install.
• Terraform: documentation how to install.
• Github CLI: documentation how to install.
• Install Enos: documentation how to install using homebrew, also binaries available here.
• Install Doormat CLI: documentation how to install.
• Install AWS CLI (Optional): documentation how to install.

Getting Started

We will cover everything you need to set up before being able to run Enos + the tests.

Set up Boundary CLI

Ensure the boundary CLI is available on the path. You can validate this by making sure this command returns something.

which boundary

Otherwise, follow instructions to install Boundary or install it locally running make install within Boundary repository.

Setup Boundary UI

Ensure you install project dependencies and playwright needs.

cd boundary-ui/e2e-tests/admin
yarn install
npx playwright install # this installs the browsers used by Playwright

Setup AWS:

Region awareness: take note of the AWS region you are setting up because we will need it later to configure enos. We recommend us-east-1. More information about AWS regions.

SSH Key pair:

You need to provide an SSH Key pair for the EC2 instance. We recommend creating specific e2e SSH keys and not re-use existing ones. To differentiate them, we recommend adding enos to the name, i.e: name_enos.

How to create EC2 SSH key pair.

If you would like to do it through the command line:

doormat login
source <(doormat aws export --account boundary_team_acctest_dev)
# Feel free to use a custom name for your key pair if you don't want to use $USER, just make sure it follows the <name>_enos format so it can be identified. e.g. zed_enos
ssh-keygen -N '' -t ed25519 -f ~/.ssh/"$USER"_enos
aws ec2 import-key-pair \
  --region us-east-1 \
  --key-name "$USER"_enos \
  --public-key-material fileb://~/.ssh/"$USER"_enos.pub

Setup Enos:

Make sure you're in the correct boundary repository for the version of Boundary you plan on using. The enos scenarios for CE need to be run in boundary and the enos scenarios for enterprise need to be run in boundary-enterprise.

Enos needs some configuration variables to run the scenario successfully. See the configuration file. The file has comments per each variable, but some awareness:

• boundary_edition: The edition of Boundary you are using. Options are oss or enterprise.
• aws_region: The AWS region you are using. Very important as mentioned within the EC2 Setup.
• aws_ssh_keypair_name: The name of the AWS keypair.
• aws_ssh_private_key_path: The path to the private key associated with your keypair.
• enos_user: The user name to use for tagged resources in AWS.
• e2e_debug_no_run: Make sure this is set to true.

More documentation about scenario variables.

Tip

To run e2e tests against a local branch for Admin UI, update the UI commit in boundary so it gets picked up when enos builds the binary

By default, enos will build the version of the UI that is specified in the VERSION commit for boundary. To use a specific branch, update the version first before running the scenario.

In the boundary or boundary-enterprise repository:

UI_COMMITISH=<commit sha> make update-ui-version

# Alternatively if you want to just be able to run the same command for the most recent commit
UI_COMMITISH=$(git -C <boundary-ui(-enterprise) repo path> rev-parse HEAD) make update-ui-version

Run tests:

Before running the e2e test locally, we need to launch an Enos Scenario. Make sure you followed all the steps within the Getting started section.

It is not necessary, but from this point we recommend having 2 terminals open.

• Terminal 1: Will be used to run enos (Boundary).
• Terminal 2: Will be used to run e2e UI tests (Boundary UI).

Launch Enos Scenario

Using Terminal 1:

• cd boundary/enos or cd boundary_enterprise/enos.
• doormat login. Login with Doormat.
• eval "$(doormat aws export --account boundary_team_acctest_dev)". Exporting AWS env variables from doormat to your terminal.
• enos scenario launch e2e_ui_aws builder:local or enos scenario launch e2e_ui_aws_ent builder:local if in enterprise.
  • Launches enos scenario, this will take from 5 to 10 minutes. When its done, you will see a Enos Operations finished! within your terminal. Check out more scenarios here.
• bash scripts/test_e2e_env.sh. Prints all the env variables within Enos scenario. Copy the output and paste it within your Terminal 2 (Boundary UI). These env variables are need within Boundary UI to run the test against the enos scenario.

Alternatively, you can also redirect the output to an .env file that will get picked up by tests automatically which is useful if you're running tests in your IDE:

bash scripts/test_e2e_env.sh > (boundary-ui directory)/e2e-tests/.env

Important

Be aware that once the scenario is launched you will create and run resources within AWS. After you are done using the scenario, you should destroy it.

Admin

Using Terminal 2:

Set the env variables test_e2e_env.sh script output in this terminal.

cd boundary-ui/e2e-tests
yarn run admin

Here are some additional commands to assist with debugging.

PWDEBUG=console yarn playwright test --headed --config admin/playwright.config.js login.spec.js
PWDEBUG=console yarn playwright test --headed --config admin/playwright.config.js login.spec.js:13 --debug
PWDEBUG=console yarn playwright test --headed --config admin/playwright.config.js login.spec.js --debug

To run all tests for a certain configuration, you can use the following shortcuts

# Runs all tests pertaining to the Community Edition
yarn run e2e:ce:aws
yarn run e2e:ce:docker

# Runs all tests pertaining to the Enterprise Edition
yarn run e2e:ent:aws
yarn run e2e:ent:docker

Desktop

Note

Currently the desktop client e2e tests assumes that the Boundary server is an enterprise edition.

Using Terminal 2:

Set the env variables test_e2e_env.sh script output in this terminal.

cd boundary-ui/e2e-tests
yarn run desktop

Playwright documentation about running tests.

Destroy Enos Scenario

Using Terminal 1:

# Community Edition
enos scenario destroy e2e_ui_aws builder:local

# Enterprise Edition
enos scenario destroy e2e_ui_aws_ent builder:local

After all the steps pass, you should see a Enos operations finished!.

Developing Tests

It is recommended to use locators that resemble how users interact with the application. getByRole should be the most prioritized locator as this is the closest way to how users and accessibility features perceive the page. getByLabel should be used for form fields.

Additional information can be found here and here.