Timothy Messier
624bdbf0c2
This sets up the configuration for running a new suite of tests that are
written in sql, to test the sql logic implemented in plpgsql functions
and triggers. It is primarily designed to provides tests for the data
warehouse functionality that is implemented purely in sql.
The infrastructure to run these tests leverages docker. It creates one
docker container using the `library/postgres` image and provisions the
database with the boundary sql migrations and test helper functions. A
separate docker container is run that has the pgtap extension and
pg_prove cli to execute the tests.
As noted, pgtap is used for the tests. This postgresql extension
provides a number of helpful assertion functions to make writing sql
tests easier. It also, along with the pg_prove cli, provides readable
test output.
Along with the new testing infrastructure this commit establishes a
baseline of tests for the wh_user_dimension:
- Unit tests for individual views and functions:
- whx_user_dimension_source
- whx_user_dimension_target
- wh_upsert_user
- Integration tests for that verify that the wh_user_dimension gets
populated correctly when sessions are inserted or updated. This
ensures that:
- the trigger runs on session insert
- the trigger results in the correct changes to the
wh_user_dimension.
This commit provides a `Makefile` to run the tests locally that provides
support for:
- Setting the postgres docker image to easily test multiple versions.
- Passing through options to pg_prove
- Passing the test list to run individual tests
- Additional targets to separately startup the database image to quickly
re-run tests while developing new tests.
Lastly configures circleci to run this test suite as a matrix build
in parallel with the other tests. The matrix build tests multiple
versions of postgres:
- latest
- 13-alpine
- 12-alpine
- 11-alpine
See:
https://pgtap.org/
1.8 KiB
1.8 KiB
SQL Tests
This test suite is used to test behavior of the database logic. In particular the data warehouse implementation is completely in sql via plpgsql, functions, and triggers. This test suite is also implemented directly in sql.
Organization
initdb.d: contains init scripts/sql that is run on the test database when it starts. It ensures the sql migrations are run and creates test helper functions.tests: contains the tests. Each file can contain only a single test that runs in a transaction so it can rollback.
The tests leverage the pgTap postgres extension to make assertions and provide more readable test output.
Usage
To run the test run make or make test. This will:
- Start a docker
postgrescontainer and initialize it. - Start a docker
pgtapcontainer to execute the tests.
When writing new tests it can be faster to keep the database up and just re-run the tests. This can be done by running:
# starts database docker image in the background
make database-up
# runs the tests, call this multiple times while implementing new tests.
make run-tests
# to clean up
make clean
You can also run individual tests:
# run a single test file
make TEST=tests/setup/wtt_load.sql
# run a single test file with the database already created.
make run-tests TEST=tests/setup/wtt_load.sql
# run a directory of tests
make TEST=tests/setup/*.sql
You can pass through options to pg_prove.
See the docs
for available options, i.e:
# run tests in parallel with verbose output
make PROVE_OPTS='-j9 -v'
Different versions of postgres can easily be tested:
make PG_DOCKER_TAG=latest
make PG_DOCKER_TAG=13-alpine
make PG_DOCKER_TAG=12-alpine
make PG_DOCKER_TAG=11-alpine