Timothy Messier
624bdbf0c2
This sets up the configuration for running a new suite of tests that are
written in sql, to test the sql logic implemented in plpgsql functions
and triggers. It is primarily designed to provides tests for the data
warehouse functionality that is implemented purely in sql.
The infrastructure to run these tests leverages docker. It creates one
docker container using the `library/postgres` image and provisions the
database with the boundary sql migrations and test helper functions. A
separate docker container is run that has the pgtap extension and
pg_prove cli to execute the tests.
As noted, pgtap is used for the tests. This postgresql extension
provides a number of helpful assertion functions to make writing sql
tests easier. It also, along with the pg_prove cli, provides readable
test output.
Along with the new testing infrastructure this commit establishes a
baseline of tests for the wh_user_dimension:
- Unit tests for individual views and functions:
- whx_user_dimension_source
- whx_user_dimension_target
- wh_upsert_user
- Integration tests for that verify that the wh_user_dimension gets
populated correctly when sessions are inserted or updated. This
ensures that:
- the trigger runs on session insert
- the trigger results in the correct changes to the
wh_user_dimension.
This commit provides a `Makefile` to run the tests locally that provides
support for:
- Setting the postgres docker image to easily test multiple versions.
- Passing through options to pg_prove
- Passing the test list to run individual tests
- Additional targets to separately startup the database image to quickly
re-run tests while developing new tests.
Lastly configures circleci to run this test suite as a matrix build
in parallel with the other tests. The matrix build tests multiple
versions of postgres:
- latest
- 13-alpine
- 12-alpine
- 11-alpine
See:
https://pgtap.org/
db package
Usage
Just some high-level usage highlights to get you started. Read the godocs for a complete list of capabilities and their documentation.
conn, _ := gorm.Open("postgres", url)
// Db implements both the Reader and Writer interfaces
rw := Db{Tx: conn}
// There are writer methods like: Create, Update and Delete
// that will write Gorm struct to the db. These writer methods
// all support options for writing Oplog entries for the
// caller: WithOplog(yourWrapper, yourMetadata)
// the caller is responsible for the transaction life cycle of the writer
// and if an error is returned the caller must decide what to do with
// the transaction, which is almost always a rollback for the caller.
err = rw.Create(context.Background(), user)
// There are reader methods like: LookupByPublicId,
// LookupByName, SearchBy, LookupBy, etc
// which will lookup resources for you and scan them into your Gorm struct
err = rw.LookupByPublicId(context.Background(), foundUser)
// There's reader ScanRows that facilitates scanning rows from
// a query into your Gorm struct
where := "select * from test_users where name in ($1, $2)"
rows, err := rw.Query(context.Background(), where, []interface{}{"alice", "bob"})
defer rows.Close()
for rows.Next() {
user := db_test.NewTestUser()
// scan the row into your Gorm struct
if err := rw.ScanRows(rows, &user); err != nil {
return err
}
// Do something with the Gorm user struct
}
// DoTx is a writer function that wraps a TxHandler
// in a retryable transaction. You simply implement a
// TxHandler that does your writes and hand the handler
// to DoTx, which will wrap the writes in a retryable
// transaction with the retry attempts and backoff
// strategy you specify via options.
_, err = w.DoTx(
context.Background(),
10, // ten retries
ExpBackoff{}, // exponential backoff
func(w Writer) error {
// the TxHandler updates the user's friendly name
return w.Update(context.Background(), user, []string{"FriendlyName"},
// write oplogs for this update
WithOplog(
InitTestWrapper(t),
oplog.Metadata{
"key-only": nil,
"deployment": []string{"amex"},
"project": []string{"central-info-systems", "local-info-systems"},
},
),
)
})