Four new columns were added to the `wh_user_dimension` table to support
OIDC. The data type for the columns was originally `wh_dim_text` which
is a domain type with a `not null` restriction. Adding the new columns
caused the migration to fail if the `wh_user_dimension` table had any
rows in it because the existing rows would violate the `not null`
restriction.
Closes#1469
* refactor(worker): Move tcp_proxy into its own package
This refactors:
- the tcp_proxy.go into worker/proxy/tcp
- add a missing unit test for the HandleTcpProxyV1
- refactors session.go into worker/session, needed to avoid import cycle
- adds a baseline for a mock controller session client
- go lint fixes
* Add sleep between spinning listener goroutine and dialing
Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
This adds new columns to the wh_user_dimension to provide additional
data from OIDC that does not apply to Password Auth.
Note: Given the specifics of the data, it is possible these columns may
be used by other auth methods in the future, hence the lack of `oidc` in
the column names.
The specific columns are:
- auth_method_external_id
For OIDC this is the Issuer
- auth_account_external_id
For OIDC this is the Subject
- auth_account_full_name
For OIDC this is the account's Full Name, and can be `None`.
- auth_account_email
For OIDC this is the account's Email, and can be `None`.
For all of these new columns, they will be `Not Applicable` if the auth
method is Password.
This sets up the configuration for running a new suite of tests that are
written in sql, to test the sql logic implemented in plpgsql functions
and triggers. It is primarily designed to provides tests for the data
warehouse functionality that is implemented purely in sql.
The infrastructure to run these tests leverages docker. It creates one
docker container using the `library/postgres` image and provisions the
database with the boundary sql migrations and test helper functions. A
separate docker container is run that has the pgtap extension and
pg_prove cli to execute the tests.
As noted, pgtap is used for the tests. This postgresql extension
provides a number of helpful assertion functions to make writing sql
tests easier. It also, along with the pg_prove cli, provides readable
test output.
Along with the new testing infrastructure this commit establishes a
baseline of tests for the wh_user_dimension:
- Unit tests for individual views and functions:
- whx_user_dimension_source
- whx_user_dimension_target
- wh_upsert_user
- Integration tests for that verify that the wh_user_dimension gets
populated correctly when sessions are inserted or updated. This
ensures that:
- the trigger runs on session insert
- the trigger results in the correct changes to the
wh_user_dimension.
This commit provides a `Makefile` to run the tests locally that provides
support for:
- Setting the postgres docker image to easily test multiple versions.
- Passing through options to pg_prove
- Passing the test list to run individual tests
- Additional targets to separately startup the database image to quickly
re-run tests while developing new tests.
Lastly configures circleci to run this test suite as a matrix build
in parallel with the other tests. The matrix build tests multiple
versions of postgres:
- latest
- 13-alpine
- 12-alpine
- 11-alpine
See:
https://pgtap.org/
* Normalize event configuration
This updates event configuration stanzas to more closely match behavior
for listeners and kmses:
* Singular "sink" instead of "sinks" (probably not the most correct
option, but a consistent option, and one that makes sense for HCL)
* Allows sinks to have the type defined at the start of the block
* Moves type-specific configuration to a specific named block
Co-authored-by: Jim <jlambert@hashicorp.com>
* server: Add back controller shutdown on worker startup failure
This adds back the controller shutdown call on worker failure.
This was added in a15961e4 to address a panic discovered in testing, and
was mistakenly thought to be unnecessary. This, however, covers
multi-role scenarios where we have already started the controller, which
needs to be shut down gracefully if possible if we can't start the
worker.
A nil check has been added to ensure the panic does not resurface.
Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
* add new error funcs which take a ctx and deprecate existing funcs
* use context where available when writing error events
* remove circular internal/errors pkg dep
* suppress some chatty errors using errors.WithoutEvent()
* convert auth oidc funcs to take context where needed to write events
Co-authored-by: Jim <jlambert@hashicorp.com>
This adds functionality to ensure that connections are closed on a
worker when the server is shut down or reloaded.
Connections are closed, and then the shutdown process waits for the next
report to the controller to go through, waiting the grace period that
was added in #1330. The rest of shutdown is then proceeded with.
Listener shutdown has been moved to the start of the process to ensure
that new connections can't be made while we are shutting down.
Additionally, functionality has been added to force shut down servers
after a second interrupt has been received after the first one. This
takes the form of code in both the server and dev mode code paths that
just performs an os.Exit if another shutdown is received during the
normal shutdown process.
Additionally, the commit also includes some refactoring to tests so that
we could use some of the new E2E testing code to test session
connections during shutdown and reload.
Additionally, this fixes a panic in worker startup when there are no
controllers defined. Looks like the shutdown call was attempting to call
into the controller, not the worker.
Finally, this also removes the extra shutdown within the outer worker
startup.
Shutdown is actually handled within the inner StartWorker(), attempting
shutdown on errors was actually leading to potential use of an
uninitialized worker, which is leading to panics in testing.
* updated dockerfile to support multi-arch; added new section in Make for multi-arch
* adding a line to changelog
* switching base image to hashicorp mirror
Jeff Mitchell
Todd Knight
Michael Gaffney
Louis RuchandJeff Mitchell
Jim
Timothy Messier
Taylor Dolezal
Susmitha Girumala
Mike Wickett
s-christoff
Chris MarchesiandJeff Mitchell
Bryce Kalow
Randall Morey
Jacob Mammoliti
Nico Vibert
xuwhao