* feat(controller): New health service and server
This change also implements a new listener purpose, `ops`.
`ops` (short for operations) is a new listener purpose that is
responsible for serving Boundary's operations endpoints, like health
and possibly metrics.
At Startup: `ops` listeners get configured as late as possible to catch
any errors with the startup process;
At Shutdown: If the health endpoint has been set-up, we introduce a
configurable delay in the process. This is to enable the health endpoint
to report unhealthy for that amount of time. `ops` listeners are also
the last listeners to be shutdown.
In general, these changes were introduced in the same vein of the
changes to controller `startListeners`.
Specifically, these changes also decouple the Controller and the Worker
shutdown processes. Currently, a call to `stopListeners` on an instance
that is running as both a Controller and a Worker would stop the
listeners for both.
Lastly, we also add unit/integration tests to the shutdown code.
This contains to related things:
1. An adapter allowing an hclog interface to take in information and send them to the eventer
2. A gate on the eventer, which is the analogue to the behavior prior to eventing where we'd ensure logs were not printed prior to the startup information. Without this, plugin logs using the hclog adapter will start showing up too early.
The previous PR only initialized plugins if they were required, which
meant that there may not actually be a plugin manager running because a
second node would not actually create the dev database, which is where
the plugin was being added. The assumption that a plugin manager is
running is still a fine one at the moment, but we just need to make sure
it's running in _all_ cases.
This moves all of the tcp target functionality into a tcp package. It
also establishes a Register function in the target package to allow
specific target types to be registered at initilazation time. When
registering, a target type needs to provide its Subtype, as well as some
RepositoryHooks that are used by the target Repository.
This puts it more in line with other upcoming options that use "dir"
instead of "directory" (plus it's easier to type), and makes it clear
that this directory and the passthrough behavior is related to the UI as
opposed to other upcoming items.
* refact: Changes required for gorm v2
* refact: Add db.DB wrapper and refact all test fixtures to use it
Stop leaking the gorm abstraction within our unit tests.
* refact: Migrate direct calls to pq to pgx driver
The refactor will remove the remaining direct dependencies on
the pq driver which has been deprecated in favor of the pgx
driver.
It includes the addition of common.SqlOpen which is a wrapper around
sql.Open, which is needed to deal with the driver name changing from
"postgres" to "pgx" but in our code base the dialect and driver are
synonyms.
* add new error funcs which take a ctx and deprecate existing funcs
* use context where available when writing error events
* remove circular internal/errors pkg dep
* suppress some chatty errors using errors.WithoutEvent()
* convert auth oidc funcs to take context where needed to write events
Co-authored-by: Jim <jlambert@hashicorp.com>
This adds functionality to ensure that connections are closed on a
worker when the server is shut down or reloaded.
Connections are closed, and then the shutdown process waits for the next
report to the controller to go through, waiting the grace period that
was added in #1330. The rest of shutdown is then proceeded with.
Listener shutdown has been moved to the start of the process to ensure
that new connections can't be made while we are shutting down.
Additionally, functionality has been added to force shut down servers
after a second interrupt has been received after the first one. This
takes the form of code in both the server and dev mode code paths that
just performs an os.Exit if another shutdown is received during the
normal shutdown process.
Additionally, the commit also includes some refactoring to tests so that
we could use some of the new E2E testing code to test session
connections during shutdown and reload.
Additionally, this fixes a panic in worker startup when there are no
controllers defined. Looks like the shutdown call was attempting to call
into the controller, not the worker.
Finally, this also removes the extra shutdown within the outer worker
startup.
Shutdown is actually handled within the inner StartWorker(), attempting
shutdown on errors was actually leading to potential use of an
uninitialized worker, which is leading to panics in testing.
This commit adds the support to do the following:
* Mark connections for non-reporting workers as closed. This is the
controller counterpart to the worker functionality (see #1330). This is
written as a scheduled job that does the work DB-side in a single atomic
query.
* Works to reconcile states if such a broken controller-worker
connection resumes and a worker reports a connection as connected that
should be disconnected. In this case, the controller will send an update
request, and the worker will honor it and terminate the connection.
* Further refinement of the grace period setting has been added here.
We have converged on the current server "liveness" setting as our
default here, which is half of the previous 30s (15 seconds, in other
words). Additionally, this is now configurable on the controller and
worker side, with the caveat that it's currently impossible to do so in
config as the setting has been untagged in HCL. This is exposed so that
we can run some sophisticated testing scenarios where we skew the grace
period to either the controller or worker to ensure the aforementioned
reconciliation works.
* Some repository functions have been added to support the new
functionality, in addition to some test code to the worker to allow
querying of session state while testing.
* Finally, we've added some add timestamp subtraction functions
as well, basically serving as the opposite of the addition functions.
* Sessions support for read:self and cancel:self permissions. This is automatic with fallback; no API change is needed.
* Strips out values from listing if only self actions are allowed on the resource and the user ID is different. This prevents a default ACL of id=*;type=session;actions=read:self,cancel:self from causing all values to be visible when listing.
* Adds an unprivileged user to development/testing instances. This user has only default permissions plus the ability to authorize sessions in the default target.
Create Schema Manager and propagate contexts into the DB calls downstream.
Add checks at controller startup and database init for correct schema version and schema migration dirty bit being set.
Hugo Vieira
Jeff Mitchell
Jim
Timothy Messier
s-christoffandJim
Chris Marchesi
