* `recovery` is only used by the API (and the main module, for decryption)
* `wrapper` is only used by the CLI right now
Moving these brings them closer to the code that uses them, and makes
the SDK package a bit more clear in terms of functionality.
* server: Add back controller shutdown on worker startup failure
This adds back the controller shutdown call on worker failure.
This was added in a15961e4 to address a panic discovered in testing, and
was mistakenly thought to be unnecessary. This, however, covers
multi-role scenarios where we have already started the controller, which
needs to be shut down gracefully if possible if we can't start the
worker.
A nil check has been added to ensure the panic does not resurface.
Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
* add new error funcs which take a ctx and deprecate existing funcs
* use context where available when writing error events
* remove circular internal/errors pkg dep
* suppress some chatty errors using errors.WithoutEvent()
* convert auth oidc funcs to take context where needed to write events
Co-authored-by: Jim <jlambert@hashicorp.com>
This adds functionality to ensure that connections are closed on a
worker when the server is shut down or reloaded.
Connections are closed, and then the shutdown process waits for the next
report to the controller to go through, waiting the grace period that
was added in #1330. The rest of shutdown is then proceeded with.
Listener shutdown has been moved to the start of the process to ensure
that new connections can't be made while we are shutting down.
Additionally, functionality has been added to force shut down servers
after a second interrupt has been received after the first one. This
takes the form of code in both the server and dev mode code paths that
just performs an os.Exit if another shutdown is received during the
normal shutdown process.
Additionally, the commit also includes some refactoring to tests so that
we could use some of the new E2E testing code to test session
connections during shutdown and reload.
Additionally, this fixes a panic in worker startup when there are no
controllers defined. Looks like the shutdown call was attempting to call
into the controller, not the worker.
Finally, this also removes the extra shutdown within the outer worker
startup.
Shutdown is actually handled within the inner StartWorker(), attempting
shutdown on errors was actually leading to potential use of an
uninitialized worker, which is leading to panics in testing.
This commit adds the support to do the following:
* Mark connections for non-reporting workers as closed. This is the
controller counterpart to the worker functionality (see #1330). This is
written as a scheduled job that does the work DB-side in a single atomic
query.
* Works to reconcile states if such a broken controller-worker
connection resumes and a worker reports a connection as connected that
should be disconnected. In this case, the controller will send an update
request, and the worker will honor it and terminate the connection.
* Further refinement of the grace period setting has been added here.
We have converged on the current server "liveness" setting as our
default here, which is half of the previous 30s (15 seconds, in other
words). Additionally, this is now configurable on the controller and
worker side, with the caveat that it's currently impossible to do so in
config as the setting has been untagged in HCL. This is exposed so that
we can run some sophisticated testing scenarios where we skew the grace
period to either the controller or worker to ensure the aforementioned
reconciliation works.
* Some repository functions have been added to support the new
functionality, in addition to some test code to the worker to allow
querying of session state while testing.
* Finally, we've added some add timestamp subtraction functions
as well, basically serving as the opposite of the addition functions.
When running single-server mode, use public_cluster_addr if given
Right now you don't need to specify a controller address and we use the
cluster listener address; this allows overriding it to the
public_cluster_addr if one is given.
If `controllers` is specified we allow it to be either the cluster
listener or the public cluster address if one is specified before trying
to parse and use it.
Create Schema Manager and propagate contexts into the DB calls downstream.
Add checks at controller startup and database init for correct schema version and schema migration dirty bit being set.
This adds logic to error on certain invalid listener config:
* If an address in the worker's "controller" set is unspecified or
multicast or global unicast, error
* If the controller's cluster address is unspecified, ensure
public_cluster_addr is set
This has been a source of some confusing behavior on some GitHub tickets
and even on the team, so preventing it is probably a good idea.
It also relaxes the restriction around controller address matching cluster address in single-server mode when using DNS names.
I don't think the conditions exist here that necessitated it being
introduced into Vault, but I do sometimes see a baffling condition where
the address coming from the flag is empty. I don't think we need this so
take it out.
Jeff Mitchell
Chris MarchesiandJeff Mitchell
Jim
s-christoffandJim
Dylan Scott
Todd Knight
Michael GaffneyandJeff Mitchell