* Return worker process information in the health endpoint.
Experimental: The worker_info and returned worker_process_info fields are experimental and can change or be removed without notice.
When a user passes in "worker_info=1" or "worker_info=true" and a worker is running the returned json will be returned with a key named "worker_process_info" which value is a json object with information about the worker process.
This commit extracts 3 separate pieces of functionality into their own
function/method:
- Extracts the database opening with the intention to have a function
that performs all that functionality but returns the created database
object rather than setting it to the Server structure.
- Extracts database validation logic, to provide a common code path for
upcoming functionality.
- Expose schema manager on the Command struct as well as providing
common functions to acquire shared lock/close.
In cases where the server was started with both a controller and worker
stanza, the reload of the config was not properly setting the initial
upstream value on the new config. In this case the value is derived from
other config values and not necessarily set directly.
This now properly initializes the new config so that initial upstream is
set in the same manner as the first load.
Blame: 5d0cdf680f
* feat(workers): Add ability to read and reinitialize cert authority (#2312)
* feat(workers): Return Release Version on worker list and read (#2377)
* feat(cli): Read and reinitialize worker certificate authority (#2387)
Using the new Permissions object, we can build a list of the level of
access a user should have based on the grants they've been given,
meaining we don't need to fetch all resources from the database to then
remove the ones the user shouldn't be able to see.
This commit also removes some unused options in `ListTargets`
for the sake of simplicity.
This adds a context to the session.RepositoryFactory, allowing
errors.NewDeprecated to be replaced with errors.New. It also removes the
common.SessionRepoFactory, instead packages just directly import from
the session package.
This builds on previous performance improvements to the list behavior by
reducing the number of queries against the database and the amount of
data transfered in order to retrieve a list of sessions. Previously, a
query was performed to retrieve a tuple of `(session_id, scope_id,
user_id)` for the requested scope, including all child scopes if the
request was recursive. These tuples were evaluated against the
requesting user's grants to identify the sessions that they were
authorized to list. In the case of clusters with a large number of
sessions, this would result in a large amount of data needing to be
transfered for sessions that would not be used. Then a second query
would be used to retrieve the full session information for the session
ids that were authorized.
This changes the list behavior to evaluate a set of permissions from
the requesting user's grants, use these permissions when constructing a
session repository, and retrieve the list of sessions the user is
authorized to list in a signle query.
Ref:
d74103482932070678dc
Disabling the auth was resulting in some unrealistic tests setup and
expectations. It also makes it difficult to refactor any behavior in the
authn/authz flow since this option can bypass most of it.
Disabling the auth was resulting in some unrealistic tests setup and
expectations. It also makes it difficult to refactor any behavior in the
authn/authz flow since this option can bypass most of it.
* feat(workers): added latencies histogram for server side multihop workers
* refactor(daemon): combine common grpc metric functions into daemon/internal/metric
Co-authored-by: Todd <github@quaddi.com>
* bug(vault): Correctly handle credential stores with expired tokens
Boundary makes use of a database view to perform CRUD operations on
Vault credential stores and libraries. This view did not include
credential stores with tokens that have expired in Vault, causing
errors when attempting to perform any action against them.
This adds an action to the API that allows pulling out an activation
token that can be given to a worker in its config (directly or via env
or file). The worker can pass this along with its normal request in
place of its normal nonce and the server will automatically accept it.
Co-authored-by: Irena Rindos <irenarindos@users.noreply.github.com>
On a SIGHUP, if the worker's `initial_upstreams` config option has
changed, it will use the new values as the address to send its status.
Any previously discovered upstream addresses from previous status
requests are dropped. Once a successful status request is made to the
new addresses, the worker will resume its normal behavior of using the
addresses from the response to the status request.
This allows a worker to be deployed in a more stable way in a dynamic
environment where controller IPs could change. In particular this helps
with a case where all controllers are replaced with new controllers
running on new IPs.
To illustrate with an example, suppose there is an instance group of
controllers instances. And there are worker instances that have their
config rendered from consul-template that sets `initial_upstreams` to
the IPs of the controller instances. If all controller instances are
restarted or replaced at the same time and assigned new IPs, a new
config will be rendered for the worker. Previously the only way for the
worker to use the new value would be for the worker process to be
restarted. Otherwise the worker would only know about the previous IPs
and would fail to connect to the new controllers. However, a restart
would mean any active sessions and session connections on the worker
instance would be terminated. With this update, consul-template can send
a SIGHUP to the worker, the worker will connect to the new controllers
and resume normal status reporting, allowing the sessions and session
connections to remain.
The same principles would apply for other examples of dynamic
environments such as nomad and kubernetes.
See:
https://github.com/hashicorp/consul-templatehttps://www.nomadproject.io/docs/job-specification/template
Although there is a compile time check on the type, it will always
succeed due to the Unimplemented embedding requirement.
As a result, this PR also changes Unimplemented calls to Unsafe calls,
which were added later "for those that prefer their code to break"; the
comments on these state:
// UnsafeXServiceServer may be embedded to opt out of forward compatibility for this service.
// Use of this interface is not recommended, as added methods to XServiceServer will
// result in compilation errors.
See https://github.com/grpc/grpc-go/issues/4500#issuecomment-852597871
for some more context.
I think we're fine with compilation errors though; we own both the
server and client side of things and generally are going to add new
methods only when we intend to also add them on the client side.
Notably, what the comment does not indicate is that it will break at
runtime; already it should return a suitable (404 or 405) error if the
client requests a method that does not exist.
This fixes occurences of a loop variable being captured in parallel
tests. With the previous code, only the last test case is actually
exercised. To work around this problem, we create a local copy of the
range variable before the parallel test, as suggested in the
documentation for the `testing` package:
https://pkg.go.dev/testing#hdr-Subtests_and_Sub_benchmarks
Issues were found automatically using the `loopvarcapture` linter.
* chore: update go-dbw dependencies
This update included an update of the upstream gorm dependency.
In the latest update, gorm appears to have become more sensitive when
a slice is passed into operations. The ptr to slice must now
be a ptr to slice of ptrs. This change required an update to how
boundary handles slices of PrincipalRole, which now must return a ptr
to PrincipalRole. There were also some tests that needed to be
updated because of this change.
* fixup! chore: update go-dbw dependencies
The intention of the code was to have one-hop-away workers list KMS
workers to get initial upstreams, and have other workers continue using
what was in their config file. However, nothing stops people from having
initial_upstreams but _also_ adding a cluster ID in their config file,
in which case we'd always run this check and replace the config-set
initial upstreams. So, ensure the config-set initial upstreams is also
len zero.
* Clean up some dev config kms code
Although it didn't matter in the combined state, the KMS for auth
storage was not actually being set properly if only a worker was being
instantiated.
Irena Rindos
Jeff Mitchell
Louis Ruch
Todd
Damian Debkowski
Hugo Vieira
Timothy Messier
HaotianandTodd
Johan Brandhorst-Satzkorn
Renato Costa
Damian Debkowski
Jim
