feat(config): Support multiple files
This updates the three subcommands that take a `-config` flag to
to support multiple config files via a repeated `-config` flag.
It updates the following subcommands:
- `server`
- `database init`
- `database migrate`
Note that the multiple file support only works if all config files are
HCL, not JSON. Also note that the config files are concatenated
together. There is no attempt to merge config files, so having the same
stanza in multiple files can yield unexpected results, with only one of
the stanzas being used.
When used with these limitations understood, this provides a way to take
different actions for different kinds of dynamic configuration changes.
An example would be when using consul-template to render the
configuration files for boundary. With the multi file support
consul-template can render one config file for the controller stanza,
and provide boundary with a dynamic database credential via vault. Then
when the credential is rotated, consul-template can send boundary a
SIGHUP to switch to the new credential without a restart. A separate
template could be used to render a config for a stanza that does not
support SIGHUP, such as the events stanza that gets its values via
consul's KV. If these values change, consul-template can restart
boundary so the new configuration is used.
Co-authored-by: Timothy Messier <tim.messier@gmail.com>
Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
In cases where the server was started with both a controller and worker
stanza, the reload of the config was not properly setting the initial
upstream value on the new config. In this case the value is derived from
other config values and not necessarily set directly.
This now properly initializes the new config so that initial upstream is
set in the same manner as the first load.
Blame: 5d0cdf680f
This adds an action to the API that allows pulling out an activation
token that can be given to a worker in its config (directly or via env
or file). The worker can pass this along with its normal request in
place of its normal nonce and the server will automatically accept it.
Co-authored-by: Irena Rindos <irenarindos@users.noreply.github.com>
* adds CLI commands for add/set/removing worker tags, and respective tests
* adds StringSliceMapVar Flag to convert a string from the CLI to a map[string][]string value
* Clean up some dev config kms code
Although it didn't matter in the combined state, the KMS for auth
storage was not actually being set properly if only a worker was being
instantiated.
This adds two new config options for the `database` stanza:
- `max_idle_connections`
- `max_idle_time`
These correspond to settings for the underlying `sql.DB` instance. They
can be used to have finer control over the controller's database
connection pool. For example this can be used to mitigate the high
connection overhead when connecting to postgres by setting a high
`max_idle_connections` and either setting `max_idle_time` to zero or to
a high duration.
Alternatively, if using a connection pool like pgpool-II or pgbouncer,
`max_idle_connections` can be set to 0.
See:
https://pkg.go.dev/database/sql#DB.SetMaxIdleConnshttps://pkg.go.dev/database/sql#DB.SetConnMaxIdleTimehttps://www.pgpool.net/mediawiki/index.php/Main_Pagehttps://www.pgbouncer.org/
* Don't automatically create worker names, as this is now invalid for
PKI workers
* Fix a check in dev that clears out name/description in exactly the
wrong cases
* Change eventing setup to use a hostname-based ID instead of the name
(which could change on every start anyways)
* Adds WorkerAuthStorage wrapper for worker credential encryption using nodeenrollment library.
* Adds AuthStoragePath field in worker config and dev mode to specify directory to store credentials.
* Remove the server_id from session table, change it to worker_id on session_connection
This is step 1 in updating the worker table. The worker is becoming a public resource so it must have
a public id. We drop the server_id column on the session_connection table and add the worker_id column
so we can use the wt_public_key type. Since db migrations are applied while the controller is offline,
it should be safe to assume all session connections are not in a running state, thus allowing us to null
out all the worker ids in the session_connection table.
For now we don't have very good enforcement of the wt_public_id constraints, since the id is currently
defined in the worker config, but this will soon change and the worker's id will be generated by the
controller.
* feat(controller): New health service and server
This change also implements a new listener purpose, `ops`.
`ops` (short for operations) is a new listener purpose that is
responsible for serving Boundary's operations endpoints, like health
and possibly metrics.
At Startup: `ops` listeners get configured as late as possible to catch
any errors with the startup process;
At Shutdown: If the health endpoint has been set-up, we introduce a
configurable delay in the process. This is to enable the health endpoint
to report unhealthy for that amount of time. `ops` listeners are also
the last listeners to be shutdown.
This feature allows the user to set the `controllers` field to a string
that points to either an environment variable or file.
We then dynamically pull out the value and parse it into the config.
This feature allows the user to set the `tags` field to a string
that points to either an environment variable or file.
We then dynamically pull out the value and parse it into the config.
This can also be done at the keys level, where you set your keys
directly in Boundary's config file, but point the values to env/file.
* Add ability to read and list hosts. Tests to follow.
* Add tests for reading and listing as well as validation checks to not modify hosts.
* Add IpAddress and DnsNames to the api resource. Populate the plugin info.
* Adding go sdk tests for reading and listing hosts and ensuring update and deleting is disallowed.
This puts it more in line with other upcoming options that use "dir"
instead of "directory" (plus it's easier to type), and makes it clear
that this directory and the passthrough behavior is related to the UI as
opposed to other upcoming items.
* feat (events): Add new options needed for AuditConfig
WithAuditWrapper and WithFilterOperations all for optional paramters
when creating AuditConfig types.
* refactor (event): Change EventerConfig.Sinks to a ptr
We need to modify the SinkConfig.AuditConfig during the NewEventer factory, so we have to use a ptr not a ref.
* feat (events): Add audit configuration types
* feat (events): add audit config to sink configuration.
* feat (events): add redact/encrypt node for audit events to Eventer
Integrate the encrypt.Filter node from the eventlogger into the
event.Eventer, so audit events can be redacted/encrypted based
on data classifications.
* test (events) Add test protobufs for events
Add test protobufs along with a test service for testing
the audit event encrypt node
* Normalize event configuration
This updates event configuration stanzas to more closely match behavior
for listeners and kmses:
* Singular "sink" instead of "sinks" (probably not the most correct
option, but a consistent option, and one that makes sense for HCL)
* Allows sinks to have the type defined at the start of the block
* Moves type-specific configuration to a specific named block
Co-authored-by: Jim <jlambert@hashicorp.com>
This commit adds the support to do the following:
* Mark connections for non-reporting workers as closed. This is the
controller counterpart to the worker functionality (see #1330). This is
written as a scheduled job that does the work DB-side in a single atomic
query.
* Works to reconcile states if such a broken controller-worker
connection resumes and a worker reports a connection as connected that
should be disconnected. In this case, the controller will send an update
request, and the worker will honor it and terminate the connection.
* Further refinement of the grace period setting has been added here.
We have converged on the current server "liveness" setting as our
default here, which is half of the previous 30s (15 seconds, in other
words). Additionally, this is now configurable on the controller and
worker side, with the caveat that it's currently impossible to do so in
config as the setting has been untagged in HCL. This is exposed so that
we can run some sophisticated testing scenarios where we skew the grace
period to either the controller or worker to ensure the aforementioned
reconciliation works.
* Some repository functions have been added to support the new
functionality, in addition to some test code to the worker to allow
querying of session state while testing.
* Finally, we've added some add timestamp subtraction functions
as well, basically serving as the opposite of the addition functions.
Timothy MessierandJeff Mitchell
Louis Ruch
Haotian
Johan Brandhorst-Satzkorn
irenarindos
Todd
Hugo
Irena Rindos
Todd
Jim
Chris Marchesi