This commit fixes the behaviour of scopes where whitespace was allowed
(and thus persisted) in the Name and Description fields.
It also adds some more tests for scope Create and Updateto cover more
error cases.
For scope update, `handlers.ValidateUpdateRequest` exists, however this
was not used as it does not work properly for requests where the
`global` scope is updated.
In `ValidateUpdateRequest`, an ID check is performed and it assumes
there's always a resource prefix (eg: "p", "o") and an identifier
(random numbers and letters) after the prefix. This fails for `global`.
See `handlers.ValidId`.
`action.List` is a collection action, that is, it is not part of the set
of actions that can be executed against a single instance of any given
resource.
As part of the `ListPermissions` call, `idActions` is expected, that is,
all the actions that can be executed against a resource instance.
`action.List` will never be part of this. This commit updates the tests
to reflect this fact.
The new `ExternalName` field is an optional field that is used to store
the cloud providers' human-identifiable name for a given compute
instance. This field is not user-editable, but it is kept in sync with
the cloud provider's through Boundary's `SetSyncJob`.
Adds the schema for ssh targets. This also changes the target listing
func to not error out when it encounters a target with an unknown type.
Instead, it now ignores it and continues to the next one.
If an update call was made to a target that removed an address, for a
target that already did not have an address, the incorrect number of
rows updated was being reported. During the update the target's version
was correctly being updated, resulting in one row being updated.
However then the delete call would result in zero rows being deleted,
since there was no address associated with the target, and this would
replace the value for the returned rowsUpdated. This resulted in the
target service thinking that no update happened, and it would report an
error.
Creates a target using an address for boundary dev and when first
init-ing boundary (boundary database init).
It also makes this newly created target the default one
(ttcp_1234567890). The target using host sources is now a secondary
target (ttcp_0987654321).
This commit adds CLI support for addresses to be attached directly to a
Target, for all current types of Target (ssh and tcp).
It also clarifies the relevant documentation regarding the autogen of
API option functionality.
This fixes a theoretical issue with database interaction. Because the
fields in the database are signed 64-bit integers, it's theoretically
possible to overflow them if we're using unsigned 64-bit integers in the
application code.
With this commit, the Worker now sends BytesUp and BytesDown for each
of the connections it handles to the Controller in the Status request.
The Controller then persists these fields to the database. There is also
a new trigger to prevent bytes_up and bytes_down to change after a
connection has been closed, as the last update to a session_connection
row should be the one that is performed as part of the connection
closure process.
Introduces data about bytes read and written to the connection close
payload. No changes are required in the Controller because the RPC
messages already contained the required fields. As of this commit,
bytes_up and bytes_down columns in the session_connection table are now
populated when a connection closes.
Refs: #2509
This commit introduces a new net.Conn implementation (countingConn) that
will keep track of bytes read and written for a particular client
connection. This is then implemented into the worker code using
interface composition.
Refs: #2501
If the database url is updated on Boundary's config and then a SIGHUP
triggered, we now connect to the new database, ensure it's OK to use
and use that one going forwards instead.
This allows a controller to be deployed in a more dynamic infrastructure
where, for example, database credentials could change, without having
to restart the controller.
This operation is done atomically to prevent errors and inconsistency.
A timeout is present so that if the object is swapped after the
existing one has been looked up it isn't immediately invalidated.
The timeout is a maximum; a ticker once a second that looks at
stats to determine how many database connections are still in use
and then closes the driver as soon as none are in use.
If the database url is updated on Boundary's config and then a SIGHUP
triggered, we now connect to the new database, ensure it's OK to use
and use that one going forwards instead.
This allows a controller to be deployed in a more dynamic infrastructure
where, for example, database credentials could change, without having
to restart the controller. In this case only a database url config
change and a SIGHUP signal to all controllers would be required.
This uses an atomic.Pointer to store wrapped DB objects. This allows
atomically swapping to the new object. A timeout is introduced so that
if the object is swapped after the existing one has been looked up it
isn't immediately invalidated. The timeout is a maximum; a ticker once a
second that looks at stats to determine how many database connections
are still in use and then closes the driver as soon as none are in use.
This could be done with mutexes to avoid checking stats in a loop, but
that carries its own risk: if a write lock is attempted to be acquired
while a query is holding a read lock, the write lock will block, and any
subsequent read locks will also block since write locks take priority.
As a result, if a query is long-running or timing out, all other queries
in the system will become blocked. Our default user request timeout is
90 seconds and some jobs have no timeout at all, so this is not a purely
theoretical concern, even if it's unlikely.
Refs: #2435
Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
This commit extracts 3 separate pieces of functionality into their own
function/method:
- Extracts the database opening with the intention to have a function
that performs all that functionality but returns the created database
object rather than setting it to the Server structure.
- Extracts database validation logic, to provide a common code path for
upcoming functionality.
- Expose schema manager on the Command struct as well as providing
common functions to acquire shared lock/close.
This commit introduces closing the Schema Manager's underlying driver
connection. We were only releasing the shared lock but not actually
cleaning up the db connection and relied on the fact that the process
would exit to do that for us.
Using the new Permissions object, we can build a list of the level of
access a user should have based on the grants they've been given,
meaining we don't need to fetch all resources from the database to then
remove the ones the user shouldn't be able to see.
This commit also removes some unused options in `ListTargets`
for the sake of simplicity.
* feat(worker): Instrumentation for Controller Cluster Communication
Adds request latency observations to the worker side (client) of
worker <-> controller cluster communications by integrating a new
grpc client interceptor.
* style(controller): make-gen-deltas changes
This change introduces a `nonceFn` field that can be changed
by a test but at runtime is transparent to the code that calls it.
The current nonce-generating logic defaults to a test-only field which
opens us up to this value inadvertently being set.
With this change, the default behavior becomes using `base62` to
generate a random nonce, and any tests have to explicitly change that.
With this change, the Worker is now aware of its own listeners in a
more codified and structured way.
These changes have already been made in the controller on
8d48a8515c. They're now made on the
worker for similar reasons.
* feat(controller): New health service and server
This change also implements a new listener purpose, `ops`.
`ops` (short for operations) is a new listener purpose that is
responsible for serving Boundary's operations endpoints, like health
and possibly metrics.
At Startup: `ops` listeners get configured as late as possible to catch
any errors with the startup process;
At Shutdown: If the health endpoint has been set-up, we introduce a
configurable delay in the process. This is to enable the health endpoint
to report unhealthy for that amount of time. `ops` listeners are also
the last listeners to be shutdown.
In general, these changes were introduced in the same vein of the
changes to controller `startListeners`.
Specifically, these changes also decouple the Controller and the Worker
shutdown processes. Currently, a call to `stopListeners` on an instance
that is running as both a Controller and a Worker would stop the
listeners for both.
Lastly, we also add unit/integration tests to the shutdown code.
Previously, we initialized the listeners on the order they were provided
to us on the config file. In the future, we'll want to add new listener
purposes that depend on existing ones, so the init order needs to be
deterministic.
With this change, we'll initialize components in this order:
1. gRPC Server for Controller API.
2. All `api` HTTP Servers & gRPC Gateway Muxes.
3. `cluster` gRPC Server.
This commit introduces some renames for the sake of clarity as well as
improved test coverage:
- Removes `gatewayMux` reference from Controller object. This object was
misleading and is now uneeded.
- Removed uneeded `context.Context` arguments.
- Separates gRPC Server and gRPC Gateway concerns, both in functionality
and in naming. These two were conflated before.
- Added unit tests on `startListeners`.
With this change, the Controller is now aware of its own listeners in a
more codified and structured way.
The reasoning behind these changes are as follows:
1. Currently, we're iterating over these listeners in multiple spots
along this initialization flow to check each listeners' purpose. By
doing this once when we create a Controller and essentially storing the
result, we can clean up the downstream code to use those results.
2. Initial effort to centralize Controller validation. At the moment, we
are doing a lot of validation all over the `cmd` and `config` packages.
From a conceptual perspective, Controller-related validation should
be performed in the function that is responsible for constructing the
Controller object itself. This enables any developer to look in a
single place for all the constraints that we place on creating a
Controller.
This feature allows the user to set the `controllers` field to a string
that points to either an environment variable or file.
We then dynamically pull out the value and parse it into the config.
This feature allows the user to set the `tags` field to a string
that points to either an environment variable or file.
We then dynamically pull out the value and parse it into the config.
This can also be done at the keys level, where you set your keys
directly in Boundary's config file, but point the values to env/file.
Adds tests to `SetupControllerPublicClusterAddress` and
`SetupWorkerPublicAddress` ahead of changes in these functions to
ensure functionality is preserved.
Hugo
