* oss: feat(proto): statistics rpc added to server coordination service
* oss: feat(session): add update statistics status service
* oss: feat(handlers): add Statistics method to worker service server
* oss: feat(worker): add statistics ticking
* oss: internal/proto: define the new RoutingInfo RPC
* oss: internal/server: repository changes for RoutingInfo RPC
* oss: internal/daemon: add RoutingInfo handler to worker service
The new RoutingInfo RPC is used to get information from the worker
that is used to make routing decisions on the controller. It also
handles the initial request from the worker.
* oss: feat(db): server_worker_session_info table
* oss: feat(proto): session info rpc for server coordination service
* oss: feat(server): worker session info request data type
* oss: feat(worker): session info ticker
* oss: feat(handler): server coordination handler for session info
* oss: feat(session): utilize session info request time for session cleanup
* oss: proto: fix some gotags comments
Some of these fields were missing comments, while others had
incorrectly formatted comments.
* oss: worker: send connection status to server
This is required for the correct tracking of orphaned connections
* oss: internal/daemon/worker: add function to wait for Statistics RPC
* oss: internal/daemon/worker: add routing info RPC
The new RoutingInfo RPC replaces part of the Status RPC,
namely the part which is used by the controller and worker
to perform decisions around the establishment of new sessions
and new worker connections.
* oss: internal/daemon/worker: remove unnecessary error check
* oss: internal/daemon/worker: lock access to config and upstreams
The new confLock protects access to the conf value and to the
addressReceivers slice.
* oss: worker: replace Status RPC with RoutingInfo
* oss: worker: deprecate the Status RPC and related types
The Status RPC and related messages have been replaced
by the RoutingInfo/SessionInfo/Statistics RPCs. The controller
will continue to support the Status RPC until two new minor
versions have been released.
* oss: internal/event: add default deny for new worker RPCs
* oss: internal/daemon/worker: allow RPC interval to be set
The new SessionInfo/RoutingInfo/Statistics RPCs run at
long intervals, which make some of our cluster tests take
a long time to complete. This allows their interval to be
configured in tests.
* oss: all: use TestWorkerRPCInterval in slow tests
This should speed up these tests significantly.
* oss: internal/tests/cluster: reset sys eventer in test
---------
Co-authored-by: Damian Debkowski <damian.debkowski@hashicorp.com>
* feat(server): Refactor LookupWorkerByName to be a test method
* feat(server): Split worker tags table
* refact(worker): Export api and config tags
* refact(server): Return worker id from UpsertWorkerStatus
* refact(server): Create ListWorkerStorageBucketUpdates domain function
* refact(server): Use dedicated HCPb-worker list func
* refact(server): Create ConnectionRoute domain function
* refact(handlers): Move worker selection logic to internal/server package
This commit moves all the logic to select a worker to handle a session
to the internal/server package. Most functions were moved to this
package with no changes, however, there are a couple of notable
exceptions:
- Given that we don't use GRPC errors in repos, all errors were adjusted
to use the internal/errors package instead.
- StorageBucketFilterCredIdFn is now used to prevent the internal/server
package from importing internal/storage/plugin, which would cause a
dep cycle. Callers of SelectSessionWorkers must implement this func
which is responsible for returning a storage bucket's worker filter as
well as its associated credential id.
- authorizeSessionWithWorkerFilter now receives an
internal/server.(*Repository) object when called. This is transparent
to SelectSessionWorkers callers as it is handled internally.
* feat(server): Add FilteredWithFeatures fn and new server options
* refact(server): create new list and lookup worker queries
* test(server): Add WorkerList unit tests
* feat(server): Implement WorkerList randomness using crypto/rand pkg
* chore(db): add index to server_worker table
* refact(server): prefix test method with Test
* refact(server): rename Downstreamers interface to Graph
* chore(server): unexport and rename method
* chore: add doc string to WorkerAddress (#5420)
---------
Co-authored-by: Irena Rindos <irenarindos@users.noreply.github.com>
Co-authored-by: irenarindos <irena.rindos@gmail.com>
Before this change, upon the completion of a job run, the scheduler
would update a job_run entry with the completed status as well as the
number of objects affected by the run. Additionally, we then had a
cleaner job whose scope was deleting these completed entries from
job_run.
This commit changes this design to drop the cleaner job the and rolls
its functionality directly into the scheduler. Now, when any given job
run is complete, the scheduler will automatically delete it from the
job_run table itself.
In addition, the completed status has been removed as a valid status as
it is not set anymore.
This change follows a decision that given the data in these job_run
entries is already being deleted by the cleaner job, updating its state
when a job completes just to have it deleted is not efficient.
Additionally, no use-case for this data was surfaced.
This adds retries as an attribute to JobStatus allowing a job to
indicate liveness to the scheduler when work has not progressed.
A job could use this and the statusThreshold passed into the Run method
to break work up into batches that would otherwise timeout if all the
work was attempted at once, for example a long running SQL delete
command.
* feat: Add support for capturing URL actions in request info
- Pass request context to services to validate actions.
- Validate actions for auth methods during authentication. `/v1/auth-methods/{{AUTH_METHOD_ID}}:authenticate:callback` is currently only supported on `OIDC` auth methods. An error is thrown if `:callback` is not supported
* api: remove deprecated grant_scope_id field
The grant_scope_id field was deprecated in 0.15.0, so we can remove
it for 0.17.0.
* Fix tests
* Fix more tests
* Remove an option and some related code that stuck around
* Fix bats tests
---------
Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
* oss:feat(storage): add observation tag in proto for telemetry
* oss:feat(storage): add observation tag in proto service
* oss:feat(storage): generate proto files with changes
* oss:feat(storage): add storage bucket id to target proto
* Add observation tags to session_recording.proto
* remove scope_info observation tags
* Remove quotes and add tag to storage_bucket_id
* Add tags for session recording service
* Update proto definitions
- Remove gotags on messages
- Remove quotes
* Add ListResolvableAliases to the user service
There are a few non common ways this method uses our ACL
and grant system. A user is able to list resolvable
aliases for another user if they are granted permission
to do so. That means we had to load the grants for
the user being listed for and not the requester when
determining if an alias is able to resolve to a destination
for which the requested user has permission.
Similarly, the grant hash used for the list pagination
is required to be that of the user being listed and not
the requester.
* internal/gen: better API docs
Add various pieces of metadata that can be used by the API docs renderer.
Improve the service labels by specifying explicit tags.
Specify error responses. Specify version at generation time.
* internal/gen: add better errors
The default error entry is now used to cover all errors, as
adding a new error for every possible status code was a bit much.
This also properly includes the real error type now!
* Add various pieces of metadata that can be used by the API docs renderer. Improve the service labels by specifying explicit tags.
* make auth method API docs
* make API docs better
* make auth methods API docs nicer
* internal/gen: better attribute examples
Replace Output Only comments with a field behavior annotation in accounts.
Mark some fields as required.
Add elaborate description for attributes.
* Make API docs better
* Make API docs better
* Make API docs better
* internal/proto: regenerate files
* internal/proto: finish off account service
* Make API docs better
* internal/proto: add better API docs for aliases
* Tidy up service descriptions
* Added description for Boundary controller API doc
* internal/gen: add better API intro
Copy parts of the API docs into the description
* sdk: add openapi dependency
* internal/gen: move global API doc to separate file
* internal/gen: remove required annotations
Unfortunately, while these help with the creating of resources, they still
appear required in contexts where they are not, such as when
updating a resource.
* Review comments
---------
Co-authored-by: jigar-hashicorp <jigar.shah@hashicorp.com>
Co-authored-by: Isaac Christian (he/him) <112639904+isaacchristian@users.noreply.github.com>
- Add new `local_storage_state` column to `server_worker` table which keeps track of the state of the local disk space.
- The valid values for `local_storage_state` are:
-- `available`: The worker local storage state is at an acceptable state
-- `lowStorage`: The worker is below the minimum threshold for local storage
-- `criticallyLowStorage`: The worker local storage state is below the critical minimum threshold for local storage
-- `outOfStorage`: The worker is out of local disk space
-- `notConfigured`: The worker does not have a local storage path configured
-- `unknown`: The default local storage state of a worker. Used when the local storage state of a worker is not yet known
* Add aliases to target resource and alias resolution to target service
* Wire up the target alias domain package to the target service handler
* Adding alias resolution to target api tests
This allows arbitrary programs to have an easy mechanism to make
connections through Boundary using Go. Internal cluster tests and test
helpers have been migrated to use this package, as well as the `connect`
command.
---------
Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Aditya SharmaandJohan Brandhorst-Satzkorn
Michael Gaffney
Elim Tsiagbey
Guy J Grigsby
Todd
Sepehr
Uruemu S Aganbi
Louis Ruch
Michael Milton
