Rename root, api and sdk module paths (imports, go.mod, go_package,
ldflags, build scripts, buf configs) and regenerate protobuf/gateway
code with the project's pinned toolchain. Wire identifiers unchanged.
api gets a local replace for sdk; root/api/sdk require v0.1.0.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* oss: feat(proto): statistics rpc added to server coordination service
* oss: feat(session): add update statistics status service
* oss: feat(handlers): add Statistics method to worker service server
* oss: feat(worker): add statistics ticking
* oss: internal/proto: define the new RoutingInfo RPC
* oss: internal/server: repository changes for RoutingInfo RPC
* oss: internal/daemon: add RoutingInfo handler to worker service
The new RoutingInfo RPC is used to get information from the worker
that is used to make routing decisions on the controller. It also
handles the initial request from the worker.
* oss: feat(db): server_worker_session_info table
* oss: feat(proto): session info rpc for server coordination service
* oss: feat(server): worker session info request data type
* oss: feat(worker): session info ticker
* oss: feat(handler): server coordination handler for session info
* oss: feat(session): utilize session info request time for session cleanup
* oss: proto: fix some gotags comments
Some of these fields were missing comments, while others had
incorrectly formatted comments.
* oss: worker: send connection status to server
This is required for the correct tracking of orphaned connections
* oss: internal/daemon/worker: add function to wait for Statistics RPC
* oss: internal/daemon/worker: add routing info RPC
The new RoutingInfo RPC replaces part of the Status RPC,
namely the part which is used by the controller and worker
to perform decisions around the establishment of new sessions
and new worker connections.
* oss: internal/daemon/worker: remove unnecessary error check
* oss: internal/daemon/worker: lock access to config and upstreams
The new confLock protects access to the conf value and to the
addressReceivers slice.
* oss: worker: replace Status RPC with RoutingInfo
* oss: worker: deprecate the Status RPC and related types
The Status RPC and related messages have been replaced
by the RoutingInfo/SessionInfo/Statistics RPCs. The controller
will continue to support the Status RPC until two new minor
versions have been released.
* oss: internal/event: add default deny for new worker RPCs
* oss: internal/daemon/worker: allow RPC interval to be set
The new SessionInfo/RoutingInfo/Statistics RPCs run at
long intervals, which make some of our cluster tests take
a long time to complete. This allows their interval to be
configured in tests.
* oss: all: use TestWorkerRPCInterval in slow tests
This should speed up these tests significantly.
* oss: internal/tests/cluster: reset sys eventer in test
---------
Co-authored-by: Damian Debkowski <damian.debkowski@hashicorp.com>
This has been superceded by the nodeenrollment-based KMS worker auth
mechanism. This commit only removes the ability to connect with this
mechanism; some upgrade logic will remain intact for a bit and
can be removed later.
* Adding explicit MPL license for sub-package.
This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.
* Adding explicit MPL license for sub-package
This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.
* Updating the license from MPL to Business Source License
Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl.
* Update copyright file headers to BUS-1.1
* Rerun make gen
This will pick up the last of the license changes
* Revert "Rerun make gen"
This reverts commit dbe0968bffd92745d8663c6940debae098395a6b.
* Remove copywrite from generation script
The current version of copywrite is not sophisticated enough
to only apply the copywrite changes to certain subdirectories,
so to avoid accidentally relicensing our api and sdk modules,
we will stop running the copywrite script until such time
that it can do the right thing.
* License protobuf API as MPL
The protobuf API files are used to generate the SDK,
since we want the SDK to be MPL, the protobuf files
must also be MPL. This extends to all protobuf files
whose sources end up in the SDK directory, including:
- internal/proto/controller/api
- internal/proto/controller/custom_options
- internal/proto/plugin
It also includes the generated service directory in
internal/gen/controller/api/services, since they are
generated from the protobuf sources used for the SDK.
* Change website license file name
This makes the naming scheme consistent with the rest
of the repository.
* Add note on licensing to README
* Set copywrite license to BUSL
* Revert "Add note on licensing to README"
This reverts commit 4d865a91a7a4f11605f0dc4609355d7e3c3cfffe.
---------
Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Most of the changes here are changes or enhancements that are useful for
tests. The crux of the issue was that I had pulled up one var to the
package level so it could be read from tests, but that var was not being
reset between tests. It's not even used in tests anymore, so I put it
back to function-local.
* Add more logging to worker auth rotation and test flags
This makes it much easier to figure out when and why rotation is
happening and control it for tests/debugging.
* Update auth rotation logic
Because we were looking at validity period of client certs, which may
not have matched the actual rotation interval, it was possible (mostly
in test scenarios) for the rotation to happen multiple times on the
roots before we actually hit the worker rotation interval.
This changes how the interval is calculated by looking at the difference
between not-after values on the certs to understand the rotation
interval being used by the controller. We then use that as the starting
point for calculating a next-rotation time.
Do not allow PKI-KMS workers to have names updated via API
This would cause their public IDs to change, which we do not want.
Descriptions are also disallowed, since those come from the config file.
This also adds logic to allow upgrading from old KMS method to new, with
a test.
Co-authored-by: Irena Rindos <irenarindos@users.noreply.github.com>
This adds a new mechanism for worker authentication that uses the
third-party arbitration of the original KMS flow by leveraging
recently-added capabilities in nodeenrollment to support a registration
wrapper. This allows for workers to be registered to the system with
just a name and a matching wrapper, while at the same time allowing
these workers to be used for private Vault access and multi-hop as they
contain per-worker encryption keys and support rotation.
Nodes using this mode will generate and rotate credentials in-memory
(using another recently added nodeenrollment capability) so will have a
new set of keys on every startup. This has the side effect of ensuring
that there is never a conflict or re-use of these credentials.
Note that the normal test worker flow was using the now-deprecated KMS
method; there is still a test that uses this (explicitly) but all other
tests using test workers that did not specify a specific PKI flow now
actually use this mechanism. This means there is fairly decent test
coverage in a general sense, plus the specific tests updated/added for
this feature.
This also adds a new KMS type that can be used to have
distinct KMSes for upstream authentication vs. accepting from
downstream.
* Clean up some dev config kms code
Although it didn't matter in the combined state, the KMS for auth
storage was not actually being set properly if only a worker was being
instantiated.
This essentially worked for non-dev mode before but would create storage
directories that weren't actually used or needed, and it didn't work in
dev mode because it made assumptions about the presence of a worker auth
kms or not. This fixes those behaviors.
* Don't automatically create worker names, as this is now invalid for
PKI workers
* Fix a check in dev that clears out name/description in exactly the
wrong cases
* Change eventing setup to use a hostname-based ID instead of the name
(which could change on every start anyways)
* Adds WorkerAuthStorage wrapper for worker credential encryption using nodeenrollment library.
* Adds AuthStoragePath field in worker config and dev mode to specify directory to store credentials.
* Discontinue support for old Server message in ServerCoordinationService. Split message into ServerWorkerStatus and UpstreamServer (which currently contains controller addresses). Remove unnecessary fields. Modify workerStatusServer to use new fields.
Co-authored-by: Todd <github@quaddi.com>
* Remove the server_id from session table, change it to worker_id on session_connection
This is step 1 in updating the worker table. The worker is becoming a public resource so it must have
a public id. We drop the server_id column on the session_connection table and add the worker_id column
so we can use the wt_public_key type. Since db migrations are applied while the controller is offline,
it should be safe to assume all session connections are not in a running state, thus allowing us to null
out all the worker ids in the session_connection table.
For now we don't have very good enforcement of the wt_public_id constraints, since the id is currently
defined in the worker config, but this will soon change and the worker's id will be generated by the
controller.
* Move the server daemons into their own package
With this change the internal/server package now is dedicated to the server domain (workers and controllers) and not to the server daemons.
The hope is that this will make it easier to keep the daemons decoupled from the representation of the servers in the database.
shadowdaoandClaude Sonnet 5.5
Chandrasekhar Gadde
aditya2548
Johan Brandhorst-SatzkornandDamian Debkowski
daniandDamian Debkowski
Jeff MitchellandDamian Debkowski
Irena Rindos
Louis Ruch
Jim
Haotian