Commit Graph
18 Commits
Author SHA1 Message Date
Johan Brandhorst-SatzkornandDamian Debkowski ec0a3f964b Worker status refactor (#5417)
* oss: feat(proto): statistics rpc added to server coordination service

* oss: feat(session): add update statistics status service

* oss: feat(handlers): add Statistics method to worker service server

* oss: feat(worker): add statistics ticking

* oss: internal/proto: define the new RoutingInfo RPC

* oss: internal/server: repository changes for RoutingInfo RPC

* oss: internal/daemon: add RoutingInfo handler to worker service

The new RoutingInfo RPC is used to get information from the worker
that is used to make routing decisions on the controller. It also
handles the initial request from the worker.

* oss: feat(db): server_worker_session_info table

* oss: feat(proto): session info rpc for server coordination service

* oss: feat(server): worker session info request data type

* oss: feat(worker): session info ticker

* oss: feat(handler): server coordination handler for session info

* oss: feat(session): utilize session info request time for session cleanup

* oss: proto: fix some gotags comments

Some of these fields were missing comments, while others had
incorrectly formatted comments.

* oss: worker: send connection status to server

This is required for the correct tracking of orphaned connections

* oss: internal/daemon/worker: add function to wait for Statistics RPC

* oss: internal/daemon/worker: add routing info RPC

The new RoutingInfo RPC replaces part of the Status RPC,
namely the part which is used by the controller and worker
to perform decisions around the establishment of new sessions
and new worker connections.

* oss: internal/daemon/worker: remove unnecessary error check

* oss: internal/daemon/worker: lock access to config and upstreams

The new confLock protects access to the conf value and to the
addressReceivers slice.

* oss: worker: replace Status RPC with RoutingInfo

* oss: worker: deprecate the Status RPC and related types

The Status RPC and related messages have been replaced
by the RoutingInfo/SessionInfo/Statistics RPCs. The controller
will continue to support the Status RPC until two new minor
versions have been released.

* oss: internal/event: add default deny for new worker RPCs

* oss: internal/daemon/worker: allow RPC interval to be set

The new SessionInfo/RoutingInfo/Statistics RPCs run at
long intervals, which make some of our cluster tests take
a long time to complete. This allows their interval to be
configured in tests.

* oss: all: use TestWorkerRPCInterval in slow tests

This should speed up these tests significantly.

* oss: internal/tests/cluster: reset sys eventer in test

---------

Co-authored-by: Damian Debkowski <damian.debkowski@hashicorp.com>
2025-01-09 10:26:37 -08:00
846c849589 Worker Domain Refactor (#5338)
* feat(server): Refactor LookupWorkerByName to be a test method

* feat(server): Split worker tags table

* refact(worker): Export api and config tags

* refact(server): Return worker id from UpsertWorkerStatus

* refact(server): Create ListWorkerStorageBucketUpdates domain function

* refact(server): Use dedicated HCPb-worker list func

* refact(server): Create ConnectionRoute domain function

* refact(handlers): Move worker selection logic to internal/server package

This commit moves all the logic to select a worker to handle a session
to the internal/server package. Most functions were moved to this
package with no changes, however, there are a couple of notable
exceptions:

- Given that we don't use GRPC errors in repos, all errors were adjusted
  to use the internal/errors package instead.
- StorageBucketFilterCredIdFn is now used to prevent the internal/server
  package from importing internal/storage/plugin, which would cause a
  dep cycle. Callers of SelectSessionWorkers must implement this func
  which is responsible for returning a storage bucket's worker filter as
  well as its associated credential id.
- authorizeSessionWithWorkerFilter now receives an
  internal/server.(*Repository) object when called. This is transparent
  to SelectSessionWorkers callers as it is handled internally.

* feat(server): Add FilteredWithFeatures fn and new server options

* refact(server): create new list and lookup worker queries

* test(server): Add WorkerList unit tests

* feat(server): Implement WorkerList randomness using crypto/rand pkg

* chore(db): add index to server_worker table

* refact(server): prefix test method with Test

* refact(server): rename Downstreamers interface to Graph

* chore(server): unexport and rename method

* chore: add doc string to WorkerAddress (#5420)

---------

Co-authored-by: Irena Rindos <irenarindos@users.noreply.github.com>
Co-authored-by: irenarindos <irena.rindos@gmail.com>
2025-01-08 15:18:03 +00:00
Michael Gaffney 9354a4ebcf refactor(scheduler): Add statusThreshold parameter to job.Run
The scheduler monitors running jobs to make sure they are not hung. If
the status of a running job does not change after a duration (the
interruptThreshold in the scheduler), the scheduler kills the job. This
change passes the scheduler's interruptThreshold to the job when it
calls Run() on the job.
2024-10-25 10:25:51 -04:00
Johan Brandhorst-Satzkorn a830892fec Some test fixes (#5084)
* internal/session: update variable name in test

This was missed in #5071

* internal/tests/cluster: skip failing tests

These tests will need to be fixed or removed. It is not
clear why they are failing, but they are always failing.

* Makefile: try upgrading golangci-lint

* internal/host/plugin: fix flaky test

Hosts contain IP addresses in string form, whose
order is not defined.
2024-09-09 09:22:34 -07:00
Irena Rindos 2191aa1d03 refact(session connection): remove session connection state table (#4617)
* refact(session connection): remove session connection state table
2024-07-23 14:19:21 -04:00
Jeff MitchellandDamian Debkowski 35cef97bd4 Linter fixes PR 3 (#4415)
* Linter fixes

* fix(bsr): update test to register TEST protocol once

---------

Co-authored-by: Damian Debkowski <damian.debkowski@hashicorp.com>
2024-02-16 08:30:16 -05:00
hashicorp-copywrite[bot]andJohan Brandhorst-Satzkorn 29da0bcb92 [COMPLIANCE] License changes (#3567)
* Adding explicit MPL license for sub-package.

This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.

* Adding explicit MPL license for sub-package

This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.

* Updating the license from MPL to Business Source License

Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl.

* Update copyright file headers to BUS-1.1

* Rerun make gen

This will pick up the last of the license changes

* Revert "Rerun make gen"

This reverts commit dbe0968bffd92745d8663c6940debae098395a6b.

* Remove copywrite from generation script

The current version of copywrite is not sophisticated enough
to only apply the copywrite changes to certain subdirectories,
so to avoid accidentally relicensing our api and sdk modules,
we will stop running the copywrite script until such time
that it can do the right thing.

* License protobuf API as MPL

The protobuf API files are used to generate the SDK,
since we want the SDK to be MPL, the protobuf files
must also be MPL. This extends to all protobuf files
whose sources end up in the SDK directory, including:

- internal/proto/controller/api
- internal/proto/controller/custom_options
- internal/proto/plugin

It also includes the generated service directory in
internal/gen/controller/api/services, since they are
generated from the protobuf sources used for the SDK.

* Change website license file name

This makes the naming scheme consistent with the rest
of the repository.

* Add note on licensing to README

* Set copywrite license to BUSL

* Revert "Add note on licensing to README"

This reverts commit 4d865a91a7a4f11605f0dc4609355d7e3c3cfffe.

---------

Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
2023-08-11 10:31:53 -07:00
Jeff Mitchell 8322916fb4 Remove all deprecated Err/Wrap/E calls (#3286) 2023-06-14 21:55:24 -04:00
Johan Brandhorst-Satzkorn 3c29308673 chore: Add license headers to all files 2023-02-03 15:54:27 -08:00
Danielle 46037cf85a Add Staticcheck to Build Pipeline (#2421)
Run staticcheck linters on pull requests
2022-12-13 12:07:18 -05:00
Jeff MitchellandTodd 0c96c6ff6c Split grace period into multiple config values (#2578) (#2603)
Co-authored-by: Todd <github@quaddi.com>
2022-11-10 13:14:57 -05:00
Timothy Messier e65f1f32a7 chore(session): Remove repository method that was only used in tests
This unexported method was only used by a single test, and was taking
advantage of its implmentation to populate Connections rather than
Sessions. This repository should only interact with Sessions, since
there is a separate repository for Connections.
2022-09-07 17:53:40 +00:00
Timothy Messier eb61ac6304 refact(session): Add context to session factory
This adds a context to the session.RepositoryFactory, allowing
errors.NewDeprecated to be replaced with errors.New. It also removes the
common.SessionRepoFactory, instead packages just directly import from
the session package.
2022-09-07 17:53:40 +00:00
Todd 19b549c44a Rename package servers to server (#2222) 2022-06-20 18:20:17 -07:00
Todd e379234259 Remove CanonicalAddress, GetApiTags(), and NewWorkerForStatus. (#2196)
Minor testing tweaks are also included.
2022-06-15 11:47:38 -07:00
Todd beecbbb8a2 Upsert creates a new KMS and Updates PKI workers (#2187) 2022-06-15 08:27:08 -07:00
Jeff Mitchell 5d3facf561 Merge remote-tracking branch 'origin/main' into llb-byow 2022-06-10 19:05:27 -04:00
Timothy Messier a4d0e3c7f8 refact(session): Rename files with job to use a prefix
This makes it so jobs will all be naturally grouped together in file
listing.
2022-06-09 19:28:03 +00:00