This unexported method was only used by a single test, and was taking
advantage of its implmentation to populate Connections rather than
Sessions. This repository should only interact with Sessions, since
there is a separate repository for Connections.
This adds a context to the session.RepositoryFactory, allowing
errors.NewDeprecated to be replaced with errors.New. It also removes the
common.SessionRepoFactory, instead packages just directly import from
the session package.
The session repository can now be passed an optional set of permissions.
The permissions are used to restrict results from the repository.
Currently this only impacts the List method. This enabled the sessions
service to use a single query to retrieve sessions and replaces the
previous behavior where the session service would first request all
sessions for a given set of scopes, evaluate the permissions, and then
request sessions based on a set of session ids.
* feat(credentials): Refactor credential purposes
* Application credentials have been refactored to Brokered credentials
* Egress credentials have been refactored to Injected Application
credentials
Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
It was determined that it is a more common use case for users to request
non-terminated sessions (i.e. sessions in `pending`, `active`, or
`canceling` when requesting sessions. Notably, end users would need
non-terminated sessions to:
- Establish a new connection to a session.
- Terminate a session
So now by default a list request to `/v1/sessions` will *not* include
terminated sessions.
To still enable use cases where terminated sessions are needed, this
adds a new query parameter to include the terminated session:
`/v1/sessions?include_terminated=true`.
Note that this query parameter is considered experimental and subject to
change.
Including the nested connection information in the session list response
had a negative impact on performance as it created a significant
multiplier on the number of rows that needed to be fetched from the
database.
It was also determined that this information was used by neither the
admin ui nor the desktop client.
Note that nested connection information can still be retrieved via a
read request for a specific session.
* Remove the server_id from session table, change it to worker_id on session_connection
This is step 1 in updating the worker table. The worker is becoming a public resource so it must have
a public id. We drop the server_id column on the session_connection table and add the worker_id column
so we can use the wt_public_key type. Since db migrations are applied while the controller is offline,
it should be safe to assume all session connections are not in a running state, thus allowing us to null
out all the worker ids in the session_connection table.
For now we don't have very good enforcement of the wt_public_id constraints, since the id is currently
defined in the worker config, but this will soon change and the worker's id will be generated by the
controller.
* Remove the server_id from session table, change it to worker_id on session_connection
This is step 1 in updating the worker table. The worker is becoming a public resource so it must have
a public id. We drop the server_id column on the session_connection table and add the worker_id column
so we can use the wt_public_key type. Since db migrations are applied while the controller is offline,
it should be safe to assume all session connections are not in a running state, thus allowing us to null
out all the worker ids in the session_connection table.
For now we don't have very good enforcement of the wt_public_id contraints, since the id is currently
defined in the worker config, but this will soon change and the worker's id will be generated by the
controller.
* Adding worker tag db writing test.
* Move the server table back to the top.
* Update comment for the worker object.
* Update comment for the worker table.
* Add worker tag sources ("config" or "api")
* Revert renaming of UpsertWorker until later.
* Remove no longer needed files.
* Moving the TestWorker into the servers package. Fix the worker tags test.
* Update internal/proto/controller/storage/servers/store/v1/worker.proto
Co-authored-by: Irena Rindos <irenarindos@users.noreply.github.com>
* Place the repos back together so they can be refactored in a different PR.
Co-authored-by: Irena Rindos <irenarindos@users.noreply.github.com>
* Remove the server_id from session table, change it to worker_id on session_connection
This is step 1 in updating the worker table. The worker is becoming a public resource so it must have
a public id. We drop the server_id column on the session_connection table and add the worker_id column
so we can use the wt_public_key type. Since db migrations are applied while the controller is offline,
it should be safe to assume all session connections are not in a running state, thus allowing us to null
out all the worker ids in the session_connection table.
For now we don't have very good enforcement of the wt_public_id contraints, since the id is currently
defined in the worker config, but this will soon change and the worker's id will be generated by the
controller.
* Updating server_id references to controller_id.
* Adding worker tag db writing test.
* Move the server table back to the top.
* Update comment for the worker object.
* Update comment for the worker table.
* Update the newWorkerId call to take a context and fix its error.
* Correct comment for the wt_network_address domain.
* Remove the server_id from session table, change it to worker_id on session_connection
This is step 1 in updating the worker table. The worker is becoming a public resource so it must have
a public id. We drop the server_id column on the session_connection table and add the worker_id column
so we can use the wt_public_key type. Since db migrations are applied while the controller is offline,
it should be safe to assume all session connections are not in a running state, thus allowing us to null
out all the worker ids in the session_connection table.
For now we don't have very good enforcement of the wt_public_id constraints, since the id is currently
defined in the worker config, but this will soon change and the worker's id will be generated by the
controller.
* feature (db): add Db.UnderlyingDB()
This is needed to allow for integration with bits that need a *dbw.DB
* feature (kms): add new go-kms-wrapping/extras/kms dependency
* refactor (kms): schema changes required to adopt extras/kms pkg
* refactor (kms/iam): rename to DeprecateCreateKeysTx(...)
* refactor (kms): remove deprecated test
* refactor (kms): refactor Kms type
This refactor includes renaming the existing type to DeprecatedKms
and then refactoring the Kms type to become a wrapper of
go-kms-wrapping/extras/kms
* refactor (kms): delete all the unneeded bits
All this stuff is not needed now that we're using
go-kms-wrapping/extras/kms
* refactor (db): fixup tests after kms refactor
* refactor (kms/tests): use testing.TB for test helpers
Our test helper packages were not possible to use
with benchmarks. This refactor switches to testing.TB
to allow both tests and benchmarks to use the helpers.
* tests (session benchmarks): refactor to use new Kms type
* refactor (schema): rm test which relies on deprecated Kms code
* refactor (kms): post rebase on main changes required
* chore (kms refactor): re-add unit test from main
* refactor (kms): uses migrations that don't include KMS schema changes
* Move the session cleanup job logic into the session package.
This is part of a broader effort to more narrowly define the purpose of the internal/server directory to be for interacting with servers themselves. Future changes will continue to remove things from this directory until the purpose of the directory is clearly working primarily with the boundary servers resource.
When applying the limit and order by, the where clause also needs to be
applied to yield the correct results. This can be seen with an example.
Assuming a data set where there are 100 sessions, with 10 sessions
belonging to a user with id `u_user1` and 90 sessions belonging to
`u_user2`. If the constructed query is like:
select *
from
(select public_id from session limit 10) s,
session_list ss
where
s.public_id = ss.public_id
and s.user_id = 'u_user1'
order by create_time asc;
The initial sub-query with the limit could return sessions only
belonging to `u_user2`, then it would apply the where clause and exclude
all of the rows.
Instead if the query is:
with
session_ids as (
select public_id
from sessions as s
where
s.user_id = 'u_user1'
order by create_time asc
limit 10
)
select *
from session_list
where
session_list.public_id in (select * from session_ids)
order by create_time asc;
The CTE applies the where clause ensuring that only sessions for
`u_user1` are used for the final select from the view.
Note that the CTE is use primarily for readability.
This also adds an index on `session (create_time)` to aide the query
planner in performing the order by.
Sometimes when creating a lot of sessions concurrently we
can run into a unique constraint error in the warehouse triggers
in the database. Simply retrying the call will allow the
session creation to succeed.
Our test helper packages were not possible to use
with benchmarks. This refactor switches to testing.TB
to allow both tests and benchmarks to use the helpers.
* Reorder session authz checks
This implements a mechanism for service handlers to pass a repo
satisfying an interface into the scope IDs listing function. The purpose
of this interface is to allow the same function to perform authorization
checking against resources rather than requiring service handlers to
fetch all resources in the determined set of scopes and then perform
this checking. While this could be done in each service handler, this
approach centralizes the logic, which has the nice benefit of removing
some boilerplate from service handler list functions that have adapted
to it.
Tests haven't changed, which is expected -- the function should return
exactly what was being returned before, simply faster.
This allows us to use SNI for actual host (e.g. routing) information. We
don't have the client cert yet so we can't look at that, so this
provides a way for us to convey the information needed to look up that
session's TLS stack.
Using SNI for hosts means we also run into the fact that we don't have
automatic agreement in terms of SANs. So when generating the certs we
now also pass worker address information to the function to be encoded
in the cert.
Finally, there is a change in how the websocket dialing happens, because
http.RoundTripper tries to be too clever for its own good and overwrites
NextProtos on a whim.
This removes or reduces the time that is waited in a number of tests by
allowing the grace period time be a configuration option on the
connection repository.
This creates a domain service function `session.WorkerStatusReport`. The
service performs two functions:
1. Checks the status for all of the reported sessions, returning a
report of the sessions that are in a "non-active" state, i.e.
"cacneling" or "terminated".
2. Looks for any "orphaned" session connections, i.e. connections that
are active but were not reported by the worker. These connections are
marked as closed.
This also removes the "ShouldCloseConnectionsOnWorker" logic, as this
was determined to not be a possible state, and was unnecessary.
The query for CloseDeadConnectionsForWorker includes a time comparison
to exclude results that are too close to the current time. This is done
to account for a delay in reporting by the worker. However, since the
delay was hardcoded in the query, it made it difficult to test.
This moves the delay duration into a parameter that is set on the
repository. The amount of the delay can be set at repository creation
time using the new WithWorkerStateDelay option. This defaults to the
previously hardcoded amount.
This will allow tests to control the delay to exercise this
functionality fully.
Hugo Vieira
Timothy Messier
Johan Brandhorst-Satzkorn
Damian Debkowski
Louis Ruch
Jeff Malnick
Todd
Jim
irenarindos
Lars Lehtonen