Including the nested connection information in the session list response
had a negative impact on performance as it created a significant
multiplier on the number of rows that needed to be fetched from the
database.
It was also determined that this information was used by neither the
admin ui nor the desktop client.
Note that nested connection information can still be retrieved via a
read request for a specific session.
There is a new periodic job that will delete terminated sessions that
are older than one hour. However, since until now terminated sessions
were not deleted, the first time the job runs there could be a large
number of sessions that need deleted. This could have a negative impact
on a running cluster. To mitigate the operational risk, this adds a
migration to perform an initial delete while the cluster is offline.
* Remove the server_id from session table, change it to worker_id on session_connection
This is step 1 in updating the worker table. The worker is becoming a public resource so it must have
a public id. We drop the server_id column on the session_connection table and add the worker_id column
so we can use the wt_public_key type. Since db migrations are applied while the controller is offline,
it should be safe to assume all session connections are not in a running state, thus allowing us to null
out all the worker ids in the session_connection table.
For now we don't have very good enforcement of the wt_public_id constraints, since the id is currently
defined in the worker config, but this will soon change and the worker's id will be generated by the
controller.
* Remove the server_id from session table, change it to worker_id on session_connection
This is step 1 in updating the worker table. The worker is becoming a public resource so it must have
a public id. We drop the server_id column on the session_connection table and add the worker_id column
so we can use the wt_public_key type. Since db migrations are applied while the controller is offline,
it should be safe to assume all session connections are not in a running state, thus allowing us to null
out all the worker ids in the session_connection table.
For now we don't have very good enforcement of the wt_public_id contraints, since the id is currently
defined in the worker config, but this will soon change and the worker's id will be generated by the
controller.
* Adding worker tag db writing test.
* Move the server table back to the top.
* Update comment for the worker object.
* Update comment for the worker table.
* Add worker tag sources ("config" or "api")
* Revert renaming of UpsertWorker until later.
* Remove no longer needed files.
* Moving the TestWorker into the servers package. Fix the worker tags test.
* Update internal/proto/controller/storage/servers/store/v1/worker.proto
Co-authored-by: Irena Rindos <irenarindos@users.noreply.github.com>
* Place the repos back together so they can be refactored in a different PR.
Co-authored-by: Irena Rindos <irenarindos@users.noreply.github.com>
* Remove the server_id from session table, change it to worker_id on session_connection
This is step 1 in updating the worker table. The worker is becoming a public resource so it must have
a public id. We drop the server_id column on the session_connection table and add the worker_id column
so we can use the wt_public_key type. Since db migrations are applied while the controller is offline,
it should be safe to assume all session connections are not in a running state, thus allowing us to null
out all the worker ids in the session_connection table.
For now we don't have very good enforcement of the wt_public_id contraints, since the id is currently
defined in the worker config, but this will soon change and the worker's id will be generated by the
controller.
* Updating server_id references to controller_id.
* Adding worker tag db writing test.
* Move the server table back to the top.
* Update comment for the worker object.
* Update comment for the worker table.
* Update the newWorkerId call to take a context and fix its error.
* Correct comment for the wt_network_address domain.
* Remove the server_id from session table, change it to worker_id on session_connection
This is step 1 in updating the worker table. The worker is becoming a public resource so it must have
a public id. We drop the server_id column on the session_connection table and add the worker_id column
so we can use the wt_public_key type. Since db migrations are applied while the controller is offline,
it should be safe to assume all session connections are not in a running state, thus allowing us to null
out all the worker ids in the session_connection table.
For now we don't have very good enforcement of the wt_public_id constraints, since the id is currently
defined in the worker config, but this will soon change and the worker's id will be generated by the
controller.
* feature (db): add Db.UnderlyingDB()
This is needed to allow for integration with bits that need a *dbw.DB
* feature (kms): add new go-kms-wrapping/extras/kms dependency
* refactor (kms): schema changes required to adopt extras/kms pkg
* refactor (kms/iam): rename to DeprecateCreateKeysTx(...)
* refactor (kms): remove deprecated test
* refactor (kms): refactor Kms type
This refactor includes renaming the existing type to DeprecatedKms
and then refactoring the Kms type to become a wrapper of
go-kms-wrapping/extras/kms
* refactor (kms): delete all the unneeded bits
All this stuff is not needed now that we're using
go-kms-wrapping/extras/kms
* refactor (db): fixup tests after kms refactor
* refactor (kms/tests): use testing.TB for test helpers
Our test helper packages were not possible to use
with benchmarks. This refactor switches to testing.TB
to allow both tests and benchmarks to use the helpers.
* tests (session benchmarks): refactor to use new Kms type
* refactor (schema): rm test which relies on deprecated Kms code
* refactor (kms): post rebase on main changes required
* chore (kms refactor): re-add unit test from main
* refactor (kms): uses migrations that don't include KMS schema changes
* Move the server daemons into their own package
With this change the internal/server package now is dedicated to the server domain (workers and controllers) and not to the server daemons.
The hope is that this will make it easier to keep the daemons decoupled from the representation of the servers in the database.
When applying the limit and order by, the where clause also needs to be
applied to yield the correct results. This can be seen with an example.
Assuming a data set where there are 100 sessions, with 10 sessions
belonging to a user with id `u_user1` and 90 sessions belonging to
`u_user2`. If the constructed query is like:
select *
from
(select public_id from session limit 10) s,
session_list ss
where
s.public_id = ss.public_id
and s.user_id = 'u_user1'
order by create_time asc;
The initial sub-query with the limit could return sessions only
belonging to `u_user2`, then it would apply the where clause and exclude
all of the rows.
Instead if the query is:
with
session_ids as (
select public_id
from sessions as s
where
s.user_id = 'u_user1'
order by create_time asc
limit 10
)
select *
from session_list
where
session_list.public_id in (select * from session_ids)
order by create_time asc;
The CTE applies the where clause ensuring that only sessions for
`u_user1` are used for the final select from the view.
Note that the CTE is use primarily for readability.
This also adds an index on `session (create_time)` to aide the query
planner in performing the order by.
* feat(host): Add host set information to static host
Updates LookupHost and ListHost API to include sets a static host is part of, as like with plugin hosts.
* test: updated TestList_Plugin and TestCrud to be more consistent under race conditions
Creates a new session_credential table and repository functions
to add and list credentials attached to a session. All credentials
are encrypted using the session scope id.
This change adds a credential type attribute to credential library and
allows it to be set when creating a credential library.
Specifying the type of credential returned from a library allows for new
features and additional error checking to be added over time. A
User Password credential type and an Unspecified credential type are
included with the expectation that additional credential types will be
added in the future. This change also adds the credential type attribute
to existing credential libraries and sets it to Unspecified.
Known Limitations:
- Credential type is only supported on the repository create method.
Support for the other CRUD methods will be added in the future.
Raising a warning in the default trigger used for setting the value in
create_time columns creates unnecessary noise in the PostgreSQL log
files.
Closes#1744
The embed package always uses Unix-style forward slashes. filepath.Join
utilization looking for files would cause the slashes to be the wrong
direction on Windows and result in the files not being found.
Currently host does not use a version, which means it's not protected
against race conditions of either itself or its value objects. This adds
and plumbs through version to ensure that if a host or its value objects
change that they are not going to be affected by a race condition.
* Reference PreferredEndpoints by top level struct name instead of embedded proto GetPreferredEndpoint method.
Reorganize the sql files a little.
* Make PreferredEndpoints replayable. Test refactoring to remove variables that persist across test calls. DeleteItems instead of trying to use WithWhere.
This adds a parameter to host sets allowing the sync interval to be
user-specified on a per-host-set basis. A value of -1 disables syncing;
a value of null uses the current default.
This should help prevent issues like the one fixed in
f2ad4de554.
A version for a particular edition should only exist once, previously
which ever file was read last for a version would be included, which
would lead to some migrations not running.
Todd
Jeff Mitchell
irenarindos
Timothy Messier
Jim
Haotian
Louis Ruch
Michael Gaffney
Michael Gaffney
Thor
Todd Knight