When a resource relating to a session is deleted,
notably a target, host, or auth_token, the corresponding sessions are
marked as canceled via a database trigger that calls the cancel_session
function. This function was performing a select as part of the insert
call to ensure that it would not attempt to insert a session_sate of
canceled if it was an invalid state transition. However, since this
function was first introduced, there has been additional constraints
added that would prevent an invalid state transition. In addition,
performing a select as part of an insert increases the chances for
contention with other queries that interact with the session_state
table, especially since a large number of sessions and session_state
rows can be included in a single transaction when things like
auth_tokens or targets get deleted.
Now, the cancel_function will perform the insert, and catch any of the
potential violations, allowing the query to continue without needing to
perform the select, thus reduction the potential for contention.
This function was returning the set of deletion tables. A view seems
better suited for this task since it would allow for applying additional
filters of the result set. This was particularly necessary to easily
make changes to some sqltests due to switching the delete trigger for
the session table.
(cherry picked from commit 32d41639892950b21fafc39b50c10b997c6d5121)
* test(billing): Calculate time deltas from a fixed time
Some time calculations are impacted when using the current day vs. the start of the month.
* test(db): Conditionally skip tests at end of month
Certain tests fail when run on the last day of the month because of edge cases in the setup of test data (for example, if today is May 31 and NZ time is June 1). This adds conditional logic to skip those tests when run on the last day of a month.
* CR: typo
- Add new `local_storage_state` column to `server_worker` table which keeps track of the state of the local disk space.
- The valid values for `local_storage_state` are:
-- `available`: The worker local storage state is at an acceptable state
-- `lowStorage`: The worker is below the minimum threshold for local storage
-- `criticallyLowStorage`: The worker local storage state is below the critical minimum threshold for local storage
-- `outOfStorage`: The worker is out of local disk space
-- `notConfigured`: The worker does not have a local storage path configured
-- `unknown`: The default local storage state of a worker. Used when the local storage state of a worker is not yet known
This adds two sql functions that can be used to access the views to
retrieve monthly active user counts. The functions ensure that any time
calculates are performed in UTC, so that the month bounds align with the
start of the month in UTC.
1. hcp_billing_monthly_active_users_last_2_months() will return monthly
active users for the current and previous month.
2. hcp_billing_monthly_active_users_all() will return monthly active
users for all months. This function can also be provided a start time
and or end time to filter the results to a specific range.
Now that there is an auth token fact table in the data warehouse, it is
not possible to insert an auth_token with an old public_id even after
that auth_token was deleted, since it now causes a constraint violation
on the data warehouse table.
With the addition of an after insert trigger to the auth_token table,
entries are added to the wh_user_dimension table when auth token's are
inserted with the 'token issued' status. This caused a number of tests
to fail that assumed the user dimension table would be empty prior to
inserting sessions. This introduces a number of changes to the tests:
- In the test setup, auth tokens are not created with an explicit
expiration time and status. The expiration time ensures that the
tokens are 'valid' for the test. The status allows test to make
assertions on tokens in different statuses, and allow the status to
change during the test.
- Tests make the correct assertions on the number of entries in the user
dimension table after test setup.
- User dimension tests modify a user after the standard setup to ensure
that a new user dimension is added when a session is inserted.
This adds a function and after delete trigger to the auth_token table to
ensure that a corresponding auth token fact is updated with the deleted
time and sets the auth token valid time range. The data warehouse will
only be updated if the token that was deleted is in the 'token issued'
status.
This adds a function and update trigger to auth_token to ensure that:
1. If the auth token is updated to the issued state, a auth token fact
is added to the warehouse.
2. If the auth token's approximate last access time is updated, the
corresponding auth token fact is updated with the new time.
When running the sqltests, sometimes it is helpful to see the full
database container logs, especially when there is an error in a
migration or database setup. However, it also makes it difficult to see
the test results.
This adds a new variable to the makefile to easily toggle the output of
these logs on or off. To run the tests without printing the dtabase logs
use:
DB_LOGS=0 make test
To enable the logs use:
DB_LOGS=1 make test
By default the logs are not printed.
Some of the prepared statement identifies used in these tests exceeded
the identifier character limit. This results in postgres truncating the
name, which is fine for a test, but does result in some noise in the
test output. This change shortens the identifies to eliminate noise like
the following:
psql:/test/credential/vault/credential_vault_library_ssh_private_key_mapping_override.sql:49: NOTICE: identifier "insert_credential_vault_library_ssh_private_key_mapping_override" will be truncated to "insert_credential_vault_library_ssh_private_key_mapping_overrid"
The tests should no longer create notices like these.
* wip
* implement list pagination for auth methods
* remove unused code
* update tests to fix sorting
* address PR comments
* simplify query, add token tests
* change test name
* wip sql testing
* add boolean column to indicate whether auth methods should be returned to unauthenticated users, remove secrets from api response
* is_active_public not null
* remove secrets and unwrapping
* move not null to immediately after data migration
* order public_id descending
* fix(db): Fix auth method migrations (#4193)
* wip
* fix db tests
* fix failing api tests
* reverse order of create time and update time in test
---------
Co-authored-by: Michael Gaffney <mike@gaffney.cc>
Timothy Messier
Hugo
dani
Louis Ruch
Irena Rindos
Damian Debkowski
Michael Li
Elim Tsiagbey
Todd
Jeff MitchellandDamian Debkowski
Johan Brandhorst-Satzkorn
Danielle Miu
Hugo
Michael Milton
