088564c3853229dec58f973a3cfabce70feefbdc
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Timothy Messier
|
21230e1c61 |
db: Re-order migrations for next release (#5146)
Refs: |
||
|
Timothy Messier
|
37bf59243a |
refact(session): Replace state start_time/end_time with time range (#5134)
This replaces the start_time, end_time, and previous_end_time columns on
the session_state table with a time range column that records when the
state was active. This allowed for dropping several constraints on the
session_state table that were used to ensure that states for a session
did not overlap. Instead this can be ensured with a single exclusion
constraint. By reducing the number of constraints, it improves the
performance for a number of queries involved with session state. Most
notably when resolving delete cascades of resources related to sessions,
such as targets, where as part of the delete transaction large numbers
of sessions get canceled. Similarly, when sessions get deleted, and the
corresponding session_state rows are deleted, these constraints all must
be checked during the transaction.
The domain layer continues to expose StartTime and EndTime fields on
session.State. However it drops PreviousEndTime, which was never used by
anything calling the domain layer.
See:
https://www.postgresql.org/docs/current/ddl-constraints.html#DDL-CONSTRAINTS-EXCLUSION
https://www.postgresql.org/docs/current/rangetypes.html
|
||
|
Timothy Messier
|
5405a275ab | feat(census): Update census tables to support additional metrics | ||
  hashicorp-copywrite[bot]andJohan Brandhorst-Satzkorn
|
29da0bcb92 |
[COMPLIANCE] License changes (#3567)
* Adding explicit MPL license for sub-package. This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl. * Update copyright file headers to BUS-1.1 * Rerun make gen This will pick up the last of the license changes * Revert "Rerun make gen" This reverts commit dbe0968bffd92745d8663c6940debae098395a6b. * Remove copywrite from generation script The current version of copywrite is not sophisticated enough to only apply the copywrite changes to certain subdirectories, so to avoid accidentally relicensing our api and sdk modules, we will stop running the copywrite script until such time that it can do the right thing. * License protobuf API as MPL The protobuf API files are used to generate the SDK, since we want the SDK to be MPL, the protobuf files must also be MPL. This extends to all protobuf files whose sources end up in the SDK directory, including: - internal/proto/controller/api - internal/proto/controller/custom_options - internal/proto/plugin It also includes the generated service directory in internal/gen/controller/api/services, since they are generated from the protobuf sources used for the SDK. * Change website license file name This makes the naming scheme consistent with the rest of the repository. * Add note on licensing to README * Set copywrite license to BUSL * Revert "Add note on licensing to README" This reverts commit 4d865a91a7a4f11605f0dc4609355d7e3c3cfffe. --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> |
||
|
Timothy Messier
|
876f9bb07f |
fix(session): List performance with large number of session connections
As part of the performance improvements for listing sessions for v0.9.0, the view used to retrieve sessions for listing was updated to no longer include a join on session_connection. However, a later migration that was adding additional columns to this view inadvertently re-added the join and session_connection columns. This resulted in requests to list sessions performing this join, and returning additional rows for each connection in a session. The application would ignore this additional data, since it is no longer returned via the API. This would be particularly impactful in cases where there are a large number of connections per session, as this would greatly increase the amount of data returned by the database. This fix removes the session_connection join from this view, which should restore the original performance improvements even in cases where there is a large number of session connections per session. See: https://github.com/hashicorp/boundary/pull/2160 Ref: |
||
 Michael Gaffney
|
99a8777866 |
fix(db): Add index to session connection table
This index improves the performance for two queries on the
`session_connection` table. The execution time for the first query went
from 40.105 ms to 1.405 ms. The execution time for the second query went
from 13.808 ms to 0.217 ms.
Query 1 explain plan before index:
Update on session_connection (cost=6465.17..6557.68 rows=11 width=50) (actual time=38.868..39.147 rows=3 loops=1)
-> Nested Loop (cost=6465.17..6557.68 rows=11 width=50) (actual time=38.471..38.506 rows=3 loops=1)
-> HashAggregate (cost=6464.75..6464.86 rows=11 width=40) (actual time=38.447..38.450 rows=3 loops=1)
Group Key: (session_connection_1.public_id)::text
Batches: 1 Memory Usage: 24kB
-> Nested Loop Semi Join (cost=0.42..6464.72 rows=11 width=40) (actual time=29.184..38.434 rows=3 loops=1)
-> Seq Scan on session_connection session_connection_1 (cost=0.00..6329.38 rows=16 width=20) (actual time=15.744..38.014 rows=23 loops=1)
Filter: ((worker_id)::text = 'w__________k'::text)
Rows Removed by Filter: 163607
-> Index Scan using session_connection_state_connection_id_end_time_key on session_connection_state (cost=0.42..8.45 rows=1 width=20) (actual time=0.016..0.016 rows=0 loops=23)
Index Cond: (((connection_id)::text = (session_connection_1.public_id)::text) AND (end_time IS NULL))
Filter: ((state = ANY ('{authorized,connected}'::text[])) AND (start_time < (CURRENT_TIMESTAMP - '00:00:05'::interval)))
Rows Removed by Filter: 1
-> Index Scan using session_connection_pkey on session_connection (cost=0.42..8.44 rows=1 width=20) (actual time=0.016..0.016 rows=1 loops=3)
Index Cond: ((public_id)::text = (session_connection_1.public_id)::text)
Planning Time: 0.793 ms
Trigger immutable_columns: time=0.474 calls=3
Trigger update_connection_state_on_closed_reason: time=0.159 calls=3
Trigger update_time_column: time=0.034 calls=3
Trigger update_version_column: time=0.041 calls=3
Trigger wh_update_session_connection: time=0.688 calls=3
Execution Time: 40.105 ms
Query 1 explain plan after index:
Update on session_connection (cost=152.71..219.87 rows=8 width=50) (actual time=0.497..0.683 rows=3 loops=1)
-> Nested Loop (cost=152.71..219.87 rows=8 width=50) (actual time=0.266..0.292 rows=3 loops=1)
-> HashAggregate (cost=152.29..152.37 rows=8 width=40) (actual time=0.256..0.258 rows=3 loops=1)
Group Key: (session_connection_1.public_id)::text
Batches: 1 Memory Usage: 24kB
-> Nested Loop Semi Join (cost=4.94..152.27 rows=8 width=40) (actual time=0.199..0.251 rows=3 loops=1)
-> Bitmap Heap Scan on session_connection session_connection_1 (cost=4.51..50.76 rows=12 width=20) (actual time=0.017..0.035 rows=10 loops=1)
Recheck Cond: ((worker_id)::text = 'w__________k'::text)
Heap Blocks: exact=9
-> Bitmap Index Scan on session_connection_worker_id_closed_reason_ix (cost=0.00..4.51 rows=12 width=0) (actual time=0.011..0.011 rows=10 loops=1)
Index Cond: ((worker_id)::text = 'w__________k'::text)
-> Index Scan using session_connection_state_connection_id_end_time_key on session_connection_state (cost=0.42..8.45 rows=1 width=20) (actual time=0.021..0.021 rows=0 loops=10)
Index Cond: (((connection_id)::text = (session_connection_1.public_id)::text) AND (end_time IS NULL))
Filter: ((state = ANY ('{authorized,connected}'::text[])) AND (start_time < (CURRENT_TIMESTAMP - '00:00:05'::interval)))
Rows Removed by Filter: 1
-> Index Scan using session_connection_pkey on session_connection (cost=0.42..8.44 rows=1 width=20) (actual time=0.010..0.010 rows=1 loops=3)
Index Cond: ((public_id)::text = (session_connection_1.public_id)::text)
Planning Time: 0.614 ms
Trigger immutable_columns: time=0.279 calls=3
Trigger update_connection_state_on_closed_reason: time=0.185 calls=3
Trigger update_time_column: time=0.027 calls=3
Trigger update_version_column: time=0.032 calls=3
Trigger wh_update_session_connection: time=0.450 calls=3
Execution Time: 1.405 ms
Query 2 explain plan before index:
Limit (cost=1002.40..6147.94 rows=1 width=253) (actual time=12.244..13.726 rows=1 loops=1)
-> Nested Loop Left Join (cost=1002.40..6147.93 rows=1 width=253) (actual time=12.243..13.724 rows=1 loops=1)
Join Filter: ((w.public_id)::text = (session_connection.worker_id)::text)
CTE worker_config_tags
-> GroupAggregate (cost=1.71..2.26 rows=22 width=59) (actual time=0.044..0.082 rows=22 loops=1)
Group Key: ct_1.worker_id, ct_1.source
-> Sort (cost=1.71..1.77 rows=22 width=49) (actual time=0.031..0.034 rows=22 loops=1)
Sort Key: ct_1.worker_id, ct_1.source
Sort Method: quicksort Memory: 27kB
-> Seq Scan on server_worker_tag ct_1 (cost=0.00..1.22 rows=22 width=49) (actual time=0.007..0.010 rows=22 loops=1)
-> Nested Loop Left Join (cost=0.14..9.28 rows=1 width=245) (actual time=0.106..0.108 rows=1 loops=1)
Join Filter: ((w.public_id)::text = (ct.worker_id)::text)
-> Nested Loop Left Join (cost=0.14..8.72 rows=1 width=213) (actual time=0.103..0.104 rows=1 loops=1)
Join Filter: ((w.public_id)::text = (wt.worker_id)::text)
-> Index Scan using server_worker_pkey on server_worker w (cost=0.14..8.15 rows=1 width=181) (actual time=0.012..0.012 rows=1 loops=1)
Index Cond: ((public_id)::text = 'w__________k'::text)
-> CTE Scan on worker_config_tags wt (cost=0.00..0.55 rows=1 width=64) (actual time=0.090..0.090 rows=0 loops=1)
Filter: (((worker_id)::text = 'w__________k'::text) AND (source = 'api'::text))
Rows Removed by Filter: 22
-> CTE Scan on worker_config_tags ct (cost=0.00..0.55 rows=1 width=64) (actual time=0.001..0.002 rows=1 loops=1)
Filter: (((worker_id)::text = 'w__________k'::text) AND (source = 'configuration'::text))
Rows Removed by Filter: 8
-> GroupAggregate (cost=1000.00..6136.36 rows=1 width=21) (actual time=12.133..13.610 rows=0 loops=1)
Group Key: session_connection.worker_id
-> Gather (cost=1000.00..6136.35 rows=1 width=13) (actual time=12.132..13.608 rows=0 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Parallel Seq Scan on session_connection (cost=0.00..5136.25 rows=1 width=13) (actual time=9.012..9.012 rows=0 loops=3)
Filter: ((closed_reason IS NULL) AND ((worker_id)::text = 'w__________k'::text))
Rows Removed by Filter: 54544
Planning Time: 0.398 ms
Execution Time: 13.808 ms
Query 2 explain plan after index:
Limit (cost=2.82..20.03 rows=1 width=253) (actual time=0.132..0.134 rows=1 loops=1)
-> Nested Loop Left Join (cost=2.82..20.02 rows=1 width=253) (actual time=0.131..0.133 rows=1 loops=1)
Join Filter: ((w.public_id)::text = (session_connection.worker_id)::text)
CTE worker_config_tags
-> GroupAggregate (cost=1.71..2.26 rows=22 width=59) (actual time=0.046..0.086 rows=22 loops=1)
Group Key: ct_1.worker_id, ct_1.source
-> Sort (cost=1.71..1.77 rows=22 width=49) (actual time=0.033..0.035 rows=22 loops=1)
Sort Key: ct_1.worker_id, ct_1.source
Sort Method: quicksort Memory: 27kB
-> Seq Scan on server_worker_tag ct_1 (cost=0.00..1.22 rows=22 width=49) (actual time=0.007..0.011 rows=22 loops=1)
-> Nested Loop Left Join (cost=0.14..9.28 rows=1 width=245) (actual time=0.120..0.120 rows=1 loops=1)
Join Filter: ((w.public_id)::text = (ct.worker_id)::text)
-> Nested Loop Left Join (cost=0.14..8.72 rows=1 width=213) (actual time=0.117..0.118 rows=1 loops=1)
Join Filter: ((w.public_id)::text = (wt.worker_id)::text)
-> Index Scan using server_worker_pkey on server_worker w (cost=0.14..8.15 rows=1 width=181) (actual time=0.011..0.012 rows=1 loops=1)
Index Cond: ((public_id)::text = 'w__________k'::text)
-> CTE Scan on worker_config_tags wt (cost=0.00..0.55 rows=1 width=64) (actual time=0.104..0.105 rows=0 loops=1)
Filter: (((worker_id)::text = 'w__________k'::text) AND (source = 'api'::text))
Rows Removed by Filter: 22
-> CTE Scan on worker_config_tags ct (cost=0.00..0.55 rows=1 width=64) (actual time=0.002..0.002 rows=1 loops=1)
Filter: (((worker_id)::text = 'w__________k'::text) AND (source = 'configuration'::text))
Rows Removed by Filter: 8
-> GroupAggregate (cost=0.42..8.45 rows=1 width=21) (actual time=0.010..0.010 rows=0 loops=1)
Group Key: session_connection.worker_id
-> Index Only Scan using session_connection_worker_id_closed_reason_ix on session_connection (cost=0.42..8.44 rows=1 width=13) (actual time=0.010..0.010 rows=0 loops=1)
Index Cond: ((worker_id = 'w__________k'::text) AND (closed_reason IS NULL))
Heap Fetches: 0
Planning Time: 0.544 ms
Execution Time: 0.217 ms
Closes #3281
|
||
|
Michael Gaffney
|
053e74f66c | feat(db): Add table for census upload job status |