Commit Graph
4358 Commits
Author SHA1 Message Date
Timothy Messier 632d048f7b lint: Fix linter errors
(cherry picked from commit 51a2b2035f9c6bb51fe8f5f1defeeb514183fecd)
2024-09-24 14:18:54 +00:00
Timothy Messier a92c88dfb5 refact(db): Replace get_deleted_tables function with a view
This function was returning the set of deletion tables. A view seems
better suited for this task since it would allow for applying additional
filters of the result set. This was particularly necessary to easily
make changes to some sqltests due to switching the delete trigger for
the session table.

(cherry picked from commit 32d41639892950b21fafc39b50c10b997c6d5121)
2024-09-24 14:18:54 +00:00
Timothy Messier 33037b4887 perf(db): Use statement trigger for marking deleted sessions
When sessions are deleted, a trigger is used to insert records into the
session_deleted table. This table is utilized by the sessions list
endpoint when using a refresh token to inform a client of the sessions
that have been deleted since the last request. We delete sessions in
bulk via a controller job to delete sessions that have been terminated
over an hour ago, which results in the trigger running a large number of
separate insert statements while processing the delete statement.

This changes the trigger to run once for the delete statement, instead
of for each row, resulting in a single bulk insert statement to the
session_deleted table.

This new trigger function also avoids the use of `on conflict`. When
testing this function, while the single statement was still faster than
running multiple inserts, the `on conflict` still added significant
overhead, even when there were no conflicts. It should be safe to
perform the insert without the `on conflict`, since the same ID should
never be deleted more than once if it is successfully deleted.

(cherry picked from commit 3569b36c6eb2bee7a4e36e7584fcc6bbf1124f95)
2024-09-24 14:18:54 +00:00
Timothy Messier 091ed6bacb perf(db): Reorder index columns for primary keys
When processing a controller API request, system uses a query to fetch
the grants for the requesting user. This query requires pulling together
information from several tables, and is currently performing many
sequential scans to do so. For a number of the tables involved, there
are indexes from multi-column primary keys, however, due to the order of
the columns in the index, the postgres planner would need to do a full
scan of the index, which can be less efficient than a sequential scan,
so instead it will perform a sequential scan of the table.

This recreates the primary keys while swapping the order of the columns
in the primary key definition, and thus the order in the index. By doing
so, the planner will not need to perform a full index scan, and will be
more likely to use the index when executing the grants query.

(cherry picked from commit 5577b3c358e9d45cd7ec02fc951e45d52e50595e)
2024-09-24 14:18:54 +00:00
Timothy Messier c9578b441a perf(db): Add indexes on foreign key to improve delete performance
This adds indexes for a few categories:

1. Foreign keys on the session table. These are set to `null` when the
   referenced row is deleted. These indexes will help to more
   efficiently set these `null` values in the case where a target is
   deleted, or and auth token is deleted.
2. Foreign keys from other tables to the session table. These are either
   set to `null` or cascade deleted when a session is deleted. These
   indexes help with these update/deletes when a session is deleted.
3. A multi-column index on session_state. This helps with the query that
   is used to delete all terminated sessions that have been terminated
   for over an hour.

(cherry picked from commit 263a96e9ddd197cb36f2c4c74e91ac034112321e)
2024-09-24 14:18:54 +00:00
Jeff Mitchell 62d4f8453b Don't explode grants from the DB (#5104)
This removes the Cartesian product in the DB for GrantsForUser in favor
of returning the actual grant scope information and dealing with it in
application code.

(cherry picked from commit 913cd536823d73994acc4f4d833208c1cc05f91b)
2024-09-24 14:18:53 +00:00
Irena Rindos 2945692bf4 Client cache API paging (#5107)
* internal/clientcache: add -force-reset-schema flag

* clientcache: stream list pages directly to DB

---------

Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
(cherry picked from commit ccea8a4ce53f7b0d1105c1c1e4875e443635d96a)
2024-09-24 14:18:53 +00:00
Jeff Mitchell 3a4ee032d9 Paginate targets in cache (#5101)
* Update Go API to support pagination
* Implement pagination of targets in cache

(cherry picked from commit cd25fad42f325fc050b7ea3bec3336780d2b5e16)
2024-09-24 14:18:53 +00:00
Jim 2a87dcad48 fix (cache/db): add indexes for desktop queries (#5091)
(cherry picked from commit bfdbae1f042c4eb1b05e0373e1aaf2db55c9bb3b)
2024-09-24 14:18:53 +00:00
Jim 85afd56266 fix (cache): support for DB debug logging (#5087)
(cherry picked from commit a7fe48b8d827d1bf68ff8eea205428dd5faae381)
2024-09-24 14:18:53 +00:00
Jim 863359aa26 test (cache/search): fix NewTestServer(...) storage (#5081)
Fix ensures that the TestServer returned uses the
correct directory for writing its sqlite files,
which was created via t.TempDir()

(cherry picked from commit 85f91043a164bdacc1c6738ae0e7c50077d55c79)
2024-09-24 14:18:53 +00:00
Jeff Mitchell 829ea02e6f Ensure only one refresh is running for a user/resource at a time (#5054)
This ensures we don't have competing update processes running. When doing the
inline RefreshForSearch, a specific error will be returned so we can hint in
the API that we're in the middle of a refresh.

(cherry picked from commit 54776091e32af2f2f110f2bf10d0617bd5677aba)
2024-09-24 14:18:53 +00:00
Jeff Mitchell b04de5ac4b Add implicit-scopes resource type to cache (#5053)
This allows getting a list of scope IDs known by the cache via cached
targets and sessions. It is not refreshed from the controller. The name
contains `implicit` in case we ever want e.g. `scopes`.

(cherry picked from commit c0a318c58367d8ec55365447f41cf007eab64ce3)
2024-09-24 14:18:53 +00:00
Jim 4d8a7dd39a feat (clientcache): use a persistent cache (#5051)
By default, we will use a persistent cache for the
client cache. This will allow us to keep the cache
across restarts of the server and it will also
reduce the amount of time it takes to start the
server while reducing the amount of memory used.

This change also includes a validation of the cache
schema version. If the schema version is different
from the one expected, the cache will be
reset/recreated.

(cherry picked from commit cf8a51f3834033b7f986a9a7e091ab31c4328415)
2024-09-24 14:18:53 +00:00
Jeff Mitchell 384fc332be Restrict cache results to 250 entries by default (#5049)
A query parameter `max_result_set_size` can be specified to set a
per-request limit, with `-1` meaning all values. If there were more
entries than the max size used for the request, an `incomplete` boolean
is set in the response.

(cherry picked from commit 3cdd07889be5e4e6f21d044cb60fad1face4be08)
2024-09-24 14:18:53 +00:00
Michael Li 30145b1f9c chore: Update use of deprecated hub command (#5127) 2024-09-23 14:37:22 -04:00
Johan Brandhorst-Satzkorn 0ef88d0f05 Add semgrep rules (#5042)
* Add semgrep rules

Instrument our security scanner with the ability to use custom semgrep
rules. This is inspired by the HVS teams use of semgrep.

* semgrep: add rule to detect string interpolation in WithInfoMsg

The event.WithInfoMsg function expects a set of key value tuples,
but it can easily be misunderstood to accept string interpolation
arguments. This check will error if any string interpolation verbs are
used in calls to event.WithInfoMsg
2024-09-23 10:07:57 -07:00
Irena Rindos 35679c6a5b chore(config): remove deprecated controllers field (#5125)
* chore(config): remove deprecated controllers field
2024-09-23 11:45:34 -04:00
Damian Debkowski 79bf12bd23 fix(cmd): resolve make gen issue (#5124) 2024-09-20 13:29:26 -07:00
Michael Li 40efebf7ab test(e2e): Add check for target authorize-session using scope name and target name (#5118) 2024-09-20 15:28:05 -04:00
Damian Debkowski 29efd5e6c6 test: ensure ipv6 compatability (#5110) 2024-09-20 11:30:27 -07:00
Irena Rindos eb33649ded (clientagentcmd): plumb thru env var (#5123)
* (clientagentcmd): plumb thru env var
2024-09-20 12:33:48 -04:00
Timothy Messier fe129b4268 fix(metrics): Fix matching logic for authorize session endpoint (#5121)
The target authorize session endpoint allows for alternative target
identifiers besides the public id. For this to function correctly, the
grpc gateway path definition for the endpoint deviates from the common
pattern used for other endpoints that include ids. When this change was
done, it was not accounted for in the logic used to capture the
prometheus metrics for api requests to this particular endpoint,
resulting in the metrics never incrementing for the authorize session
endpoint.

This fix updates the matching logic used for the metrics to ensure that
the metrics are incremented for this endpoint.

See: c149bc4a79
2024-09-19 14:02:41 -04:00
dani 1b24cab932 update bsr tests to use ::1 (#5120) 2024-09-19 12:30:10 -04:00
Damian Debkowski 9576ed40dc test: improve test to ensure ipv6 compatability (#5113) 2024-09-19 08:55:27 -07:00
David Bond 61c1544446 Fix panic in credential renewal job (#5116)
This commit modifies the credential renewal job to handle a case where the `renewLease` method can
return `nil, nil`. Currently, a panic is caused by usage of the `renewedCred` variable in this scenario.

This commit adds a nil check and returns an error if the `renewedCred` variable is returned as `nil`.

Signed-off-by: David Bond <davidsbond93@gmail.com>
2024-09-18 17:46:12 +01:00
Jeff Mitchell b120a03f43 Fix tests (#5114) 2024-09-18 11:09:15 -04:00
Ryan Derr 913e6d755f ICU-11661 Add scope name in options slice for authorized-sessions (#5108) 2024-09-13 15:09:38 -05:00
Damian Debkowski 390fca3306 test(server): validate ipv6 address for upsert worker status (#5079) 2024-09-11 13:02:19 -07:00
Damian Debkowski 26a07c1418 fix(sbc): ensure secrets cannot be empty & add unit tests (#5102) 2024-09-11 13:01:45 -07:00
Johan Brandhorst-Satzkorn 41de35f2b1 internal/daemon/controller+worker: propagate downstream timeout (#5083)
Propagate the downstream worker timeout into the
downstream receiver and downstream ticker
so we have a configurable consistent timeout
value to use for worker to worker interactions.

Also change the atomic.Pointer to an atomic.Int64
for consistency with other values.
2024-09-11 12:19:11 -04:00
Irena Rindos 83b61275d5 (cmd): unhide client agent commands (#5097) 2024-09-11 11:20:18 -04:00
15d440a90d docs: Clarify credential workflow for SSH target types (#5082)
* docs: Clarify credential workflow for SSH target types

* adds clarifier about brokered creds on SSH targets

* Update website/content/docs/configuration/credential-management/configure-credential-injection.mdx

Co-authored-by: Robin Beck <stellarsquall@users.noreply.github.com>

---------

Co-authored-by: stellarsquall <stellarsquall@protonmail.ch>
Co-authored-by: Robin Beck <stellarsquall@users.noreply.github.com>
2024-09-11 11:10:59 -04:00
Irena Rindos 0bf038b001 update changelog to docoument gRPC update (#5094) 2024-09-09 14:59:11 -04:00
Dan Heath 81690c30ff docs: Add link to Postgres HA (#5080)
* docs: Add link to Postgres HA

* docs: Add new section to HA topic

* docs: Trigger a new build
2024-09-09 14:25:02 -04:00
Michael Gaffney 68c83eb419 chore: Bump to Go version 1.23.1
Go 1.23.1 includes security fixes to the encoding/gob,
go/build/constraint, and go/parser packages, as well as bug fixes to the
compiler, the go command, the runtime, and the database/sql, go/types,
os, runtime/trace, and unique packages.

See: https://groups.google.com/g/golang-announce/c/K-cEzDeCtpc
2024-09-09 13:41:49 -04:00
Mark Collao 0b4e17162d update scan workflow conditions (#5089) 2024-09-09 09:27:22 -07:00
Johan Brandhorst-Satzkorn a830892fec Some test fixes (#5084)
* internal/session: update variable name in test

This was missed in #5071

* internal/tests/cluster: skip failing tests

These tests will need to be fixed or removed. It is not
clear why they are failing, but they are always failing.

* Makefile: try upgrading golangci-lint

* internal/host/plugin: fix flaky test

Hosts contain IP addresses in string form, whose
order is not defined.
2024-09-09 09:22:34 -07:00
dependabot[bot] d45aa402d8 chore(deps): bump github.com/opencontainers/runc (#5075)
Bumps [github.com/opencontainers/runc](https://github.com/opencontainers/runc) from 1.2.0-rc.1 to 1.2.0-rc.3.
- [Release notes](https://github.com/opencontainers/runc/releases)
- [Changelog](https://github.com/opencontainers/runc/blob/main/CHANGELOG.md)
- [Commits](https://github.com/opencontainers/runc/compare/v1.2.0-rc.1...v1.2.0-rc.3)

---
updated-dependencies:
- dependency-name: github.com/opencontainers/runc
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-06 15:38:08 -07:00
Michael Li 08433affc8 doc(e2e): Update enos readme (#5033) 2024-09-06 14:17:40 -04:00
Damian Debkowski 89c9a309ce test(host): validate ipv6 address for hosts (#5052) 2024-09-05 10:44:55 -07:00
Damian Debkowski bd03c99f89 test(target): validate ipv6 address for tcp targets (#5076) 2024-09-05 10:30:30 -07:00
Dan Heath a77d04edbb docs: Update session recording docs with asciicast info (#5072)
* docs: Update session recording docs with asciicast info

* docs: Remove mention of warnings
2024-09-05 12:09:04 -04:00
Michael Li 30b999e261 test(hcp): Define TF module for long-lived resources (#5045)
* test(hcp): Define TF module for long-lived resources

* CR: Move outputs and variables to its own files
2024-09-05 12:08:07 -04:00
dani 557591967a add tests for sql ip address columns to ensure ipv6 capabiltiy (#5062) 2024-09-05 11:03:05 -04:00
Jeff Mitchell 66af7dba98 Allow reloading some server timings (#5071)
Allows changing grace period and a few other things timing-related via
HUP instead of just a restart.

Tested manually via config changes and HUP.
2024-09-04 15:40:44 -04:00
Jeff Mitchell 35d1df7388 Two fixes in our Go API code: (#5066)
* Replace `interface{}` with `any`
* For options that are bools, don't always set true; set to the actual
  value passed in (some options already behaved this way, but they all
  should)
2024-09-04 11:40:43 -04:00
hashicorp-tsccr[bot]andhashicorp-tsccr[bot] 4e8e069c9c Result of tsccr-helper -log-level=info gha update -latest . (#5063)
Co-authored-by: hashicorp-tsccr[bot] <hashicorp-tsccr[bot]@users.noreply.github.com>
2024-09-03 15:19:21 -04:00
David Bond 660e2063a4 Return HTTP 400 status for inactive OIDC auth methods (#5048)
This commit modifies the OIDC authentication start method to return a 400 (Bad Request) error
if a client attempts to start the authentication flow against an OIDC auth method that is set to
inactive. This is something we noticed in our error logs in HCP Boundary that is impacting our
SLOs.

Previously this returned its own error code which is mapped to a 500 response. Tests have been updated
to catch this new scenario.

Signed-off-by: David Bond <davidsbond93@gmail.com>
2024-08-30 16:33:18 +01:00
Johan Brandhorst-Satzkorn 59fd2d4595 e2e: replace '%s' with %q (#5058)
%q handles quotes inside the quoted strings better.
2024-08-29 15:50:19 -07:00