This adds the ability to define hooks that will run prior to a schema
migration. The hooks should only be used when a migration would fail due
to data being in an invalid state, likely due to a bug in the previous
schema definition. For example if a column was missing a constraint, and
a new migration is attempting to add the constraint, there might already
be values that violate the new constraint. In such cases, we would want
the user to make a decision to address the data, and need to provide
them with details about what data is problematic in case they want to
manually address the problem. To help the user, if there is a sane way
to address the data, we should provide an option to automatically
"repair" the data, allowing the migration to continue. However, this
still should be an explicit opt-in from the user, and we need to clearly
describe what the repair option would do to the user so they understand
the consequences of opting in.
To accomplish this each migration can be provided a hook that contains a
CheckFunc, RepairFunc, and RepairDescription. These functions must
operate at the point-in-time of the migration immediately preceding the
current migration, as as such they should not use and domain packages or
repositories, instead they should use direct sql with the sql library.
The description is used to explain what the RepairFunc will do.
Hooks should be defined in `internal/db/schema/migrations/oss/oss.go`
and registered with the schema edition.
See the example in `internal/db/schema/manager_example_test.go` for more
details.
* chore(deps): Update grpc-gateway dependency
This fixes an issue which would allow an attacker to trigger a
(recovered) panic in the gateway handler.
* fix(server): Use cmp.Diff instead of assert.Equal
Testify is not great at handling protobuf types, and
was for some reason returning a diff here when in
reality there is none. Switch to go-cmp for the
comparison to remove the error.
* chore: update go-dbw dependencies
This update included an update of the upstream gorm dependency.
In the latest update, gorm appears to have become more sensitive when
a slice is passed into operations. The ptr to slice must now
be a ptr to slice of ptrs. This change required an update to how
boundary handles slices of PrincipalRole, which now must return a ptr
to PrincipalRole. There were also some tests that needed to be
updated because of this change.
* fixup! chore: update go-dbw dependencies
* feat(credentials): Refactor credential purposes
* Application credentials have been refactored to Brokered credentials
* Egress credentials have been refactored to Injected Application
credentials
Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
* Add support for ssh_private_key credential type in Vault libraries
* Add support for ssh_private_key credential type
* Support jsonpointer for Vault library mapping overrides
This adds two new config options for the `database` stanza:
- `max_idle_connections`
- `max_idle_time`
These correspond to settings for the underlying `sql.DB` instance. They
can be used to have finer control over the controller's database
connection pool. For example this can be used to mitigate the high
connection overhead when connecting to postgres by setting a high
`max_idle_connections` and either setting `max_idle_time` to zero or to
a high duration.
Alternatively, if using a connection pool like pgpool-II or pgbouncer,
`max_idle_connections` can be set to 0.
See:
https://pkg.go.dev/database/sql#DB.SetMaxIdleConnshttps://pkg.go.dev/database/sql#DB.SetConnMaxIdleTimehttps://www.pgpool.net/mediawiki/index.php/Main_Pagehttps://www.pgbouncer.org/
* bug(scheduler): Fix controller id length check
The job_run table has a foreign key to the server_controller private_id.
While this column is called a prviate_id it does not implement the wt_private_id
postgres domain type, and therefore does not have a minimum length of 10 characters.
This removes a lot of log clutter from normal test execution with `go
test -v`. There are a number of test that expect no results from the
database or other error cases from the database. The previous default
log level of dbw.Error would cause these cases to generate logs in the
success case, creating noise in the test output.
The new default for tests is dbw.Silent. This can be overriden in two
ways. A test can be updated to pass in the db.WithTestLogLevel option,
which allows for targeting specific tests to include additional output.
This may make sense for tests that normally should not cause errors, but
would report an error if it fails.
Alternatively a new environment variable can be set,
DEFAULT_TEST_LOG_LEVEL. This allows for adjusting the log level without
changing any test file. The environment variable should be set to the
string value of the log level, and is treated as case insensitive, ie:
export DEFAULT_TEST_LOG_LEVEL="error"
* (refactor) change all sql references of user_password to username_password
* refactor(pkg) renamed pkg userpassword to usernamepassword
* refactor(proto) updated protocol buffers to use UsernamePassword
* refactor(code) updated references from UserPassword to UsernamePassword in golang files
* refactor(test) updated unit tests to reference UsernamePassword changes
* refactor(ddtest) updated references in dbtests to use UsernamePassword
* chore(sql-test) fixed spelling issue
* chore(sql) added and updated comments in migartion & tables. fixed indentations
* chore(proto) updated comments to reflect username_password changes
* refactor(tests) updated unit tests to replace user-password with username-password
* refactor(cmd) updated cmd help text to use username
* chore(tests) update test name & comments
* fix(cmd) fixed cmd usage definition for credentials update
* refactor(tests) removed out of scope changes
* fix(sql) incremented sql migration id & updated migration comments
* Send more info on KMS connect (#2182)
* Update nodeenrollment to v0.1.0
* bug(workersAPI): do not allow nonprintable or uppercase name or desc
* Put checks on create too
* Apply printable check to description in DB
* chore (worker): address PR feedback
Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
Co-authored-by: Jim <jlambert@hashicorp.com>
* feature (workers): required schema changes
Changes required for consolidation of worker name, description
and address fields
* fixup! feature (workers): required schema changes
* fixup! feature (workers): required schema changes
* fixup! feature (workers): required schema changes
* Fixing up address constraint.
* last_status_time = now() on insert only if the worker is type = 'kms'
* update last_status_time for pki and kms based on different columns being updated.
* tweaking the constraints on last_status_time and address.
Co-authored-by: Todd <github@quaddi.com>
Including the nested connection information in the session list response
had a negative impact on performance as it created a significant
multiplier on the number of rows that needed to be fetched from the
database.
It was also determined that this information was used by neither the
admin ui nor the desktop client.
Note that nested connection information can still be retrieved via a
read request for a specific session.
There is a new periodic job that will delete terminated sessions that
are older than one hour. However, since until now terminated sessions
were not deleted, the first time the job runs there could be a large
number of sessions that need deleted. This could have a negative impact
on a running cluster. To mitigate the operational risk, this adds a
migration to perform an initial delete while the cluster is offline.
* server refuses to start if max_open_connections is below 5
* suggested changes - Update internal/db/db.go
Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Johan Brandhorst-Satzkorn
Danielle
Jim
Louis Ruch
Damian Debkowski
Timothy Messier
Michael Gaffney
irenarindos
Jeff Malnick
Damian Debkowski
Todd
Gabriel SantosandJohan Brandhorst-Satzkorn