This adds the ability to define hooks that will run prior to a schema
migration. The hooks should only be used when a migration would fail due
to data being in an invalid state, likely due to a bug in the previous
schema definition. For example if a column was missing a constraint, and
a new migration is attempting to add the constraint, there might already
be values that violate the new constraint. In such cases, we would want
the user to make a decision to address the data, and need to provide
them with details about what data is problematic in case they want to
manually address the problem. To help the user, if there is a sane way
to address the data, we should provide an option to automatically
"repair" the data, allowing the migration to continue. However, this
still should be an explicit opt-in from the user, and we need to clearly
describe what the repair option would do to the user so they understand
the consequences of opting in.
To accomplish this each migration can be provided a hook that contains a
CheckFunc, RepairFunc, and RepairDescription. These functions must
operate at the point-in-time of the migration immediately preceding the
current migration, as as such they should not use and domain packages or
repositories, instead they should use direct sql with the sql library.
The description is used to explain what the RepairFunc will do.
Hooks should be defined in `internal/db/schema/migrations/oss/oss.go`
and registered with the schema edition.
See the example in `internal/db/schema/manager_example_test.go` for more
details.
This adds two new config options for the `database` stanza:
- `max_idle_connections`
- `max_idle_time`
These correspond to settings for the underlying `sql.DB` instance. They
can be used to have finer control over the controller's database
connection pool. For example this can be used to mitigate the high
connection overhead when connecting to postgres by setting a high
`max_idle_connections` and either setting `max_idle_time` to zero or to
a high duration.
Alternatively, if using a connection pool like pgpool-II or pgbouncer,
`max_idle_connections` can be set to 0.
See:
https://pkg.go.dev/database/sql#DB.SetMaxIdleConnshttps://pkg.go.dev/database/sql#DB.SetConnMaxIdleTimehttps://www.pgpool.net/mediawiki/index.php/Main_Pagehttps://www.pgbouncer.org/
* feature (db): add Db.UnderlyingDB()
This is needed to allow for integration with bits that need a *dbw.DB
* feature (kms): add new go-kms-wrapping/extras/kms dependency
* refactor (kms): schema changes required to adopt extras/kms pkg
* refactor (kms/iam): rename to DeprecateCreateKeysTx(...)
* refactor (kms): remove deprecated test
* refactor (kms): refactor Kms type
This refactor includes renaming the existing type to DeprecatedKms
and then refactoring the Kms type to become a wrapper of
go-kms-wrapping/extras/kms
* refactor (kms): delete all the unneeded bits
All this stuff is not needed now that we're using
go-kms-wrapping/extras/kms
* refactor (db): fixup tests after kms refactor
* refactor (kms/tests): use testing.TB for test helpers
Our test helper packages were not possible to use
with benchmarks. This refactor switches to testing.TB
to allow both tests and benchmarks to use the helpers.
* tests (session benchmarks): refactor to use new Kms type
* refactor (schema): rm test which relies on deprecated Kms code
* refactor (kms): post rebase on main changes required
* chore (kms refactor): re-add unit test from main
* refactor (kms): uses migrations that don't include KMS schema changes
Timothy Messier
Jim