Raising a warning in the default trigger used for setting the value in
create_time columns creates unnecessary noise in the PostgreSQL log
files.
Closes#1744
This adds interfaces for the following credential types:
- Username and password
- Username and private key
- Certificate and private key
It also adds the following types for redacting secret data:
- Password
- PrivateKey
Four new columns were added to the `wh_user_dimension` table to support
OIDC. The data type for the columns was originally `wh_dim_text` which
is a domain type with a `not null` restriction. Adding the new columns
caused the migration to fail if the `wh_user_dimension` table had any
rows in it because the existing rows would violate the `not null`
restriction.
Closes#1469
* Fix down migrations
* Udpates due to merge
* Sentinel values only have to be greater than 0
* Create credential related sdk structs and methods.
* Post merge interface updating.
* Removing scratch code that was not cleaned up.
* updates
* Convert client certificate and client certificate keys into pem blocks and validate.
* Add newline at end of file.
* Faster check for "s" at the end of a resource name when generating from templates.
* make fmt
* updates
* Add dynamic credentials to a session
* Refactor: move contents of file
* updates
* Updates
* Add InvalidDynamicCredential error
* Assign credentials
* Allow updating client certificates seperate from the client certificate key.
* Replace application purpose string with enum value
* PR feedback / fix go test .
* Delete dead code
* Use common view
* Refactor: extract interface and rename method
* Refactor: move code to eliminate privPurpLibrary
* Add comment to requestMap
* Refactor: do not export methods on an unexported struct
* Update internal/db/schema/migrations/postgres/10/03_vault_credential.up.sql
Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>
* Add comments to credential Purpose constants
* Reorder switch statements to handle errors first
* PR feedback
* update
* fix merge
* Organizing the scheduler code and extract the individual store creation test helper function.
* Run 'make fmt'
* Create certificate when creating store. Allow vault address to be passed in as option
* Refactor: rename database views and associated structs
This change renames two database views and associated go structs to use
a consistent naming pattern. Views ending in 'private' contain encrypted
credentials needed for connecting to Vault. Views ending in 'public'
mirror the 'private' views but do not include any encrypted values.
This change does not include any changes to functionality or behavior.
* Adding SAD target operations for credential libraries to the SDK.
* Add status field and enumeration table for credential status
* Refactor: rename Status to TokenStatus
* Add status to vault credentials
* Creating the template for the CLI and helper functions.
* Add private database view for vault credentials
* Add scope_id to view and cleanup TODO comment
* Reformat embedded sql queries
* Add vault client helpers
* updates
* white space
* Add LookupLease test
* CLI commands for Store work. Renamed fields in proto w/ vault prefix.
* Test issue credentials with client TLS
* Add additional checks to vault ping
* Add an error for vault token capabilities and create a VaultToken error kind
* Add support for working with Vault policies
* Use Vault 1.7.2 for testing
* Add testing helper for creating a vault client with a non-root token
* Fix Vault renew lease test
The Vault renew lease endpoint accepts an optional duration parameter
for requesting the number of seconds to extend the lease by. However,
Vault is not required to honor this request.
* Add a var for the required capabilities of a boundary Vault token
* Enhance the test helper for adding a policy to Vault
* Allow the test role in postgresql to revoke credentials
* Remove out of date comments
* Enhance Vault test helpers for the database secrets engine
* Reformat comments
* Add a method for getting the capabilities of the current Vault token
* Add revoke lease to internal Vault client
* Reformat: combine err return line with check for nil line
* Remove redundant checks in test code
Remove checks in test code that are redundant with checks in the test
harness.
* Refactor CreateToken test helper to return the secret and the token
* Fix build failures caused by signature change in vault.CreateToken
* Fix Vault version and Vault API link in comments
* ran `make gen` after merging from mgaffney-vault.
* Remove Vault prefix from attribute fields in API, create a base CLI option for adding field prefixes. format now uses that prefix for error reporting.
* Template now allows attaching attribute field prefixes.
* Credential store uses PrefixAttributeFieldErrorsWithSubactionPrefix.
* Fixing naming of vault flags and helper flags.
* Updating more references to generated api methods.
* Make the capabilities String method more readable
* Add comment outlining PrintApiError usage of Options.
* Refactor test to move declarations closer to first use
* Fix comment
* Implement revoke method on vault repository
* Tests now verify certs, CA, and tokens, addresses can be updated in a store.
* Added test for creating store with pk/cert in same field.
* Allo updating certificates seperate from private key. Only contact backing vault service when updating token.
* Fixing bad tests and removing unused code.
* Adding tests for various ways of updating tokens.
* Add private credential
* Add token revocation job and cred renewal/revocation job
* Regen after merge.
* Fixing field mask related merge error.
* Create Library CLI. Removes vault prefix from path. Adds a WithoutCleanup testOption for vault.
* Set all credentials to revoked when a token is revoked
* Fixing missed renaming update.
* fix copy/paste error.
* Fixing code generation errors after merge.
* make gen after merge.
* Update internal/cmd/commands/credentialstorescmd/vault_funcs.go
Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>
* Update internal/cmd/commands/credentialstorescmd/vault_funcs.go
Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>
* Update internal/cmd/commands/credentialstorescmd/vault_funcs.go
Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>
* Refactor: move migrations from 10 to 12
* Removing logic from transaction.
* wrapping client errors in the calling op for CredentialStore's client() method.
* Redact token
* redact client key
* updates
* Add trigger to revoke credentials when session is canceled or terminated
* Fixing some help text for some flags.
* Add ability to load values from files (file://) or environment variables (env://).
* Updating comments
* fix bad merge
* feedback
* Adding issued credentials to sessions.
* Error on capabilities
* updates
* Expose NewVaulTestServer
* Add godoc
* Adding generated SessionCredential struct for API.
* Progress on building handler test for Authorize Session.
* feedback
* Update policies
* Add 'revoke' as a status for Vault Tokens
* Add delete_time to vault credential store
* Create dynamic credentials at session creation time.
* Tests pass with credentials attached to AuthorizedSession result.
* Only call issue if there are credentials to issue.
* Fixing output only issues. Adding type to credential library.
* Fix test to clear un/pw generated. confirm generated un/pw changes from 1 authorize session call to another.
* Update lookupTokenError
* Add delete_time to private credential store
* Refactor: move private store to separate file
* Fixing tests for AuthorizeSession and verifying error cases.
* Only create the credential repo if there are credentials to issue.
* Rename Library field to CredentialLibrary.
* Fixing missing change of Library to CredentialLibrary.
* Removing Default SDK functions for fields which are required and update CLI to match.
* Replace resource specific id variables with "id" and the input parmater for paths to be just a single string and the resource name as it should show up in the path.
* Soft delete vault credential stores
* Prevent new SQL functions from silently overwriting existing functions
* Rename "Hmac" to "HMAC" in the human readable cli response table.
* Fix tests
* Add vault database cleanup jobs
* Add ability to set the token period in the vault test harness
* updates
* Add not_null_columns func test
* remove unused struct
* Up no output timeout
* Validate current vault token on addr update
* Fix client cert change test
* add cert test
* Update revocation job where
* Fix comment
* Use Vault's default port in CLI help strings
* Remove down migrations
* Increase migration file numbers by 1
* Run make migrations
* Move migrations from 12 to 10
* Increase Circle CI timeout for acceptance tests
* Fix spelling mistake in error messages
* ASD cmds for Target and returning credentials to AuthSession request (#1338)
* Add Add/Set/Remove commands for libraries on Targets. Return credentials on AuthorizeSession request.
Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>
Co-authored-by: Louis Ruch <louisruch@gmail.com>
Co-authored-by: Todd Knight <T.Alan.Knight@gmail.com>
Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
The definition of 'subtype' is 'a secondary or subordinate type'. The
'T' should not be capitalized (unless it is referring to a type of
submarine, which its not).
* Define initial warehouse schema
* Add session id as the primary key to the session accumulating fact table
* Insert a row to use instead of nulls in the time and date dimensions
* Add missing attributes and a partial index
* Rename files
* Refactor: move fact table to a separate migration
* Add domain types for the warehouse
* Add session connection transaction fact table
* Change columns to match ingress data types
* Add views for host dimension ingress
* Create source and target views for the host and user dimensions
* Generate random ID for dimension table primary key
* Formatting changes
* Add current dimension id to target views
* Implement upsert function for host dimension
* Use domain types for upsert function parameters
* Fix indentation
* Add wh_public_id domain type
* Fix value of type columns when the other columns are null
* Implement upsert function for user dimension
* Add warehouse domain types for timestamps and text fields
* Move fact table migrations to a higher number
* Add functions for getting the current date and current time IDs
* Rename backend to endpoint in column names
* Rename fact tables
The fact tables are actually accumulating fact tables not transactional
fact tables.
* Add default value to connection_authorized_time column
* Update session fact table
* Add session_max_seconds, session_connection_limit, and default_port to wh_host_dimension
* Move session table above connection table
* Add after insert trigger to session table for warehouse ingress
* Add fk from connection warehouse table to session warehouse table
* Implement insert trigger for session connections
* Update warehouse for a user having multiple auth accounts
* Rename warehouse columns to be consistent with operational tables
* Add connection specific columns to the connection fact table
* Update session calculations in warehouse
* Rename functions and formatting changes
* Rename session columns to match names of session states
* Implement warehouse triggers for session and connection state changes
* Run 'make migrations'
* Use a static start and end date for the warehouse date dimension
* Document warehouse functions
* Remove outdated TODO comments
* Add TODO comments for additional dimensions and fact table columns
* Define session database schema
* Update comments
* Add insert trigger for session state
* Add canceling state and update foreign keys to 'set null' on delete
* Update host set member test
* Add ListSetMembers method signature and doc
* Move repository methods for host set members to new file
* Add TestSetMembers
* Implement ListSetMembers
* Return hosts from ListSetMembers
* Implement AddSetMembers
* Implement DeleteSetMembers
* Use sql.QueryContext not sql.Query
* Implement SetSetMembers
* Refactor: extract getHosts method
* Refactor: extract createMembers method
* Refactor: extract common functions
* Return correct count of changed
In SetSetMembers, retrieving the current list of hosts in the set
happens after the transaction to modify the set has completed. This
retrieval can result in an error but the number of modified rows is not
effected.
* Make list set members an unexported method
* Get hosts within the same transaction as mutations of the set
* Return hosts of set from LookupSet method
* Return hosts of set from UpdateSet method
* Remove listSetMembers
* Fix merge conflict
* Add NOTE and TODOs about using new pattern in "SetMembers" methods
* Implement CreateSet
* Implement UpdateSet
* Implement LookupSet
* Implement ListSets
* Implement DeleteSet
* Remove unused parameter from test helper function
* Update to use new kms structure
* Remove unused parameter for test helper function
* Fix resource-type string for host set
* Move static host migrations to higher number
* Add wt_host_address domain type
* Add base tables for host catalog, host set, and host
* Reformat create trigger statements
* Add subtype triggers for host catalog, host set, and host
* Remove wt_host_address domain type
* Rename columns in host_set_member
* Add delete trigger functions for base types
* Add delete after triggers to static host subtypes
* Add protobufs for host base types
* Bump generated proto code
* Add argon2 conf sql and proto files
* Add generated code for argon2 conf
* Add Resources for Argon2 configuration
* Add get and set Configuration to repository
* Add current config reference to AuthMethod
* Fix authmethod_test
* Fix TestAccount_New
* Add WithConfiguration to options
* Add sql views for current configuration
* Fix SetTableName
* Remove duplicate testAccounts function
* Make tests clearer
* Fix comment
* Rename test variable
* Use NewPrivateId function for private Ids
* Refactor: rename public_id to private_id for password_conf and password_credential
* Remove bad comment
* Check if embedded argon2 configuration is nil
* Add checks around the WithConfiguration option
* Use spaces not tabs in proto file
* Add doc to explain choice of oplog ticket
* Verify oplog in test
* Add base SQL and protobuf files
* Add options and public Ids
* Add AuthMethod and Account resources
* Add repository
* Add CreateAuthMethod to repository
* Add CreateAccount method to repository
* Remove redundant foreign keys
* Document relationships between iam and auth tables
* Document relationships between auth and auth_password tables
* Add documentation for subtype triggers
* Use wt_scope_id domain type for scope_id
* Refactor: move test helper function
* Remove check from SetTableName
* Remove check from another SetTableName
* Use org scope in tests
* Remove handling of CreateTime and UpdateTime from docs
The following error string was the end result of an error in lookupAfterWrite:
`set password configuration: update: lookup error lookup after write: failed record not found`
The error string with this change applied would be:
`set password configuration: update: lookup after write: record not found`
* database: add wt_public_id domain type
* static-hosts: add migrations
* Rename migrations
* migrations: add description column and fix foreign keys
* Add protobuf for static host
* Add HostCatalog and HostSet
* Add Options
* Add foreign key constraint
* Add package doc
* Add copied scanner code
* Add NewHostCatalog with tests
* Override default table name for static host catalog
* WIP: save static host catalog to database
* WIP: Print db url for debugging
* Add iam test helper function
* Make gorm default to null for names and descriptions
* Add foreign key constraint from host catalog to scope
* Names are not needed for the test org and project
* Avoid an ordering dependency in down migrations
* Use wt_timestamp
* Add check constraint on host address column
* Add Host
* Add HostSet
* Add HostSetMember
* Test a HostSetMember can be written to the database
* Move the getters and setters for table name to a new file
The TableName methods are needed by gorm and the SetTableName methods
are for oplog. Moving them to a new file so related functionality is in
the same file.
* Add repository
* Add NewRepository
* Close connection before cleanup of database
* Add comments to protobuf file and regenerate
* Define repository methods for HostCatalog
* Assign PublicId when resource is inserted into the database
* Wrap a nil parameter error
* Add oplog tickets for static host types
* WIP: first cut at CreateCatalog
* CreateHostCatalog: more tests
* Detect unique constraint violation
* Use const for PublicId prefixes
* Use PostgreSQL error name instead of error code
* Add TODO comments
* Fix comments
* Use PublicId not PublicID
* Refactor: move error variables from host/static to db
The error variables defined in host/static are generic and can be reused
by other packages.
* Create a new repository for each test
* Validate CreateCatalog returns a new HostCatalog
* Do not modify the HostCatalog passed in
* Implement UpdateCatalog
* Minor cleanup to make folds work
* Implement LookupCatalog
* Refactor: extract allocCatalog function
* Add comment
* Refactor: extract newCatalogMetadata function
* Make error strings more consistent
* Move database specific error check to internal/db package
* Change signature of DeleteCatalog and update docs
* Implement DeleteCatalog
* Make error strings more consistent
* Add examples
* Update docs
* Add package level doc and example for repository
* No need for using reflect.DeepEqual
* Remove redundant comment
* Clarify ErrMultipleRecords results in the transaction being rolled back
* Remove redundant error checks
* Remove unnecessary check
* Return early instead of using nested 'else' statements
* Update and Delete must return a row count
* catch err on defers (#82)
Co-authored-by: Jim Lambert <jimlambrt@Jims-MBP-3.home>
* Remove 'if assert' pattern
* Use assert package in test helper function
* Add test for updating a non existent catalog
* Use a single assert when a specific error is expected
* Use assert package when it clarifies the check
* Catch errors in cleanup functions
* Update comments in protobuf file
* Update doc for UpdateCatalog to reflect the new behavior
* Add common error values for field mask errors
* Require a field mask when calling update methods
* Add TODO for removing direct call to gorm
* Export Public ID prefixes
* Add db timestamp guards
* Remove unused option
Co-authored-by: Jim <jlambert@hashicorp.com>
Co-authored-by: Jim Lambert <jimlambrt@Jims-MBP-3.home>
* Add wt_timestamp type to database
* Swap the order of domains and oplog migrations
All migrations should use the new 'wt_timestamp' domain type for
timestamps.
* Remove dead code
* Add down migration for wt_timestamp
Since client code uses the package name as a prefix when referring to
the package contents, the names for those contents need not repeat the
package name.
- Rename base.BaseCommand to base.Command
- Rename base.BaseServer to base.Server
- Rename config.DevConfig() to config.Dev()
- Rename config.LoadConfigFile() to config.LoadFile()
- Rename config.NewConfig() to config.New()
- Rename config.ParseConfig() to config.Parse()
- Rename controller.ControllerCommand to controller.Command
- Rename controller.NewController() to controller.New()
- Rename dev.DevCommand to dev.Command
- Rename listener.ListenerFactory to listener.Factory
- Rename listener.NewListener() to listener.New()
- Rename version.GetVersion() to version.Get()
- Rename version.VersionInfo to version.Info
See:
- https://golang.org/doc/effective_go.html#package-names
- https://blog.golang.org/package-names#TOC_3.
- https://github.com/golang/go/wiki/CodeReviewComments#package-names
Michael Gaffney
