* Only call the Plugin at create/update time after the db has been partially updated. This allows more constraints to be checked prior to calling the plugin.
* Create catalog service handler for updating plugin. Add tests for handler and run make fmt. Silently drop attributes where the value is nil when creating the catalog or set.
This should help prevent issues like the one fixed in
f2ad4de554.
A version for a particular edition should only exist once, previously
which ever file was read last for a version would be included, which
would lead to some migrations not running.
* internal/host/plugin: Add UpdateCatalog
This adds support for updating plugin-based host catalogs.
The workflow is:
* The existing catalog is fetched.
* Any attribute changes are patched against it, along with other changes
such as name and description.
* Both the existing catalog, and the state of the catalog as it will
exist post-update, along with any persisted secrets, are sent to the
plugin via an OnUpdateCatalog request.
Upon success, the following is done:
* The record is updated if non-secret updates need to be made.
* Secrets are updated if they still exist.
* If an empty secret set was sent back, secrets are instead deleted.
* If the secrets field was instead null, secret update is skipped
altogether.
There has also been a lot of helper code added to the tests to help make
the update tests more readable and reduce boilerplate. While the create
tests have not been updated to use these new helpers, it might be a good
idea to do so for consistency.
* move secret/persisted state closer to plugin call
* Update internal/host/plugin/repository_host_catalog.go
Co-authored-by: Todd <T.Alan.Knight@gmail.com>
* Update internal/host/plugin/repository_host_catalog_test.go
Co-authored-by: Todd <T.Alan.Knight@gmail.com>
* move all test helpers for UpdateCatalog inline
* WIP - transparent attributes struct/wire conversion
* Revert "WIP - transparent attributes struct/wire conversion"
This reverts commit 31af1b1b7e535119aa23bd0aa936da1a80eb5aa2.
* deal with multiple attribute masks
* Note TODO for upsertQuery for HostCatalogSecret
* add version check guard
* Update internal/host/plugin/host_catalog_secret.go
Co-authored-by: Todd <T.Alan.Knight@gmail.com>
* remove redundant JIRA reference
* don't separate number of updated catalogs and secrets
* increment version when secrets is the only thing being updated
* add another test for updating both catalog and secrets at once
* correct comment for mock plugin manager section
* add first-part path matching for secrets
* update plugin lookup error to return errors.Internal
* only add name/description to plugin request if they are non-empty
* update duplicate tests to use two project scopes
* remove unused test checks
* return plugin record along with host catalog in update
* move away from using passed in catalog after patching newCatalog
* use t.Cleanup() instead of defer
Co-authored-by: Todd <T.Alan.Knight@gmail.com>
* Update the mask manager to always allow fields containing a prefix which matchs a passedThroughPrefix.
This will be useful when updating plugin based host catalog and sets since attributes and secrets cannot be contained in the mask manager mapping.
* feature (db): Add the ability to enable debug output for a connection
* refactor: Use scope.Global instead of "global"
* feature (db/migrations): Add kms audit keys
Add schema changes to support a new DEK for audit crypto operations.
* feature (kms/audit): Add audit key type
All the bits required to define a new DEK concrete type for
operating on audit entries.
* refactor (db): Remove deprecated migration unit tests.
We added a new audit DEK to kms.CreateKeysTx(...). Unfortunately,
unit tests like TestMigrations_UserDimension,
Test_PrimaryAuthMethodChanges and TestMigrations_CredentialDimension
have an indirect dependency on a specific historical set of DEKs via
an indirect call kms.CreateKeysTx. This of course fails because the
new DEK's tables is not a part of pinned migration versions that these
tests depend on.
* Add ability to read and list hosts. Tests to follow.
* Add tests for reading and listing as well as validation checks to not modify hosts.
* Add IpAddress and DnsNames to the api resource. Populate the plugin info.
* Adding go sdk tests for reading and listing hosts and ensuring update and deleting is disallowed.
This test started failing when the order of the list response changed.
Since the order does not matter, this test was updated to not depend on
the order.
Target subtypes must now implement another repository hook to validate
any CredentialLibraries that are being added/set. This allows for
subtypes to determine the valid set of CredentialLibraries.
For tcp.Targets only CredentialLibaries with
credential.ApplicationPurpose are allowed. If any other purpose is
provided an error is returned.
This change removes the hard-coded credential purpose for the Credential
Libraries. It also moves the logic of creating CredentialLibraries from
CredentialSources out of the repository and into the targets.Service.
This refactor will allow for additional credential purposes to be
supported in both the service and the repository.
This stores worker connection nonces within the database to ensure
uniqueness of connections. Prior the connection required any attacker to
be able to decrypt the incoming encrypted authentication message in
order to replay (and if the attacker has access to the KMS you have
bigger problems), but with this change even if an attacker does have
access to the KMS key, they cannot reuse the same message. (If they're
able to encrypt against the KMS key, they can still act as a worker with
a new message, so this only really helps if they only have decryption
capabilities.)
* refactor (oidc): Stop emitting error for not found token id
Stop emitting errors when authtoken repo.IssueAuthToken is called and
there's if no pending token. It's not an "normal" state and not
an error condition.
* feature (audit tagging): Add tags for auth method requests/responses
Including testing functions for asserting audit events created
when a service is called
For the target Repository's Create method, the options of
WithHostSources and WithCredentialSources where never used by the
targets.Service. As such the repository does not need to support these
options.
See: 51812ff779
* internal/libs/patchstruct: add package for patching structpb
This adds a small helper package for patching *structpb.Struct values
using the same patterns that our APIs use for updating records. The
intent of this package is to provide a fallback mechanism for working
with these values when said record update behavior is unavailable, such
as when working with plugin-based host subtypes where the exact
structure of the subtype's attributes is unavailable.
patchstruct.Patch follows the following logic:
* The source (src) map is merged into the destination map (dst).
* Values are overwritten by the source map if they exist in both.
* Values are deleted from the destination if they are set to null in the
source.
* Maps are recursively applied, meaning that a nested map at key "foo" in
the destination would be patched with a map at key "foo" in the
source.
* A map in the destination is overwritten by a non-map in the source,
and a non-map in the destination is overwritten by a map in the
source.
Patch returns the updated map as a copy, dst and src are not altered.
* Add doc comment for patchstruct.Patch
* vim snafu
* Premature commit of repository_host_catalog.go
* add nil cases
* internal/libs/patchstruct: add PatchJSON
* small comment formatting fix
* Change PatchJSON to PatchBytes
PatchJSON wasn't what was needed here - we need a helper to patch the
protobuf encoding of structpb.Struct.
* remove equality for original src/dst for patchbytes
* Add DB changes for host, set members, and host addresses for plugins.
* Add priority, address review feedback, update tests, naming, protos
Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>
Todd
Jeff Mitchell
Jim
Chris Marchesi
Timothy Messier
Louis Ruch
Michael Gaffney
Oleg Butuzov