* Add DB changes for host, set members, and host addresses for plugins.
* Add priority, address review feedback, update tests, naming, protos
Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>
Now that subtypes are registered via sub-packages, this TargetType is
not needed and should not be used. This removes it and updates the
target.Repository and targets.Service to use the updated code.
This was missed in eac433f7ff.
* refactor (controller): Use a grpc server for the grpc-gateway
This starts a grpc server for the grpc-gateway. The new gateway
server uses an in-memory listener which also supports "dialing", so
the http proxy can connect to the in-memory listener.
This change allows us to use all grpc server features and was in part
motivated by a need for an interceptor of inbound requests. With that
said, having a "proper" grpc server for the gateway will provide a lot
more flexibility and capabilities moving forward.
The request context will still be verified via middleware in the
http proxy (see wrapHandlerWithCommonFuncs). This verified context
will be serialized into a protobuf and sent along to the grpc-gateway
via metadata where an interceptor will add the verified request
info to the request's context (see requestCtxInterceptor). The
interceptor will verify the serialized request info via a shared ticket
with it's companion in-memory http proxy. FYI: the ticket is simply
random base62 string (see db.NewPrivateId).
This change also necessitated a minor change to services. We need to
return an empty response instead of nil, nil when there are no errors
and the response is empty.
* refactor (handlers): Rename gRPC Gateway filter
Originally, this was called an OutgoingInterceptor, which is an
unfortunate name since it's not an interceptor. Now that we're
refactoring the gateway stuff and have proper interceptors, renaming
this should remove ambiguity
* refactor: Rework how 204 with empty msg is handled
Recent changes to grpc gateway has necessitated a change to how
204s are handled by the gateway
* tests (bats): Check status codes for delete operations
This moves all of the tcp target functionality into a tcp package. It
also establishes a Register function in the target package to allow
specific target types to be registered at initilazation time. When
registering, a target type needs to provide its Subtype, as well as some
RepositoryHooks that are used by the target Repository.
When creating or updating a target, the following fields should have
been marked as output only:
- host_set_ids
- host_source_ids
- application_credential_source_ids
- application_credential_library_ids
This updates the protobuf and generated files including the swagger file
used by the docs site to correctly mark these are output only.
Fixes#1588
This puts it more in line with other upcoming options that use "dir"
instead of "directory" (plus it's easier to type), and makes it clear
that this directory and the passthrough behavior is related to the UI as
opposed to other upcoming items.
Rather than pick addresses totally at random, this changes the
preference to be private IP4 > public IP4 > DNS. (Ordering is not
guaranteed, but in many cases there will only be one of each type of IP
anyways.) All three of these are preferred over IP6, which also goes
public > private.
This seems likely to match the common case where you're getting into a
machine on the private side from the public side. It's harder to know if
DNS is private or not, so IPs are preferred in this case.
This updates the docker image used for tests to support the new location
for sql migration files and the concept of editions. It continues to
process the old paths for now if the new directory does not exist, so
that the same image can be used during the transition.
This refactors the schema package and subpackages to support editions.
An Edition is a collection of sql migrations. A given boundary binary
can have multiple migration editions. The `schema.Manager` is updated to
apply the necessary migrations in the correct order, leveraging an
Edition's priority to determine the order in which the separate Editions
are applied.
The sql migration files are now nested under a directory for the
edition.
This also switches from the generated code to using embed, which
eliminates the need for the `make migrations` target and the generated
code. As such this commit removes the target and generated code.
See:
https://pkg.go.dev/embed
I've pulled over files as-is for the most part so this also pulls over
some common attribute/secret flags but it's not worth dealing with the
merge later...
* feat (events): Add status code to audit events
* feat (events): Add the resp proto msg to every audit event
* chore (events): Update eventlogger dep
This latest version will redact any map and/or map field which isn't
explicitly classified via a Taggable interface
* feat (events): Begin to add tagging to response proto msgs
* refact: Changes required for gorm v2
* refact: Add db.DB wrapper and refact all test fixtures to use it
Stop leaking the gorm abstraction within our unit tests.
* refact: Migrate direct calls to pq to pgx driver
The refactor will remove the remaining direct dependencies on
the pq driver which has been deprecated in favor of the pgx
driver.
It includes the addition of common.SqlOpen which is a wrapper around
sql.Open, which is needed to deal with the driver name changing from
"postgres" to "pgx" but in our code base the dialect and driver are
synonyms.
This adds a few command sketches but that's not the main point of the PR
and filling those out will come in a different PR. This adds a new flag
type that can combine inputs to a single target slice, retaining the
ability to distinguish where it came from but also keep ordering across
different flag names. It also implements a function that does the
heuristic parsing with fallback.
Added a new Endpoint struct at the top level host package level to capture host, set, and address chosen.
Temporarily add a host_plugin_host table to capture the hosts returned by the plugins. This ensures the datawarehouse is able to query the table to decide what host was chosen. The view used to update the datawarehouse dimension table provides rows with host set ids and hosts which are not in that host set. This is ok since the function which updates the dimension table provides the host set and host id chosen so that relationship doesn't need to be captured in the view itself.
Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
* Remove Prefix Id and Plugin Name fields. Creating catalogs now require either the plugin id or the name of the plugin resource (but not both). Host Catalogs and Host Sets API resources now have a read only plugin field populated with plugin info. Host Catalogs have a plugin id field as a top level field.
* Create minimalistic plugin map and test plugin.
* Pass secrets and attributes to the plugin when creating catalogs or sets. Add tests for CreateSet and CreateCatalog repo methods.
* make fmt and extract out getPrivateCatalog and getCatalog helper methods.
* Exporting TestPluginServer so it can be used in the handlers.
* Keep separate persisted data from host catalogs lookups which facilitate passing it onto the plugins.
* Add TODO to merge get Catalog and get Persisted data.
* Adding tests for unimplemented being returned by the plugin.
* Use GetFields() instead of AsMap() to confirm attributes should be populated. Make clone consistent between catalog and sets.
Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
If a new job is started as the test successfully compeltes and begins
the scheduler shutdown process. The new job will deadlock waiting on
channel reads.
Jeff Mitchell
Jim
Oleg Butuzov
Louis Ruch
Timothy Messier
Michael Gaffney
Randall Morey