The docker package and its StartDbInDocker function should only be used
by the `boundary dev` command now. So moving it into an internal package
of `base` will prevent it from being use for tests accidentally, while
still allowing it to be used for `boundary dev`.
This updates the test-ci target to startup the test database container
first, wait for it to be ready, and then start running the tests.
It also creates some new targets for local devs. Prior to running any
tests either via `make test` or `go test`, the database should be
started with:
make test-database-up
It can take a few seconds for the database to fully initialize, the
status can be checked with:
docker logs -f boundary-sql-tests
This database can be stopped and destroyed with:
make test-database-down
In general it should only need to be restarted when new migrations are
added. But it might also be necessary to restart if some tests fail in
an unexpected way and fail to do cleanup steps.
This swaps the test initialization to use the dbtest package so each
test will get a fresh database created from a template database, instead
of a completely new docker container.
This is modeled after the internal/docker package to make it easy to use as a
replacement. Instead of creating a new docker container for each test,
it will create a new database in an existing database container, using a
template database as a source. This assumes that the template database
exists and already has the schema migrations run against it.
The default template is `boundary_template` which should already have
all migrations applied. However, some tests require an empty database to
test the migration code. In these cases the `template1` template
database should be used, ie:
dbtest.StartWithTemplate("postgres", dbtest.WithTemplate("template1"))
This images is built from the official postgres docker image, but with
updates to optimize the test execution:
- A config file for postgres to increase the performance of the
test server.
- An initialization script to create a template database with all
migrations run against it.
There are new make targets to build and publish this docker image.
The image can be pulled with:
docker pull hashicorpboundary/postgres:13-alpine
To use this image for tests start the container:
make -C internal/db/docker database-up
* feat(host): Add host plugin related schemas and tests
This adds the schema files for host plugins, plugin based catalogs and
plugin based sets. This also adds the protos and tests which utilize
gorm to ensure the schemas enforce the constraints we expect.
Along with this change was a migration which moved catalog names to the
base type table so we can ensure all catalogs in a single scope,
regardless of subtype, have a unique name.
* Updated plugin schemas to do the following:
- Add plugin_executable table
- plugin table now also has scope_id and name w/ uniqueness constraints
- host_plugin no longer has id_prefix column
- host_plugin separates name and plugin_name
- plugin_version table renamed version to symantic_version column.
* Fix typo in delete_plugin_subtype trigger. Add update test for host plugins.
* Update id prefixes. Fix public_id typos in protos and plugin_executable name in sql diagrams.
* Created internal/plugin and moved base plugin structs to it and moved host plugin subdirectory to it.
* Created internal/plugin and moved base plugin structs to it and moved host plugin subdirectory to it.
* Add first pass at enums for OS and Arch supported by plugin executables.
* Capitalizing the const enums for OS and Architectures.
* Add comments to sql files.
* Reviewer comments.
Remove unsupported OSes and Archs from SQL and go constants.
Add missing tests.
* Be more permissive in the values the db accepts.
The go logic can restrict which values it will allow getting populated into the db.
* Removing constraints on which values can be placed inside operating_system and architecture columns.
* Updating test helper functions to only create a single catalog or set.
* Move files to new migration version.
* Fixing job/plugin test.
* Update table names, named a number of constraints, remove plugin version and executable tables.
* Add constraints for plugin name and id prefix.
* Rename constraint, remove unhelpful comment block.
* Adding semantic version to plugin_host and set attributes to not null.
* Remove semantic versioning from all locations.
* Remove semantic_version from plugin_host table comment.
* Fix op copy/paste error. Don't use deprecated errors methods. Update constraint names. Add docs.
* Apply suggestions from code review
Fixing constraint names.
Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>
* `recovery` is only used by the API (and the main module, for decryption)
* `wrapper` is only used by the CLI right now
Moving these brings them closer to the code that uses them, and makes
the SDK package a bit more clear in terms of functionality.
When a session is terminated by the user, the cancel session path is
triggered, in this path the websocket connection is closed. If the
connection is already closed, an EOF is returned. In this situation an
EOF is expected and should not be logged to the user.
Example log line:
{
"id": "ezqk6w9dOQ",
"source": "https://hashicorp.com/boundary/dev-controller/boundary-dev",
"specversion": "1.0",
"type": "error",
"data": {
"error": "failed to close WebSocket: failed to read frame header: EOF",
"error_fields": {},
"id": "e_n60YPsDkaj",
"version": "v0.1",
"op": "worker.(Worker).handleProxy",
"info": {
"msg": "error closing client connection"
}
},
"datacontentype": "text/plain",
"time": "2021-09-02T10:05:46.587513-07:00"
}
The existing logic was to ignore Forbidden errors when recursively
listing in favor of looking for downstream scopes with appropriate
permissions. The problem is that when there is actually no
authentication token at all, what comes back is an Unauthenticated
error. Therefore if you didn't have permissions on the ingress scope
(e.g. the default of `global`) but did downstream for the anonymous
user, you'd get back an error.
I don't think there's any security issue with doing this; the user could
always have simply started the search at a downstream scope to begin
with so nothing more is being revealed, this just fixes behavior to
match expectations in this case.
A test was added for one of the services; the code is exactly the same
across them. The reason that the existing tests that were added with the
output fields testing didn't catch this is that even though they were
specifying `u_anon` as the user, the disabled test context skipped the
logic that would return a 401 in this case. It turns out that fixing
this is very complicated because we don't have a great way in most tests
of defining grants to put onto the generated scopes, so even if a true
anonymous context was added each test would require the anonymous user
to have list permissions on whatever resource was under test, requiring
a lot of role and grant and principal finagling. This is probably a good
longer-term TODO but not one that seemed worth it for this fix.
Fixes ICU-2301
Although we document that this is allowed, and the ACL code handles it,
some of the validation code did not. This would cause a semi
intermittent problem because that validation code is used only in some
code paths; specifically we skip it (for now) to avoid erroring out on
some deprecated grant string formats when reading grants from the DB,
but we use it to error out when validating incoming grants.
* sdk: add host plugin protobufs and interface
This adds the protobufs and interface for the upcoming plugin-based host
feature for boundary, providing a common interface for building targets
and host catalogs off of dynamic cloud data.
* Fix signature formatting and separate out/flesh out descriptions
* sdk: re-gen with protobuf v3.15.8
* sdk: update protobuf according to @talanknight's comments
* Update internal/proto/local/plugin/host_catalog.proto
Co-authored-by: Todd Knight <T.Alan.Knight@gmail.com>
* Add TODO for host catalog persisted data for size limits
* sdk: move id to external_id and make protobuild
* Move plugin interface to GRPC service
This re-uses existing protobufs where possible, a few more messages have
been created for persisted data and for the host response where it's
important that we make it clear that the external ID is *not* the same
ID as the calculated stable ID for the host within Boundary.
* Update internal/proto/local/plugin/host_plugin_service.proto
Co-authored-by: Todd Knight <T.Alan.Knight@gmail.com>
* Move proto target to plugin/proto, fix formatting
Co-authored-by: Todd Knight <T.Alan.Knight@gmail.com>
* feat (events): Add new options needed for AuditConfig
WithAuditWrapper and WithFilterOperations all for optional paramters
when creating AuditConfig types.
* refactor (event): Change EventerConfig.Sinks to a ptr
We need to modify the SinkConfig.AuditConfig during the NewEventer factory, so we have to use a ptr not a ref.
* feat (events): Add audit configuration types
* feat (events): add audit config to sink configuration.
* feat (events): add redact/encrypt node for audit events to Eventer
Integrate the encrypt.Filter node from the eventlogger into the
event.Eventer, so audit events can be redacted/encrypted based
on data classifications.
* test (events) Add test protobufs for events
Add test protobufs along with a test service for testing
the audit event encrypt node
These fields are being refactored so they conform with the new
wh_credential_dimension. This will inform Business Intelligence tools
that these attributes conform across dimensions.
Todd Knight
Timothy Messier
Susmitha Girumala
Randall MoreyandSusmitha Girumala
Jeff Mitchell
Chris Marchesi
Kevin Wang
Louis Ruch
Jim
Rolf Sommerhalder