* Remove the server_id from session table, change it to worker_id on session_connection
This is step 1 in updating the worker table. The worker is becoming a public resource so it must have
a public id. We drop the server_id column on the session_connection table and add the worker_id column
so we can use the wt_public_key type. Since db migrations are applied while the controller is offline,
it should be safe to assume all session connections are not in a running state, thus allowing us to null
out all the worker ids in the session_connection table.
For now we don't have very good enforcement of the wt_public_id constraints, since the id is currently
defined in the worker config, but this will soon change and the worker's id will be generated by the
controller.
* Move the server daemons into their own package
With this change the internal/server package now is dedicated to the server domain (workers and controllers) and not to the server daemons.
The hope is that this will make it easier to keep the daemons decoupled from the representation of the servers in the database.
Our test helper packages were not possible to use
with benchmarks. This refactor switches to testing.TB
to allow both tests and benchmarks to use the helpers.
This updates the target resources to use a oneof for attributes. This
allows requests to the targets service to leverage the new
subtypes.AttributeTransformerInterceptor so that attributes are strongly
typed prior to the service handler.
The new `subtype.AttributeTransformerInterceptor` can return an error if
it fails to transform the attributes. This is mainly if invalid
attribute fields were included for a particular subtype. This error case
used to be checked in the grpc service handler functions. However, now
that it is happening in the interceptor, the `errorInterceptor` needs to
be moved up in the stack so that it will process the response of the new
transformation interceptor.
The `errorInterceptor` used to be after the `auditRequestInterceptor`
and prior to the `auditResponseInterceptor`, meaning that the
`auditResponseInterceptor` would have access to the result of the
`errorInterceptor`. But it seems the `auditRequestInterceptor` does not
use or need any of the headers set by the `errorInterceptor` so it
should be safe to move the `errorInterceptor`. This is important since
the `subtype.AttributeTransformerInterceptor` needs to be before the
audit interceptors in the stack to properly transform requests/responses
so they can be audited with the complete classification information.
This reduces the chance that we'll hit a failure due to a mismatched
worker address, on the certificate, especially if a worker has multiple
addresses. We aren't relying on IP/DNS for security anyways, so let's
not have it be a hindrance.
This change introduces a `nonceFn` field that can be changed
by a test but at runtime is transparent to the code that calls it.
The current nonce-generating logic defaults to a test-only field which
opens us up to this value inadvertently being set.
With this change, the default behavior becomes using `base62` to
generate a random nonce, and any tests have to explicitly change that.
In general, these changes were introduced in the same vein of the
changes to controller `startListeners`.
Specifically, these changes also decouple the Controller and the Worker
shutdown processes. Currently, a call to `stopListeners` on an instance
that is running as both a Controller and a Worker would stop the
listeners for both.
Lastly, we also add unit/integration tests to the shutdown code.
This creates a domain service function `session.WorkerStatusReport`. The
service performs two functions:
1. Checks the status for all of the reported sessions, returning a
report of the sessions that are in a "non-active" state, i.e.
"cacneling" or "terminated".
2. Looks for any "orphaned" session connections, i.e. connections that
are active but were not reported by the worker. These connections are
marked as closed.
This also removes the "ShouldCloseConnectionsOnWorker" logic, as this
was determined to not be a possible state, and was unnecessary.
* feat(host): Add host set information to static host
Updates LookupHost and ListHost API to include sets a static host is part of, as like with plugin hosts.
* test: updated TestList_Plugin and TestCrud to be more consistent under race conditions
This addresses the issue of a user being created in a normal or
dev environment that is given access to authorize sessions against
specific targets but the permission hasn't been configured for the UI
clients to actually list them.
Resolves#1791
* Update sync interval seconds field.
* If the sync interval was updated, schedule the next sync job wakeup immediately.
* Calculate the next sync time for the scheduler should wake up if we only updated the sync interval.
* Create catalog service handler for updating plugin. Add tests for handler and run make fmt. Silently drop attributes where the value is nil when creating the catalog or set.
* Add ability to read and list hosts. Tests to follow.
* Add tests for reading and listing as well as validation checks to not modify hosts.
* Add IpAddress and DnsNames to the api resource. Populate the plugin info.
* Adding go sdk tests for reading and listing hosts and ensuring update and deleting is disallowed.
This stores worker connection nonces within the database to ensure
uniqueness of connections. Prior the connection required any attacker to
be able to decrypt the incoming encrypted authentication message in
order to replay (and if the attacker has access to the KMS you have
bigger problems), but with this change even if an attacker does have
access to the KMS key, they cannot reuse the same message. (If they're
able to encrypt against the KMS key, they can still act as a worker with
a new message, so this only really helps if they only have decryption
capabilities.)
* refactor (oidc): Stop emitting error for not found token id
Stop emitting errors when authtoken repo.IssueAuthToken is called and
there's if no pending token. It's not an "normal" state and not
an error condition.
* feature (audit tagging): Add tags for auth method requests/responses
Including testing functions for asserting audit events created
when a service is called
* refactor (controller): Use a grpc server for the grpc-gateway
This starts a grpc server for the grpc-gateway. The new gateway
server uses an in-memory listener which also supports "dialing", so
the http proxy can connect to the in-memory listener.
This change allows us to use all grpc server features and was in part
motivated by a need for an interceptor of inbound requests. With that
said, having a "proper" grpc server for the gateway will provide a lot
more flexibility and capabilities moving forward.
The request context will still be verified via middleware in the
http proxy (see wrapHandlerWithCommonFuncs). This verified context
will be serialized into a protobuf and sent along to the grpc-gateway
via metadata where an interceptor will add the verified request
info to the request's context (see requestCtxInterceptor). The
interceptor will verify the serialized request info via a shared ticket
with it's companion in-memory http proxy. FYI: the ticket is simply
random base62 string (see db.NewPrivateId).
This change also necessitated a minor change to services. We need to
return an empty response instead of nil, nil when there are no errors
and the response is empty.
* refactor (handlers): Rename gRPC Gateway filter
Originally, this was called an OutgoingInterceptor, which is an
unfortunate name since it's not an interceptor. Now that we're
refactoring the gateway stuff and have proper interceptors, renaming
this should remove ambiguity
* refactor: Rework how 204 with empty msg is handled
Recent changes to grpc gateway has necessitated a change to how
204s are handled by the gateway
* tests (bats): Check status codes for delete operations
This moves all of the tcp target functionality into a tcp package. It
also establishes a Register function in the target package to allow
specific target types to be registered at initilazation time. When
registering, a target type needs to provide its Subtype, as well as some
RepositoryHooks that are used by the target Repository.
The existing logic was to ignore Forbidden errors when recursively
listing in favor of looking for downstream scopes with appropriate
permissions. The problem is that when there is actually no
authentication token at all, what comes back is an Unauthenticated
error. Therefore if you didn't have permissions on the ingress scope
(e.g. the default of `global`) but did downstream for the anonymous
user, you'd get back an error.
I don't think there's any security issue with doing this; the user could
always have simply started the search at a downstream scope to begin
with so nothing more is being revealed, this just fixes behavior to
match expectations in this case.
A test was added for one of the services; the code is exactly the same
across them. The reason that the existing tests that were added with the
output fields testing didn't catch this is that even though they were
specifying `u_anon` as the user, the disabled test context skipped the
logic that would return a 401 in this case. It turns out that fixing
this is very complicated because we don't have a great way in most tests
of defining grants to put onto the generated scopes, so even if a true
anonymous context was added each test would require the anonymous user
to have list permissions on whatever resource was under test, requiring
a lot of role and grant and principal finagling. This is probably a good
longer-term TODO but not one that seemed worth it for this fix.
Fixes ICU-2301
Jim
Todd
Johan Brandhorst-Satzkorn
Timothy Messier
Jeff Mitchell
Hugo Vieira
Irena Rindos
Haotian
Louis Ruch
Todd