edd323b73a Key Rotation/Destruction (#2477) (#2607)
* Key Rotation/Destruction (#2477)

* fix: Correct kms.CreateKeys comment

This method does not return anything.

* feat(kms): Add new ListKeys method

The new ListKeys method wraps the underlying KMS wrapper,
returning all the keys in the scope specified.

* feat(scopes): Add new scope actions for key management

Adds list-keys, rotate-keys and revoke-keys actions for scopes.

* feat(scopes): Add ListKeys API endpoint

This new endpoint lists all keys in a scope

* feat(api): Add ListKeys to Scopes client

This has to be a custom function because it is a custom action
and doesn't map well to any of the existing templates, or
generalises well in a way that would make it reasonable
to create a new template.

* feat(cli): Add new scopes list-keys command

* add RotateKeys function to the kms repository

* Add Rotate Keys Endpoint to Scopes Api (#2360)

* add rotate keys endpoint

* add tests for RotateKeys endpoint

* remove that one comment I forgot about

* addressing more PR comments

* Add Rotate Keys CLI Command (#2395)

* add cli command and client

* add bats tests

* fix some info text and pr comments

* write a new rotate keys description

* Add Missing DEK Foreign Keys (#2408)

* add missing fks

* mark key as nullable in gorm

* update tests to use a new wrapper with a real key_id

* fix formatting in test and add keys for auth_token test

* replace key values

* revert postgres_40_01_test.go

* style: reformat sql for readability

* refactor: change type of key_id columns to kms_private_id

* Store key_id and scope_id with oplog entries, re-type columns referencing data key version (#2431)

* fix(db): Correct types of data key version referencers

* fix(oplog): Fix missing error handling

* feat(oplog): Store key_id and scope_id with oplog entries

This migration truncates all existing oplog entries, as
migrating existing data would have been complex and
likely unnecessary.

* Update view references

* Always delete oplog entries when dropping keys

* Fix rebase issues

* Fix sql tests

* Add schema for new kms destruction jobs (#2479)

* feat: Add schema for new kms destruction jobs

The schema lets us manage destruction jobs per-table
while providing a guarantee that only one run is running
at a time.

* Apply suggestions from code review

Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>

* Fix table reference and tests

Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>

* update domain, api, and cli to all include key versions in outputs (#2472)

* update domain, api, and cli to all include key versions in outputs

* fix go sum

* add migration to fix immutable error

* fix final test issue as well as make gen issue

* rebase and fix

* address johan pr comments

* make gen, didn't realize comments were included

* update migrations based on new migration 55 file 03

* Move migrations to 56

* feat: Add key version rewrapping function registry (#2478)

The new RegisterTableRewrapFn can be used by a domain
to register a callback to use when rewrapping data in a
specific table name.

* feat(kms): Add scopeId to RewrapFn (#2539)

This makes it much easier for rewrap functions to look up
the correct wrapper and limit the number of rows searched.

* Update Root Cert Proto Definition (#2542)

* update root cert proto definition to properly identify public_key as the table primary key

* add the test fix as well

* Rewrap Functions for Encrypted Tables (#2532)

* add oidc rewrap

* add argon2 config rewrap

* add auth token rewrapping

* add worker auth cert rewrapping

* check the err

* add username password credential rewrapping

* add ssh private key credential rewrapping

* add vault client certificate rewrap function

* add host catalog secret rewrap function

* cleanup a little

* add host vault token rewrap function

* add session credential rewrap function

* add session rewrap function

* add worker auth rewrap function

* update test to include full assert gamut

* add session rewrap function TEST

* bring all tests up to current standards, make more readable, and include full assert gamut

* rework all rewrap functions to utilize scope id

* update comments, queries, sql refs, etc, in response to PR comments

* forgot a comment

* remove hmac updates and fix a couple comments again

* Rewrapping Registered Tables Test (#2555)

* add the registered table test as well as the two missing rewrap functions

* address pr comments, add db r/w conversions, cmp.diff, sql comment, remove (now) faulty immutable test

* Add ability to list key version destruction jobs, recurring jobs and destroying key versions (#2498)

* feat(kms): Add ability to list data key version destruction jobs

* feat(scopes): Add ListKeyVersionDestructionJobs

* feat(cli): Add list-key-version-destruction-jobs

* feat(kms): Add recurring job that performs table rewrapping

The new recurring job runs for each job table with a registered
rewrapping callback. It attempts to become the running run and
start its rewrapping, gracefully handling sudden interruptions
by resuming it's work if it finds evidence of sudden interruption.

* feat(kms): Add recurring job for destroying key versions

The new job monitors the database for data key version destruction
jobs that have finished rewrapping all its data, performing
the final data key version revocation.

* feat(kms): Add DestroyKeyVersion

DestroyKeyVersion immediately destroys a key version
if it is a root key version or a data key version
which encrypts no data. If the data key version
encrypts data which needs to be rewrapped, it
queues an asynchronous job to complete the rewrapping
operation.

* feat(scopes): Add DestroyKeyVersion API endpoint

* feat(cli): Add scopes destroy-key-version CLI command

* Ensure all secret updates include key ID (#2556)

* feat(all): Ensure all secret updates include key ID

When updating a secret, it is paramount that the key ID
is also updated, as it is the only means we have of ensuring
that we only destroy keys once there is no more data
associated with the key.

* Remove fix to session repository for now

This breaks the current private key derivation method

* add new encrypt/decrypt funcs and made necessary proto changes (#2557)

* add new encrypt/decrypt funcs and made necessary proto changes

* address all PR comments

* fix test panic

* add param checking to rewraps

* fix(schema): Correctly organize migrations again

* Review comments part 1

* Review comments part 2

* Review comments part 3

* Store cert private key, encrypt tofu token (#2583)

* fix(session): store cert private key, encrypt tofu token

Previously, we would derive a session certificate private key from
the session key used in each project, and not store the key. We
would also encrypt the tofu token with the database key but not
store a reference to this key in the database.

This change fixes this by replacing our key derivation with a key
generation step, and instead store the generated key, encrypted,
in the database. We also ensure that any new tofu tokens are
encrypted with the same key.

To handle existing sessions, we lazily rewrite the sessions whenever
a user lists them. There is a minor risk that a user could end up
destroying the database key that encrypts the tofu token before
that session has been rewrapped, but this is going to be rare and
temporary.

* feat(session): Allow the random source for secrets to be configured

* Review comments

* Minor fixes

* More minor fixes

* More small fixes

Co-authored-by: Danielle Miu <dani.miu@hashicorp.com>
Co-authored-by: Danielle <29378233+DanielleMiu@users.noreply.github.com>
Co-authored-by: Michael Gaffney <mike@gaffney.cc>
Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>

* fix(session): Unset session key ID when scope is deleted

Unsetting the key ID allows the session to live beyond the lifetime of
the scope, while allowing the scope to cascade delete its keys.

* fix(migrations): Rewrite migration 56 tests

These tests can no longer use the normal test helpers
as they try to use the new oplog table structure, so we
have to manually write all the interactions.

* fix(migrations): Rename migration 58 file name

This was the name used in the release branch, update
it to the new migration number.

* fix(e2e): Add some comments and debug to kms tests

* fix(session): Handle empty project and user IDs in read

When a project or user is deleted, the session is automatically
canceled and the respective field on the session is unset.
LookupSession did not handle this case gracefully, and would
error on decryption in this state. Skip decrypting sessions that
do not have either a user or project to avoid this error. The
decrypted values are not used for a canceled session.

* Fix worker test

* Increase test timeout

Co-authored-by: Danielle Miu <dani.miu@hashicorp.com>
Co-authored-by: Danielle <29378233+DanielleMiu@users.noreply.github.com>
Co-authored-by: Michael Gaffney <mike@gaffney.cc>
Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>
2022-11-16 18:17:28 -05:00
2021-10-15 14:03:26 -04:00
2022-09-07 17:29:13 -04:00
2022-09-13 10:16:48 -07:00
2020-10-13 09:15:51 -07:00
2022-10-12 12:05:15 -04:00
2022-08-10 15:25:46 -04:00

Boundary

CircleCI

Please note: We take Boundary's security and our users' trust very seriously. If you believe you have found a security issue in Boundary, please responsibly disclose by contacting us at security@hashicorp.com.


Boundary provides simple and secure access to hosts and services.

Traditional approaches like SSH bastion hosts or VPNs require distributing and managing credentials, configuring network controls like firewalls, and exposing the private network. Boundary provides a secure way to access hosts and critical systems without having to manage credentials or expose your network, and is entirely open source.

Boundary is designed to be straightforward to understand, highly scalable, and resilient. It can run in clouds, on-prem, secure enclaves and more, and does not require an agent to be installed on every end host.

Unlike firewalls, Boundary performs per-access authentication and authorization checks, allowing for much higher level mappings of users to services or hosts than at network layers. Although complementary to secrets managers (like HashiCorp's own Vault), Boundary fills a different niche, allowing the credential that is eventually used to be hidden entirely from the user.

Getting Started

Boundary consists of two server components: Controllers, which serve the API and coordinate session requests; and Workers, which perform the actual session handling. A normal Boundary installation will consist of one or more Controllers paired with one or more Workers. A single Boundary binary can act in either of these two modes.

Additionally, Boundary provides a client that provides access to request and connect to authorized sessions.

Boundary does not require software to be installed on the endpoint hosts and services.

Requirements

Boundary has two external dependencies: a SQL database, and one or more KMSes. Both are readily available from cloud vendors, but can be satisfied by on-premises technologies as well.

  • The database contains Boundary's configuration and session information and must be accessible by Controller nodes. Values that are secrets (such as credentials) are encrypted in the database. Currently, PostgreSQL is supported as a database and has been tested with Postgres 11 and above. Boundary uses only common extensions and both hosted and self-run instances are supported. In most instances all that is needed is a database endpoint and appropriate credentials.

  • Any cloud KMS or Vault's Transit Secrets Engine can be used to satisfy the KMS requirement. Currently, two keys within the KMS are required: one for authenticating other cluster components, which must be accessible by both Controllers and Workers; and one for encrypting secret values in the database, which need only be accessible to Controllers. These keys can be changed over time (so long as the original key remains available for any decryption needs), and key derivation is used extensively to avoid key sprawl of these high-value keys. If available, other keys can be used for other purposes, such as recovery functionality and encryption of sensitive values in Boundary's config file.

Boundary has a dev mode that can be used for testing. In this mode both a Controller and Worker are started with a single command, and they have the following properties:

  • The Controller will start a PostgreSQL Docker container to use as storage. This container will be shut down and removed (if possible) when the Controller is (gracefully) shut down.
  • The Controller will use an internal KMS with ephemeral keys

Trying out Boundary

Running Boundary in a more permanent context requires a few more steps, such as writing some simple configuration files to tell the nodes how to reach their database and KMS. The steps below, along with the extra information needed for permanent installations, are detailed in our Installation Guide.

Build and Start Boundary in Dev Mode

NOTE: Do not use the main branch except for dev or test cases. Boundary 0.10 introduced release branches which should be safe to track, however, migrations in main may be renumbered if needed. The Boundary team will not be able to provide assistance if running main over the long term results in migration breakages.

If you have the following requirements met locally:

  • Golang v1.19 or greater
  • Docker
  • Either the Boundary UI Dependencies for locally building the ui assets or gh cli for downloading pre-built ui assets.

You can get up and running with Boundary quickly. Simply run:

make install

This will build Boundary. (The first time this is run it will fetch and compile UI assets; which will take a few extra minutes.) Once complete, run Boundary in dev mode:

$GOPATH/bin/boundary dev

Please note that development may require other tools; to install the set of tools at the versions used by the Boundary team, run:

make tools

Without doing so, you may encounter errors while running make install. It is important to also note that using make tools will install various tools used for Boundary development to the normal Go binary directory; this may overwrite or take precedence over tools that might already be installed on the system.

Specify a UI Commitish at Build Time

By default the UI will be built from a preselected commit ID from the UI repo. A different commitish from which to build UI assets may be specified via the UI_COMMITISH environment variable. For example:

UI_COMMITISH=feature-branch make build-ui

will update your local UI assets.

UI Build Troubleshooting

UI assets are built within a Docker container. If you encounter issues with this build step, trying increasing memory and swap available to Docker.

UI Development

It would be impractical to rebuild the binary on every change when actively developing the UI. To make UI development more convenient, the binary supports a passthrough directory. This is an arbitrary local directory from which UI assets are served. Note this option is only available in dev mode. For example:

BOUNDARY_DEV_UI_PASSTHROUGH_DIR=/boundary-ui/ui/admin/dist ~/go/bin/boundary dev

Download and Run from Release Page

Download the latest release of the server binary and appropriate desktop client(s) from our releases page

Start Boundary

Start the server binary with:

boundary dev

This will start a Controller service listening on http://127.0.0.1:9200 for incoming API requests and a Worker service listening on http://127.0.0.1:9202 for incoming session requests. It will also create various default resources and display various useful pieces of information, such as a login name and password that can be used to authenticate.

Configuring Resources

For a simple test of Boundary in dev mode you don't generally need to configure any resources at all! But it's useful to understand what dev mode did for you so you can then take further steps. By default, dev mode will create:

  • The global Scope for initial authentication, containing a Password-type Auth Method, along with an Account for login.
  • An organization Scope under global, and a project Scope inside the organization.
  • A Host Catalog with a default Host Set, which itself contains a Host with the address of the local machine (127.0.0.1)
  • A Target mapping the Host Set to a set of connection parameters, with a default port of 22 (e.g. SSH)

You can of course go into Boundary's web UI or use its API to change these default values, for instance if you want to connect to a different host or need to modify the port on which to to connect.

Making the Connection

Next, let's actually make a connection to your local SSH daemon via Boundary:

  1. Authenticate to Boundary; using default dev values, this would be boundary authenticate password -auth-method-id ampw_1234567890 -login-name admin -password password. (Note that if you do not include the password flag you will be prompted for it.)
  2. Run boundary connect ssh -target-id ttcp_1234567890. If you want to adjust the username, pass -username <name> to the command.

A lot more is possible with Boundary, even at this early stage. Check out the possibilities for target configuration to test out limiting (or increasing) the number of connections per session or setting a maximum time limit; try canceling an active session from the sessions page or via boundary sessions, make your own commands with boundary connect -exec, and so on.

Going Further

This example is a simple way to get started but omits several key steps that could be taken in a production context:

  • Using a firewall or other means to restrict the set of hosts allowed to connect to a local service to only Boundary Worker nodes, thereby making Boundary the only means of ingress to a host
  • Using the Boundary Terraform provider to easily integrate Boundary with your existing code-based infrastructure
  • Pointing a BI tool (PowerBI, Tableau, etc.) at Boundary's session warehouse to generate insights and look for anomalies with respect to session access

There are many, many more things that Boundary will do in the future in terms of integrations, features, and more. We have a long roadmap planned out, so stay tuned for information about new features and capabilities!

Contributing

Thank you for your interest in contributing! Please refer to CONTRIBUTING.md for guidance.

S
Description
Community-maintained fork of HashiCorp Boundary Community Edition (BSL 1.1). Not affiliated with HashiCorp or IBM.
Readme
235 MiB
0 Stars 1 Watchers 0 Forks
Languages
Go 90.2%
PLpgSQL 7%
HCL 1.8%
Shell 0.7%
MDX 0.1%