Files
threshold/website/content/docs/installing/no-gen-resources.mdx
T
df35699c4e Integrate with Vault to retrieve and manage per session credentials (#1308)
* Fix down migrations

* Udpates due to merge

* Sentinel values only have to be greater than 0

* Create credential related sdk structs and methods.

* Post merge interface updating.

* Removing scratch code that was not cleaned up.

* updates

* Convert client certificate and client certificate keys into pem blocks and validate.

* Add newline at end of file.

* Faster check for "s" at the end of a resource name when generating from templates.

* make fmt

* updates

* Add dynamic credentials to a session

* Refactor: move contents of file

* updates

* Updates

* Add InvalidDynamicCredential error

* Assign credentials

* Allow updating client certificates seperate from the client certificate key.

* Replace application purpose string with enum value

* PR feedback / fix go test .

* Delete dead code

* Use common view

* Refactor: extract interface and rename method

* Refactor: move code to eliminate privPurpLibrary

* Add comment to requestMap

* Refactor: do not export methods on an unexported struct

* Update internal/db/schema/migrations/postgres/10/03_vault_credential.up.sql

Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>

* Add comments to credential Purpose constants

* Reorder switch statements to handle errors first

* PR feedback

* update

* fix merge

* Organizing the scheduler code and extract the individual store creation test helper function.

* Run 'make fmt'

* Create certificate when creating store.  Allow vault address to be passed in as option

* Refactor: rename database views and associated structs

This change renames two database views and associated go structs to use
a consistent naming pattern. Views ending in 'private' contain encrypted
credentials needed for connecting to Vault. Views ending in 'public'
mirror the 'private' views but do not include any encrypted values.

This change does not include any changes to functionality or behavior.

* Adding SAD target operations for credential libraries to the SDK.

* Add status field and enumeration table for credential status

* Refactor: rename Status to TokenStatus

* Add status to vault credentials

* Creating the template for the CLI and helper functions.

* Add private database view for vault credentials

* Add scope_id to view and cleanup TODO comment

* Reformat embedded sql queries

* Add vault client helpers

* updates

* white space

* Add LookupLease test

* CLI commands for Store work.  Renamed fields in proto w/ vault prefix.

* Test issue credentials with client TLS

* Add additional checks to vault ping

* Add an error for vault token capabilities and create a VaultToken error kind

* Add support for working with Vault policies

* Use Vault 1.7.2 for testing

* Add testing helper for creating a vault client with a non-root token

* Fix Vault renew lease test

The Vault renew lease endpoint accepts an optional duration parameter
for requesting the number of seconds to extend the lease by. However,
Vault is not required to honor this request.

* Add a var for the required capabilities of a boundary Vault token

* Enhance the test helper for adding a policy to Vault

* Allow the test role in postgresql to revoke credentials

* Remove out of date comments

* Enhance Vault test helpers for the database secrets engine

* Reformat comments

* Add a method for getting the capabilities of the current Vault token

* Add revoke lease to internal Vault client

* Reformat: combine err return line with check for nil line

* Remove redundant checks in test code

Remove checks in test code that are redundant with checks in the test
harness.

* Refactor CreateToken test helper to return the secret and the token

* Fix build failures caused by signature change in vault.CreateToken

* Fix Vault version and Vault API link in comments

* ran `make gen` after merging from mgaffney-vault.

* Remove Vault prefix from attribute fields in API, create a base CLI option for adding field prefixes. format now uses that prefix for error reporting.

* Template now allows attaching attribute field prefixes.

* Credential store uses PrefixAttributeFieldErrorsWithSubactionPrefix.

* Fixing naming of vault flags and helper flags.

* Updating more references to generated api methods.

* Make the capabilities String method more readable

* Add comment outlining PrintApiError usage of Options.

* Refactor test to move declarations closer to first use

* Fix comment

* Implement revoke method on vault repository

* Tests now verify certs, CA, and tokens, addresses can be updated in a store.

* Added test for creating store with pk/cert in same field.

* Allo updating certificates seperate from private key.  Only contact backing vault service when updating token.

* Fixing bad tests and removing unused code.

* Adding tests for various ways of updating tokens.

* Add private credential

* Add token revocation job and cred renewal/revocation job

* Regen after merge.

* Fixing field mask related merge error.

* Create Library CLI. Removes vault prefix from path. Adds a WithoutCleanup testOption for vault.

* Set all credentials to revoked when a token is revoked

* Fixing missed renaming update.

* fix copy/paste error.

* Fixing code generation errors after merge.

* make gen after merge.

* Update internal/cmd/commands/credentialstorescmd/vault_funcs.go

Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>

* Update internal/cmd/commands/credentialstorescmd/vault_funcs.go

Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>

* Update internal/cmd/commands/credentialstorescmd/vault_funcs.go

Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>

* Refactor: move migrations from 10 to 12

* Removing logic from transaction.

* wrapping client errors in the calling op for CredentialStore's client() method.

* Redact token

* redact client key

* updates

* Add trigger to revoke credentials when session is canceled or terminated

* Fixing some help text for some flags.

* Add ability to load values from files (file://) or environment variables (env://).

* Updating comments

* fix bad merge

* feedback

* Adding issued credentials to sessions.

* Error on capabilities

* updates

* Expose NewVaulTestServer

* Add godoc

* Adding generated SessionCredential struct for API.

* Progress on building handler test for Authorize Session.

* feedback

* Update policies

* Add 'revoke' as a status for Vault Tokens

* Add delete_time to vault credential store

* Create dynamic credentials at session creation time.

* Tests pass with credentials attached to AuthorizedSession result.

* Only call issue if there are credentials to issue.

* Fixing output only issues. Adding type to credential library.

* Fix test to clear un/pw generated.  confirm generated un/pw changes from 1 authorize session call to another.

* Update lookupTokenError

* Add delete_time to private credential store

* Refactor: move private store to separate file

* Fixing tests for AuthorizeSession and verifying error cases.

* Only create the credential repo if there are credentials to issue.

* Rename Library field to CredentialLibrary.

* Fixing missing change of Library to CredentialLibrary.

* Removing Default SDK functions for fields which are required and update CLI to match.

* Replace resource specific id variables with "id" and the input parmater for paths to be just a single string and the resource name as it should show up in the path.

* Soft delete vault credential stores

* Prevent new SQL functions from silently overwriting existing functions

* Rename "Hmac" to "HMAC" in the human readable cli response table.

* Fix tests

* Add vault database cleanup jobs

* Add ability to set the token period in the vault test harness

* updates

* Add not_null_columns func test

* remove unused struct

* Up no output timeout

* Validate current vault token on addr update

* Fix client cert change test

* add cert test

* Update revocation job where

* Fix comment

* Use Vault's default port in CLI help strings

* Remove down migrations

* Increase migration file numbers by 1

* Run make migrations

* Move migrations from 12 to 10

* Increase Circle CI timeout for acceptance tests

* Fix spelling mistake in error messages

* ASD cmds for Target and returning credentials to AuthSession request (#1338)

* Add Add/Set/Remove commands for libraries on Targets.  Return credentials on AuthorizeSession request.

Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>

Co-authored-by: Louis Ruch <louisruch@gmail.com>
Co-authored-by: Todd Knight <T.Alan.Knight@gmail.com>
Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
2021-06-22 22:26:26 -04:00

485 lines
12 KiB
Plaintext

---
layout: docs
page_title: Running Boundary in Non-Dev Environments
description: |-
How to install boundary in non-dev environments.
---
# Installing Boundary Without Generated Resources
What are generated resources? When you run `boundary dev` or `boundary database init`, Boundary automatically
generates a number of resources to make getting started easier. Default scopes, auth methods, user, account, and
targets are just some of the resources Boundary will generate unless you tell it not to.
In a production or long-running environment, these resources are not necessary, but without them, managing
Boundary from scratch isn't straight forward. How do you create your first user and login to administer a Boundary
deployment that has no authentication methods, users, accounts, etc.? This section describes how to get your freshly
deployed Boundary installation off the ground for non-dev environments.
## Recovery KMS Workflow
Initializing Boundary without generated resources starts with your Boundary configuration file. Specifically,
the controller configuration specifies three KMS blocks:
```hcl
<truncated>
kms "aead" {
purpose = "root"
aead_type = "aes-gcm"
key = "sP1fnF5Xz85RrXyELHFeZg9Ad2qt4Z4bgNHVGtD6ung="
key_id = "global_root"
}
kms "aead" {
purpose = "worker-auth"
aead_type = "aes-gcm"
key = "8fZBjCUfN0TzjEGLQldGY4+iE9AkOvCfjh7+p0GtRBQ="
key_id = "global_worker-auth"
}
kms "aead" {
purpose = "recovery"
aead_type = "aes-gcm"
key = "8fZBjCUfN0TzjEGLQldGY4+iE9AkOvCfjh7+p0GtRBQ="
key_id = "global_recovery"
}
<truncated>
```
In this example, we're using hardcoded AEAD keys, but in a real world non-dev deployment, you
should use your cloud provider's KMS such as [AWS KMS](https://www.boundaryproject.io/docs/configuration/kms/awskms) to manage the keys Boundary
uses to encrypt sensitive information.
The KMS block we're focused on is the `recovery` block. This block specifies the key used to "recover" Boundary
but you can also use it to authenticate to Boundary and manage it as a "global" super user. This allows
you to authenticate from the CLI or from Terraform in order to manage Boundary without any generated
resources.
To authenticate to Boundary using the recovery KMS workflow:
<Tabs>
<Tab heading="CLI">
To use the recovery workflow on the CLI, you must pass the `-recovery-config <path_to_kms_recovery_config>` flag or set the environment
variable for `BOUNDARY_RECOVERY_CONFIG` for every command ran. Authentication takes place for every command
ran when using the recovery workflow, there is no `boundary authenticate` step:
```bash
$ cat << EOF > /tmp/recovery.hcl
kms "aead" {
purpose = "recovery"
aead_type = "aes-gcm"
key = "8fZBjCUfN0TzjEGLQldGY4+iE9AkOvCfjh7+p0GtRBQ="
key_id = "global_recovery"
}
EOF
$ boundary users create <truncated> -recovery-config /tmp/recovery.hcl
...
```
</Tab>
<Tab heading="Terraform">
To configure your provider to use the recovery KMS workflow, provide the KMS block as the value for
`recovery_kms_hcl`:
```hcl
provider "boundary" {
addr = 'https://boundary.mycorp.com:9200'
recovery_kms_hcl = <<EOT
kms "aead" {
purpose = "recovery"
aead_type = "aes-gcm"
key = "8fZBjCUfN0TzjEGLQldGY4+iE9AkOvCfjh7+p0GtRBQ="
key_id = "global_recovery"
}
EOT
}
```
</Tab>
</Tabs>
## Initialize the Database
Before you can start Boundary, the database must be initialized. It's useful to look at the help output for the init command:
```
$ boundary database init -h
...
```
From this command, you can see the flags available to skip the creation of auto-generated resources.
To initialize the Boundary database without generated resources:
```
$ boundary database init -skip-initial-login-role-creation -config /etc/boundary.hcl
```
When you start Boundary, you will effectively have a blank sheet to work against. The initial migrations in the database have been run (note that this includes creating special users like `u_anon` and the `global` scope) and the internal keyrings have been initialized. From here, it's required that
you use the KMS recovery workflow described above to create at a minimum an auth method, a user, an account, and a
role with sufficient grants. Otherwise, you need to continue to use the recovery workflow for management. It's important
to realize that this is effectively a global super user type of workflow and comes with security concerns.
## Creating Your First Login Account
This section covers how to configure your first auth method, user, account, and role to login to Boundary without
the recovery KMS workflow. In this example, we're going to make an admin user for the global and project level
scopes we create. This will allow our user to configure targets within those scopes and manage them.
### Create Org and Project Scopes
In this example, we're going to create an org and project scope and skip creating an administrator and admin role
for each scope. We're going to specify a role for managing these scopes by selected users in a later step.
<Tabs>
<Tab heading="CLI">
```bash
$ boundary scopes create -name 'org' -scope-id 'global' \
-recovery-config /tmp/recovery.hcl \
-skip-admin-role-creation \
-skip-default-role-creation
<truncated>
$ boundary scopes create -name 'project' -scope-id <org_scope_id_from_last_step> \
-recovery-config /tmp/recovery.hcl \
-skip-admin-role-creation \
-skip-default-role-creation
<truncated>
```
</Tab>
<Tab heading="Terraform">
```hcl
resource "boundary_scope" "org" {
scope_id = "global"
name = "organization"
description = "Organization scope"
auto_create_admin_role = false
auto_create_default_role = false
}
resource "boundary_scope" "project" {
name = "project"
description = "My first project"
scope_id = boundary_scope.org.id
auto_create_admin_role = false
auto_create_default_role = false
}
```
</Tab>
</Tabs>
### Create an Auth Method
Create an auth method in the organization scope.
<Tabs>
<Tab heading="CLI">
```bash
$ boundary auth-methods create password \
-recovery-config /tmp/recovery.hcl \
-scope-id <org_scope_id> \
-name 'my_method' \
-description 'My password auth method'
```
</Tab>
<Tab heading="Terraform">
```hcl
resource "boundary_auth_method" "password" {
name = "my_password_auth_method"
description = "Password auth method"
type = "password"
scope_id = boundary_scope.org.id
}
```
</Tab>
</Tabs>
### Create a Login Account
Create a login account for the auth method.
<Tabs>
<Tab heading="CLI">
```bash
$ boundary accounts create password \
-recovery-config /tmp/recovery.hcl \
-login-name "myuser" \
-password "foofoofoo" \
-auth-method-id <auth_method_id_from_last_step>
```
</Tab>
<Tab heading="Terraform">
```hcl
resource "boundary_account" "myuser" {
name = "myuser"
description = "User account for my user"
type = "password"
login_name = "myuser"
password = "foofoofoo"
auth_method_id = boundary_auth_method.password.id
}
```
</Tab>
</Tabs>
### Create a User
Create a user and associate the user with the login account created in the previous step.
This user will also be the principal in the role we create in the following step.
<Tabs>
<Tab heading="CLI">
```bash
$ boundary users create -scope-id <org_scope_id> \
-recovery-config /tmp/recovery.hcl \
-name "myuser" \
-description "My user!"
$ boundary users add-accounts \
-recovery-config /tmp/recovery.hcl \
-id <myuser_user_id> \
-account <myuser_account_id>
```
</Tab>
<Tab heading="Terraform">
```hcl
resource "boundary_user" "myuser" {
name = "myuser"
description = "My user!"
account_ids = [boundary_account.myuser.id]
scope_id = boundary_scope.org.id
}
```
</Tab>
</Tabs>
### Create Roles to Manage Scopes
The following describes the four baseline roles you'll need to create to manage resources within the org and project
scopes created above. These roles are similar to the roles created for you if generation had not been skipped during `boundary database init` when executed with the `-skip-initial-login-role-creation` flag, Declaring roles explicitly
allows you to manage them independently and fully within Terraform or via the CLI. In doing so, you can precisely define their access.
The following example creates 4 roles:
1. To allow anonymous (unauthenticated) users the ability to list scopes and auth methods in the global scope.
1. To allow anonymous (unauthenticated) users the ability to list scopes and auth methods in the organization scope.
1. To allow `myuser` user administration grants at the org scope.
1. To allow `myuser` user administration grants at the project scope.
#### Anonymous listing role for global scope
<Tabs>
<Tab heading="CLI">
Assumes recovery key export from above steps is still set:
```bash
# Create global anonymous listing role
$ boundary roles create -name 'global_anon_listing' \
-recovery-config /tmp/recovery.hcl \
-scope-id 'global'
$ boundary roles add-grants -id <global_anon_listing_id> \
-recovery-config /tmp/recovery.hcl \
-grant 'id=*;type=auth-method;actions=list,authenticate' \
-grant 'id=*;type=scope;actions=list,no-op' \
-grant 'id={{account.id}};actions=read,change-password'
$ boundary roles add-principals -id <global_anon_listing_id> \
-recovery-config /tmp/recovery.hcl \
-principal 'u_anon'
```
</Tab>
<Tab heading="Terraform">
```hcl
resource "boundary_role" "global_anon_listing" {
scope_id = "global"
grant_strings = [
"id=*;type=auth-method;actions=list,authenticate",
"id=*;type=scope;actions=list,no-op",
"id={{account.id}};actions=read,change-password"
]
principal_ids = ["u_anon"]
}
```
</Tab>
</Tabs>
#### Anonymous listing role for org scope
<Tabs>
<Tab heading="CLI">
Assumes recovery key export from above steps is still set:
```bash
$ boundary roles create -name 'org_anon_listing' \
-recovery-config /tmp/recovery.hcl \
-scope-id <org_scope_id>
$ boundary roles add-grants -id <org_anon_listing_id> \
-recovery-config /tmp/recovery.hcl \
-grant 'id=*;type=auth-method;actions=list,authenticate' \
-grant 'type=scope;actions=list' \
-grant 'id={{account.id}};actions=read,change-password'
$ boundary roles add-principals -id <org_anon_listing_id> \
-recovery-config /tmp/recovery.hcl \
-principal 'u_anon'
```
</Tab>
<Tab heading="Terraform">
```hcl
resource "boundary_role" "org_anon_listing" {
scope_id = boundary_scope.org.id
grant_strings = [
"id=*;type=auth-method;actions=list,authenticate",
"type=scope;actions=list",
"id={{account.id}};actions=read,change-password"
]
principal_ids = ["u_anon"]
}
```
</Tab>
</Tabs>
#### Org admin role for myuser
<Tabs>
<Tab heading="CLI">
Assumes recovery key export from above steps is still set:
```bash
$ boundary roles create -name 'org_admin' \
-recovery-config /tmp/recovery.hcl \
-scope-id 'global' \
-grant-scope-id <org_scope_id>
$ boundary roles add-grants -id <org_admin_id> \
-recovery-config /tmp/recovery.hcl \
-grant 'id=*;type=*;actions=*'
$ boundary roles add-principals -id <org_admin_id> \
-recovery-config /tmp/recovery.hcl \
-principal <myuser_user_id>
```
</Tab>
<Tab heading="Terraform">
```hcl
resource "boundary_role" "org_admin" {
scope_id = "global"
grant_scope_id = boundary_scope.org.id
grant_strings = [
"id=*;type=*;actions=*"
]
principal_ids = [boundary_user.myuser.id]
}
```
</Tab>
</Tabs>
#### Project admin for myuser
<Tabs>
<Tab heading="CLI">
Assumes recovery key export from above steps is still set:
```bash
$ boundary roles create -name 'project_admin' \
-recovery-config /tmp/recovery.hcl \
-scope-id <org_scope_id> \
-grant-scope-id <project_scope_id>
$ boundary roles add-grants -id <project_admin_id> \
-recovery-config /tmp/recovery.hcl \
-grant 'id=*;type=*;actions=*'
$ boundary roles add-principals -id <project_admin_id> \
-recovery-config /tmp/recovery.hcl \
-principal <myuser_user_id>
```
</Tab>
<Tab heading="Terraform">
```hcl
resource "boundary_role" "project_admin" {
scope_id = boundary_scope.org.id
grant_scope_id = boundary_scope.project.id
grant_strings = [
"id=*;type=*;actions=*"
]
principal_ids = [boundary_user.myuser.id]
}
```
</Tab>
</Tabs>
### Login as Your New User
```bash
boundary authenticate password \
-auth-method-id <auth_method_id> \
-login-name myuser \
-password foofoofoo
```