Docs Changes for v0.8.0 (#2024)
* added new metrics doc * Minor format fixes * Use table to list metrics * Update website/content/docs/operations/index.mdx Co-authored-by: Robin Beck <stellarsquall@users.noreply.github.com> * fixed sentence * added worker proxy http * Update website/content/docs/operations/metrics.mdx Co-authored-by: Todd <github@quaddi.com> * Update based on the changes in draft * remove WIP note * Update website/content/docs/operations/metrics.mdx Co-authored-by: Todd <github@quaddi.com> * Update website/content/docs/operations/metrics.mdx Co-authored-by: Todd <github@quaddi.com> * Update website/content/docs/operations/index.mdx Co-authored-by: Todd <github@quaddi.com> * Removed invalid metrics * Update header level * added code styling for "ops" listener and port 9203 description * added health endpoint document * resolved cherry-pick confict for 5edac3a7418f80e7d528f2fa7a461b1125c11806 * Incorporate review feedback * removed bad reference to metrics * added Other section for boundary_build_info * fixed typo * Update website/content/docs/operations/health.mdx Co-authored-by: Loann Le <84412881+taoism4504@users.noreply.github.com> * Update website/content/docs/operations/metrics.mdx Co-authored-by: Loann Le <84412881+taoism4504@users.noreply.github.com> * Update website/content/docs/operations/health.mdx Co-authored-by: Loann Le <84412881+taoism4504@users.noreply.github.com> * Update website/content/docs/operations/health.mdx Co-authored-by: Loann Le <84412881+taoism4504@users.noreply.github.com> * Update website/content/docs/operations/health.mdx Co-authored-by: Loann Le <84412881+taoism4504@users.noreply.github.com> * Update website/content/docs/operations/health.mdx Co-authored-by: Loann Le <84412881+taoism4504@users.noreply.github.com> * Changed release notes structure Removed the empty "Releases" folder and now it goes straight to "Release Notes" * Update website/content/docs/operations/health.mdx Co-authored-by: Loann Le <84412881+taoism4504@users.noreply.github.com> * Incorporate review feedback * Update website/content/docs/operations/index.mdx Co-authored-by: Loann Le <84412881+taoism4504@users.noreply.github.com> * added tutorial link * Update website/content/docs/operations/metrics.mdx Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> * Update website/content/docs/operations/metrics.mdx Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> * Update website/content/docs/operations/metrics.mdx Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> * Update website/content/docs/operations/metrics.mdx Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> * Update website/content/docs/operations/metrics.mdx Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> * Update website/content/docs/operations/metrics.mdx Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> * 0.8 Release Notes Added release notes page for 0.8 * corrected sensitive info redaction note * cleaned up visibility overview Co-authored-by: taoism4504 <loann@hashicorp.com> Co-authored-by: Yoko Hyakuna <yoko@hashicorp.com> Co-authored-by: Loann Le <84412881+taoism4504@users.noreply.github.com> Co-authored-by: Todd <github@quaddi.com> Co-authored-by: Timothy Messier <tim.messier@gmail.com> Co-authored-by: XingLu Wang <xinglu.wang@hashicorp.com> Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
This commit is contained in:
17 files changed
+365
-64
No files matched your search
@@ -0,0 +1,100 @@
|
||||
---
|
||||
layout: docs
|
||||
page_title: Boundary Health Endpoints
|
||||
description: |-
|
||||
Verify the Boundary controller server is up and able to receive requests
|
||||
---
|
||||
|
||||
## Boundary Health Endpoints
|
||||
|
||||
Boundary provides health monitoring through the `/health` path using a listener
|
||||
with the `"ops"` purpose. By default, a listener with that purpose runs on port
|
||||
`9203`. See the [example configuration](#example-configuration) section for an
|
||||
example listener stanza in a `config.hcl` file.
|
||||
|
||||
### Requirements
|
||||
|
||||
To enable the controller health endpoint, any Boundary instance must be
|
||||
started with a controller. That is, a `controller` block and a `purpose = "api"`
|
||||
listener must be defined in Boundary's configuration file. Additionally, a
|
||||
`purpose = "ops"` listener must also be defined in Boundary's configuration
|
||||
file. Under these conditions, the `ops` server (which hosts the controller health
|
||||
api) will be exposed.
|
||||
|
||||
### Shutdown Grace Period
|
||||
|
||||
Optionally, when the controller health endpoint is enabled, Boundary can be
|
||||
configured to change the controller health response to `503 Service Unavailable`
|
||||
upon receiving a shutdown signal, and wait a configurable amount of time before
|
||||
starting the shutdown process.
|
||||
|
||||
This feature is designed to integrate with load balancers to reduce the risk of
|
||||
an outgoing Boundary instance causing disruption to incoming requests.
|
||||
|
||||
In this state, Boundary is still capable of processing requests as normal, but
|
||||
will report as unhealthy through the controller health endpoint. In
|
||||
load-balanced environments, this would cause this "unhealthy" instance to be
|
||||
removed from the pool of instances eligible to handle requests, and thereby, reducing the
|
||||
likelihood that it will receive a request to handle during shutdown.
|
||||
|
||||
This feature is disabled by default, even if the controller health endpoint is
|
||||
enabled. You can set it up by defining `graceful_shutdown_wait_duration` in the
|
||||
`controller` block of Boundary's configuration file. The value should be set to
|
||||
a string that is parseable by
|
||||
[ParseDuration](https://pkg.go.dev/time#ParseDuration).
|
||||
|
||||
### API
|
||||
|
||||
The new controller health service introduces a single read-only endpoint:
|
||||
|
||||
| Status | Description |
|
||||
|--------------|----------------------------------------------------------------|
|
||||
| `200` | `GET /health` returns HTTP status 200 OK if the controller's api gRPC Server is up |
|
||||
| `5xx` | `GET /health` returns HTTP status `5XX` or request timeout if unhealthy |
|
||||
| `503` | `GET /health` returns HTTP status `503 Service Unavailable` status if the controller is shutting down |
|
||||
|
||||
All responses return empty bodies. `GET /health` does not support any input.
|
||||
|
||||
## Example configuration
|
||||
|
||||
Health checks are available for a controller defined with a `purpose = "ops"`
|
||||
listener stanza. For details on what fields are allowed in this stanza, refer to
|
||||
the documentation about [TCP Listener](/docs/configuration/listener/tcp).
|
||||
|
||||
An example listener stanza:
|
||||
|
||||
```hcl
|
||||
controller {
|
||||
name = "boundary-controller"
|
||||
database {
|
||||
url = "postgresql://<username>:<password>@10.0.0.1:5432/<database_name>"
|
||||
}
|
||||
}
|
||||
|
||||
listener "tcp" {
|
||||
purpose = "api"
|
||||
tls_disable = true
|
||||
}
|
||||
|
||||
listener "tcp" {
|
||||
purpose = "ops"
|
||||
tls_disable = true
|
||||
}
|
||||
```
|
||||
|
||||
To enable a shutdown grace period, update the `controller` block with a defined
|
||||
wait duration:
|
||||
|
||||
```hcl
|
||||
controller {
|
||||
name = "boundary-controller"
|
||||
database {
|
||||
url = "env://BOUNDARY_PG_URL"
|
||||
}
|
||||
graceful_shutdown_wait_duration = "10s"
|
||||
}
|
||||
```
|
||||
|
||||
A complete example can be viewed under the [Controller
|
||||
configuration](/docs/configuration/controller#complete-configuration-example)
|
||||
docs.
|
||||
@@ -0,0 +1,15 @@
|
||||
---
|
||||
layout: docs
|
||||
page_title: Operating Boundary
|
||||
description: |-
|
||||
Operational tasks in Boundary.
|
||||
---
|
||||
|
||||
# Maintaining and Operating Boundary
|
||||
|
||||
Visibility into various components of Boundary plays an important role in ensuring the health of its controllers and workers. While additional monitoring components are planned for subsequent Boundary releases, metrics for controllers and workers can now be exported and then collected by Prometheus. Metrics provide insight into the current state of a Boundary deployment by enabling the diagnosis of deployed controllers and workers. Enabling metrics assists in the detection and mitigation of anomalies in a deployment.
|
||||
|
||||
These docs describe the available metrics and the naming and labeling standards used. Additionally, the health of controllers can be monitored using the `/health` endpoint. The health endpoint is enabled by defining a listener in Boundary's configuration file, and a health response may be configured to add a shutdown grace period process.
|
||||
|
||||
* Refer to the [Metrics](/docs/operations/metrics) documentation to learn more about Boundary metrics.
|
||||
* Refer to the [Health Endpoint](/docs/operations/health) documentation to learn more about Boundary health endpoints.
|
||||
@@ -0,0 +1,101 @@
|
||||
---
|
||||
layout: docs
|
||||
page_title: Boundary Metrics
|
||||
description: |-
|
||||
Obtain visibility of various components of a running Boundary
|
||||
---
|
||||
|
||||
## Boundary Metrics
|
||||
|
||||
Boundary provides metrics through the `/metrics` path using a listener with the
|
||||
`"ops"` purpose. By default, a listener with that purpose runs on port `9203`.
|
||||
See the [example configuration](#example-configuration) section for an example
|
||||
listener stanza in a `config.hcl` file.
|
||||
|
||||
Metrics are important for understanding what is happening in a Boundary
|
||||
deployment, enabling diagnoses of deployed controllers and workers. The exported
|
||||
metrics grant greater visibility into the various components of a running
|
||||
Boundary deployment through Prometheus.
|
||||
|
||||
The metrics uses the [OpenMetric
|
||||
exposition](https://prometheus.io/docs/instrumenting/exposition_formats/#openmetrics-text-format)
|
||||
format which can be read by
|
||||
[Prometheus](https://prometheus.io/docs/introduction/overview/).
|
||||
|
||||
### Controller
|
||||
|
||||
The following table lists the current names and descriptions of metrics provided
|
||||
by controllers.
|
||||
|
||||
| Name | Description |
|
||||
|---------------------------------------------------------------|-----------------------------------------------|
|
||||
| `boundary_controller_api_http_request_duration_seconds` | Histogram of latencies for HTTP requests. |
|
||||
| `boundary_controller_api_http_request_size_bytes` | Histogram of request sizes for HTTP requests. |
|
||||
| `boundary_controller_api_http_response_size_bytes` | Histogram of response sizes for HTTP requests. |
|
||||
| `boundary_controller_cluster_grpc_request_duration_seconds` | Histogram of latencies for requests made to the gRPC service running on the cluster listener. |
|
||||
|
||||
### Worker
|
||||
|
||||
The following table lists the current names and descriptions of metrics provided
|
||||
by workers.
|
||||
|
||||
| Name | Description |
|
||||
|---------------------------------------------------------------|-----------------------------------------------|
|
||||
| `boundary_worker_proxy_http_write_header_duration_seconds` | Histogram of time elapsed after the TLS connection is established to when the first http header is written back from the server. |
|
||||
| `boundary_worker_proxy_websocket_active_connections` | A gauge of the current count of open proxy connections on the worker. |
|
||||
| `boundary_worker_proxy_websocket_received_bytes_total` | Count of received bytes sent over all proxy connections handled by the worker. |
|
||||
| `boundary_worker_proxy_websocket_sent_bytes_total` | Count of sent bytes sent over all proxy connections handled by the worker. |
|
||||
|
||||
### Other
|
||||
|
||||
The following table lists the current miscellaneous metrics provided by Boundary.
|
||||
|
||||
| Name | Description |
|
||||
|---------------------------------------------------------------|-----------------------------------------------|
|
||||
| `boundary_build_info` | A gauge set to 1 with label values detailing the current Go version, git revision, and Boundary version. |
|
||||
|
||||
### Metric Labels
|
||||
|
||||
Each metric uses labels to help organize and narrow its scope so that users can
|
||||
filter for different characteristics, allowing more flexible analysis of the
|
||||
measurements collected.
|
||||
|
||||
#### Metrics for HTTP requests include the following labels:
|
||||
|
||||
| Label | Description |
|
||||
|--------------|----------------------------------------------------------------|
|
||||
| `code` | The HTTP status code of the request (e.g., “200”, “400”, etc.). |
|
||||
| `path` | The path, excluding query parameters, associated with the request. The value for this label replaces resource IDs with "{id}" (for example, `/v1/groups/{id}` is a possible value for this label). Any unexpected paths receive the value "unknown." |
|
||||
| `method` | The name of the method on the HTTP request. For example "GET", “POST”, “DELETE”. |
|
||||
|
||||
|
||||
#### Metrics for gRPC requests include the following labels:
|
||||
|
||||
| Label | Description |
|
||||
|-----------------|-----------------------------------------------------------------|
|
||||
| `grpc_method` | The name of the method on the grpc service (e.g., `GetGroup`). |
|
||||
| `grpc_service` | The proto service name including the package (e.g., `controller.api.services.v1.GroupService`). |
|
||||
| `grpc_code` | The grpc [status code](https://github.com/grpc/grpc-go/blob/master/codes/codes.go) in human-readable format. For example, `OK`, `IllegalArgument`, `Unknown`. |
|
||||
|
||||
|
||||
## Example configuration
|
||||
|
||||
Defining a listener stanza in the config file is sufficient for enabling metrics
|
||||
reporting in Boundary. For details on what fields are allowed in this stanza,
|
||||
refer to the documentation about [TCP
|
||||
Listener](/docs/configuration/listener/tcp).
|
||||
|
||||
An example listener stanza:
|
||||
|
||||
```hcl
|
||||
listener "tcp" {
|
||||
purpose = "ops"
|
||||
tls_disable = true
|
||||
}
|
||||
```
|
||||
|
||||
## Tutorial
|
||||
|
||||
Refer to [Prometheus
|
||||
Metrics](https://learn.hashicorp.com/tutorials/boundary/prometheus-metrics)
|
||||
tutorial for a step-by-step introduction to metrics.
|
||||
Reference in new issue
Block a user