Commit Graph
1879 Commits
Author SHA1 Message Date
Timothy Messier 760b090567 feat(proto): Add new option for proto files to set domain
This allows for associating all of the messages in a file with a domain.
This will be used by the global subtype registry to group subtypes
within a domain. When registering a subtype, it must be unique within a
domain. This allows for cases where two domains have the same subtypes,
such as two domains both having a "plugin" subtype.
2022-04-20 13:32:31 -04:00
Johan Brandhorst-Satzkorn 6e6efeee71 feat(account): Add gotags classification
Add gotag classification to all account proto and service definitions.
2022-04-20 13:32:31 -04:00
Johan Brandhorst-Satzkorn ac5a1acca8 feat(account): Hide internal attribute fields
These fields are not intended for use by API consumers, so we hide
them from the generated API documentation by marking them as INTERNAL.
2022-04-20 13:32:31 -04:00
irenarindos eb43941771 refact(account): Use OneOf for Account attributes 2022-04-20 13:32:31 -04:00
irenarindos e76db2cd13 feat(proto): create custom option for attribute subtype 2022-04-20 13:32:31 -04:00
Johan Brandhorst-Satzkorn 6a37bd9710 Upgrade grpc-gateway version to v2.10.0 (#1954)
This new version will let us hide fields from the OpenAPI output
2022-04-20 13:32:29 -04:00
Zed 0c8dd578e6 Bump UI version for 0.8.0 release (#2029) 2022-04-19 12:00:33 -04:00
Jim addbfee593 chore: upgrade gofumpt to v0.3.1 (#2028) 2022-04-19 11:42:54 -04:00
Timothy Messier ec0f107ef5 fix(build): Run plugin script directly instead of with sh (#2025)
These github actions run on Ubuntu based machine. Ubuntu by default
links `/bin/sh` to dash. So by invoking the plugin.sh script as `/bin/sh
./plugins.sh` the script would run under `dash` despite the shebang
line. A recent change that unified some of the build process, introduced
additional bashisms into this script which is caused the build to fail.

This updated the build system to just run the script directly, which
will honor the shebang line. This seems like the best approach since the
ubuntu machine has bash and the shebang line points to bash, so it is
natural for developers to assume bashisms will work.

See:
    f370abc1da
    https://github.com/actions/virtual-environments/blob/ubuntu20/20220410.2/images/linux/Ubuntu2004-Readme.md
    https://wiki.ubuntu.com/DashAsBinSh
2022-04-19 10:00:41 -04:00
Timothy Messier b5080c30c1 fix(config): Prevent panic when setting audit config filter orverrides (#2021)
This updates the config parsing for this stanza so prevent panics like
the following:

    panic: reflect.Value.MapIndex: value of type string is not assignable to type event.DataClassification

    goroutine 1 [running]:
    reflect.Value.assignTo({0x13fbb20, 0xc000a28700, 0x461367}, {0x1693f9d, 0x16}, 0x14375c0, 0x0)
            /usr/lib/go/src/reflect/value.go:2810 +0x2ad
    reflect.Value.MapIndex({0x145b1c0, 0xc000a16570, 0xc00072ff20}, {0x13fbb20, 0xc000a28700, 0x98})
            /usr/lib/go/src/reflect/value.go:1535 +0x7e
    github.com/hashicorp/hcl.(*decoder).decodeMap(0x8, {0xc00012b0b0, 0x28}, {0x67b40a8, 0xc0005fdc80}, {0x14bac80, 0xc0005fdc20, 0x1})
            /go/pkg/mod/github.com/hashicorp/hcl@v1.0.0/decoder.go:380 +0xa95
    github.com/hashicorp/hcl.(*decoder).decode(0xc000731360, {0xc00012b0b0, 0x28}, {0x67b40a8, 0xc0005fdc80}, {0x14bac80, 0xc0005fdc20, 0x0})
            /go/pkg/mod/github.com/hashicorp/hcl@v1.0.0/decoder.go:100 +0x395
    github.com/hashicorp/hcl.(*decoder).decodeStruct(0x203000, {0xc0005bf500, 0x11}, {0x67b40a8, 0xc0005fdc08}, {0x14fada0, 0xc0005fdc20, 0x7296ce0})
            /go/pkg/mod/github.com/hashicorp/hcl@v1.0.0/decoder.go:702 +0x1349
    github.com/hashicorp/hcl.(*decoder).decode(0xc000731360, {0xc0005bf500, 0x11}, {0x67b40a8, 0xc0005fdc08}, {0x14fada0, 0xc0005fdc20, 0x0})
            /go/pkg/mod/github.com/hashicorp/hcl@v1.0.0/decoder.go:108 +0x62f
    github.com/hashicorp/hcl.(*decoder).decodePtr(0x8, {0xc0005bf500, 0x11}, {0x67b40a8, 0xc0005fdc08}, {0x1442220, 0xc0005ca2f0, 0x1})
            /go/pkg/mod/github.com/hashicorp/hcl@v1.0.0/decoder.go:405 +0xd3
    github.com/hashicorp/hcl.(*decoder).decode(0xc000731360, {0xc0005bf500, 0x11}, {0x67b40a8, 0xc0005fdc08}, {0x1442220, 0xc0005ca2f0, 0x12})
            /go/pkg/mod/github.com/hashicorp/hcl@v1.0.0/decoder.go:102 +0x3e8
    github.com/hashicorp/hcl.(*decoder).decodeStruct(0x7f8c99b698f0, {0x1676310, 0x4}, {0x67b40d0, 0xc0005c82a0}, {0x1621c80, 0xc0005ca240, 0x0})
            /go/pkg/mod/github.com/hashicorp/hcl@v1.0.0/decoder.go:702 +0x1349
    github.com/hashicorp/hcl.(*decoder).decode(0xc000731360, {0x1676310, 0x4}, {0x67b40d0, 0xc0005c82a0}, {0x1621c80, 0xc0005ca240, 0xc0005ca240})
            /go/pkg/mod/github.com/hashicorp/hcl@v1.0.0/decoder.go:108 +0x62f
    github.com/hashicorp/hcl.DecodeObject({0x1442400, 0xc0005ca240}, {0x67b40d0, 0xc0005c82a0})
            /go/pkg/mod/github.com/hashicorp/hcl@v1.0.0/decoder.go:60 +0x126
    github.com/hashicorp/boundary/internal/cmd/config.parseEventing(0xc0005fa2a0)
            /go/src/github.com/hashicorp/boundary/internal/cmd/config/config.go:622 +0x1ec
    github.com/hashicorp/boundary/internal/cmd/config.Parse({0xc000683100, 0x66c})
            /go/src/github.com/hashicorp/boundary/internal/cmd/config/config.go:553 +0x149b
    github.com/hashicorp/boundary/internal/cmd/config.LoadFile({0x7fff4c8fdeef, 0xc0008e2d40}, {0x0, 0x0})
            /go/src/github.com/hashicorp/boundary/internal/cmd/config/config.go:306 +0x6f
    github.com/hashicorp/boundary/internal/cmd/commands/database.(*InitCommand).ParseFlagsAndConfig(0xc0005ae000, {0xc000132030, 0x2, 0x2})
            /go/src/github.com/hashicorp/boundary/internal/cmd/commands/database/init.go:516 +0x605
    github.com/hashicorp/boundary/internal/cmd/commands/database.(*InitCommand).Run(0xc0005ae000, {0xc000132030, 0xc0002857a0, 0xc000285788})
            /go/src/github.com/hashicorp/boundary/internal/cmd/commands/database/init.go:175 +0x4a
    github.com/mitchellh/cli.(*CLI).Run(0xc0002b2c80)
            /go/pkg/mod/github.com/mitchellh/cli@v1.1.2/cli.go:262 +0x5f8
    github.com/hashicorp/boundary/internal/cmd.RunCustom({0xc000132010, 0x60, 0x0}, 0x0)
            /go/src/github.com/hashicorp/boundary/internal/cmd/main.go:186 +0x9d6
    github.com/hashicorp/boundary/internal/cmd.Run(...)
            /go/src/github.com/hashicorp/boundary/internal/cmd/main.go:92
    main.main()
            /go/src/github.com/hashicorp/boundary/cmd/boundary/main.go:13 +0xc9
2022-04-18 11:03:10 -04:00
Johan Brandhorst-Satzkorn 80b2c9230f refact(authmethods): Add top level token type to Authenticate (#2010)
Other parts of the system make use of a "type" field, while the
AuthenticateResponse sets a "token_type" on the response attributes
directly. Rename this field to "type", and move it up to the response
level, to be more consistent with other RPCs.

Also adds a new field to the AuthenticateRequest, "type" which will
replace the "token_type" field over time.
2022-04-18 09:26:15 -04:00
Jeff Mitchell bf9e7874b1 Use custom X.509 verification logic for proxy connections (#2008)
This reduces the chance that we'll hit a failure due to a mismatched
worker address, on the certificate, especially if a worker has multiple
addresses. We aren't relying on IP/DNS for security anyways, so let's
not have it be a hindrance.
2022-04-16 12:09:22 -04:00
Todd 631de192d3 Initialize the build info metrics using init to always register to the DefaultRegistery. (#2020)
This fixes errors related to calling NewServer or NewCommand multiple times when using autocomplete.
2022-04-15 15:48:59 -07:00
Jeff Mitchell 3d9e541930 Switch request.WithContext to request.Clone (#2017)
The Go docs now say to use request.Clone (introduced in Go
1.13) instead of request.WithContext (introduced in Go 1.7) as it deep
copies instead of shallow copies.
2022-04-15 13:28:17 -04:00
Todd 0b12feb953 Do not register the build_info collectors multiple times. (#2014)
Since we call base.NewServer multiple times when building the Commands map this was returning
an error "duplicate metrics collector registration attempted".  This PR only creates a single
server which is used for both the "server" command as well as "dev".
2022-04-15 09:30:23 -07:00
Jim 4636efce8f refactor (tests): GetEventDetails(...) will stop returning nil (#2019) 2022-04-15 12:20:44 -04:00
Jim cb93c6bdfd chore: update auditResponseInterceptor const op (#2018) 2022-04-15 12:04:20 -04:00
Jeff Mitchell f370abc1da Unify plugin building (#2013)
This causes the build.sh script to chain to plugins.sh which makes
plugin building behave the same for both local and release builds.
2022-04-15 09:17:43 -04:00
Cameron Perera f9ce13c779 chore: Add redirect for targets worker filtering (#2009) 2022-04-14 12:55:34 -05:00
Hugo cc54a834e8 feat(worker): Instrumentation for Controller Cluster Communication (#1998)
* feat(worker): Instrumentation for Controller Cluster Communication

Adds request latency observations to the worker side (client) of
worker <-> controller cluster communications by integrating a new
grpc client interceptor.

* style(controller): make-gen-deltas changes
2022-04-13 15:52:54 -07:00
Todd 36b7f3e1f8 Update the cluster server metrics to use the gRPC status.Handler. (#2000)
gRPC status.Handler gives us visibility into gRPC returned errors. In contrast, there are some situations where no metric would be counted at all using the interceptor approach (for example, in out of memory cases where gRPC would return a resource exhausted error).
2022-04-13 10:27:26 -07:00
Haotian e3f72bd0a4 feat(metrics): add build info metric on server start (#1984)
Adds a Prometheus gauge metric set to the value of 1, with labels for go version, git revision, and Boundary version. 
Assigns the Prometheus DefaultRegisterer to the Server struct as a parameter to NewServer(), to avoid "duplicate metrics collector registration" panics from test cases.
2022-04-12 17:04:11 -07:00
Johan Brandhorst-Satzkorn 8d80d10ff7 fix(controller): Cancel tests on fatal errors (#1994)
When requesting a token, any error is fatal to the test.
The previous behavior would return a nil token, which
could cause subsequent uses to panic. This will instead
interrupt any test immediately on error.
2022-04-12 16:56:04 -04:00
Louis Ruch 9b48dbc2fd fix(logs): Remove erroneous scheduler run logs at first run. (#1980)
The Scheduler is dependant on the server being inserted into the
database. Adding an upsert server call before starting scheduler
and tickers removes the race that caused log spam due to a missing
foreign key relationship.
2022-04-12 08:17:14 -07:00
Jim 8895bc08e0 fix (kms): properly reconcile DEKs for existing scopes (#1976) 2022-04-12 09:20:52 -04:00
Bryce Kalow 7f69e935bf website: website source code / content split (#1838) 2022-04-11 14:46:36 -05:00
Hugo 06c13064cc Merge pull request #1963 from hashicorp/hugoamvieira-proxyws-metrics
feat(worker): Implement proxy handler websocket instrumentation
2022-04-11 14:53:52 +01:00
Hugo Vieira 3564776517 feat(worker): Implement proxy handler websocket instrumentation
Add metrics on read/written bytes and active worker websocket
connections.
2022-04-11 14:34:40 +01:00
Louis Ruch e6b7b7f60a feat(db): Suppress not found events during db lookups (#1979) 2022-04-08 18:04:12 -07:00
Jeanne Angeles Franco 2c10159086 Remove quality-tests event (#1989) 2022-04-08 15:23:58 -07:00
Johan Brandhorst-Satzkorn f2fa1142d3 fix(test): Remove uses of t.FailNow in goroutines (#1978)
It is documented as illegal to use t.FailNow in a goroutine.
Uses of require assertions call t.FailNow. This may have led to
inconsistent test results as some assertions were subject to
undefined behavior. Note that it is still allowed to call
assert assertions in goroutines.

See: https://pkg.go.dev/testing#T.FailNow
2022-04-07 09:26:34 -04:00
Jeff Mitchell 1be55b50c7 Pull up Server to top level of Init command (#1959)
* Pull up Server to top level of Init command

Init was using a local server variable but not passing in the existing
command, resulting in the context being lost. This caused the command to
fail to run since the changes to add logging to KMS plugins were
introduced.

This also fixes a couple of places that were not using the global const
value for config KMS.

* Fix test compilation
2022-04-06 16:40:47 -04:00
Jim 0ca86c8a2c chore: remove unneeded ref to gorm.DB (#1971) 2022-04-06 09:17:17 -04:00
Todd fb068341a4 Update naming for initializing collectors and tweak some comments. (#1968) 2022-04-04 11:38:15 -07:00
Johan Brandhorst-Satzkorn b1d6a1da9a Switch to buf for protobuf generation (#1944)
* Install buf with go install

This reduces the impact of using buf on our dependency
closure, which was significant. It also avoids potential
issues with building buf given our own internal dependency
constraints. This has caused issues (including crashes!)
in the past.

See https://github.com/cockroachdb/cockroach/issues/67216#issuecomment-874176186
for more discussion on using tool dependencies generally, and
https://github.com/bufbuild/buf/issues/52
for an example of where this sort of thing caused an issue for a user.

* Upgrade buf version

We should migrate to v1 sooner rather than later, as it has some
breaking changes from v0.

* Change buf lint/breaking commands

These commands were moved up one level for v1

* Migrate to buf.yaml to v1

* Remove unused imports

These were uncovered by using "buf lint".

* Use buf v1.3.1

* Depend on googleapis and grpc-gateway via BSR

* Switch to buf for proto generation

* Remove third party dependency

* Remove unused tools and scripts

* Move local protos up one level

* Temporarily don't fail on breaking changes

* Generate CI config

* Fix generation of controller.json

* Replace use of deprecated flag

* Remove unused import, regenerate files

* Remove temporary skipping over breaking check
2022-04-04 13:55:42 -04:00
Jeff Mitchell 58a448fc6a Put session ID in ALPN (#1966)
This allows us to use SNI for actual host (e.g. routing) information. We
don't have the client cert yet so we can't look at that, so this
provides a way for us to convey the information needed to look up that
session's TLS stack.

Using SNI for hosts means we also run into the fact that we don't have
automatic agreement in terms of SANs. So when generating the certs we
now also pass worker address information to the function to be encoded
in the cert.

Finally, there is a change in how the websocket dialing happens, because
http.RoundTripper tries to be too clever for its own good and overwrites
NextProtos on a whim.
2022-04-01 16:32:20 -04:00
Haotian 62f0d18885 Initialize and provide handler for proxy worker http metrics (#1961)
*feat(metrics): implement prometheus handler for proxy worker http requests and record metric for header write duration
2022-04-01 12:07:40 -07:00
Todd 20e6c0ce6c Add collector updating interceptor to the controller cluster grpc server (#1964) 2022-04-01 10:52:36 -07:00
Jeff Mitchell 07dc3db974 Remove ALPN Muxer (#1965)
We never ended up using this and it makes it much harder to reason about the listeners.

An explanation: for the moment I separated out all of the listeners in the struct, even though one should suffice, because when I did it that way first I was hitting some really weird and hard to find intermittent behavior where it seemed like the listeners were getting crossed. So I re-did it this way with each listener very explicitly referenced. I'm happy to make a follow-up PR that attempts to combine them all into a single listener without hitting the issues I did, if desired, but at least doing it via a separate PR means that it won't be something with removing the ALPN muxer that is at fault.
2022-04-01 12:06:41 -04:00
PPacent ce7292d9c8 Docs: Adding service discovery concept page (#1886)
* Adding service discovery concept page
2022-03-31 14:54:53 -07:00
A-440Hz 95e1fe8df0 typo(metrics) changing response size to request size 2022-03-31 14:54:25 -07:00
ToddandHugo Vieira 3eb22b073a Defines the controller api metrics and registers them at controller startup (#1917)
Co-authored-by: Hugo Vieira <hugoamvieira@users.noreply.github.com>
2022-03-31 08:48:50 -07:00
Hugo 1bb2c5983d Merge pull request #1910 from hashicorp/hugoamvieira-worker-start-listeners
refactor(worker): Startup, shutdown and cluster tests changes
2022-03-31 16:28:55 +01:00
Joey Freeland 15a6ffeb6b chore: clarification on default TCP ports (#1878) 2022-03-30 09:36:30 -07:00
Lili Cosic d2602584ab docs: Add full example for using recovery KMS workflow (#1888) 2022-03-29 18:18:33 -07:00
Johan Brandhorst-Satzkorn 121a296a7f Refactor file formatting target (#1951)
Use the standard Go heuristic for determining
whether a file should be ignored by formatting
for being a generated file.

This is effectively a no-op change today, but
if we add new generated files in the future,
we should automatically ignore them in formatting.

This is a slightly updated version of #1941 that should
work across all versions of grep.
2022-03-29 15:37:25 -04:00
Hugo Vieira aae944be37 test(worker): Remove test-only hooks from actual code
This change introduces a `nonceFn` field that can be changed
by a test but at runtime is transparent to the code that calls it.

The current nonce-generating logic defaults to a test-only field which
opens us up to this value inadvertently being set.
With this change, the default behavior becomes using `base62` to
generate a random nonce, and any tests have to explicitly change that.
2022-03-29 16:46:02 +01:00
Hugo Vieira 61e310998f refactor(worker): Decouple listeners/servers shutdown
This change is the Worker-side counterpart to the change in
eebdfb88dc.
2022-03-29 16:46:02 +01:00
Hugo Vieira 657b099440 refactor(worker): Ensure startListeners consistency with controller
Implements changes on Worker's `startListeners` func to be consistent
with the changes on the Controller's side done in
b5ef468d33.
2022-03-29 16:45:56 +01:00
Hugo Vieira 1ebd7deccf refactor(worker): Validate listener configuration in New
With this change, the Worker is now aware of its own listeners in a
more codified and structured way.

These changes have already been made in the controller on
8d48a8515c. They're now made on the
worker for similar reasons.
2022-03-29 16:44:05 +01:00