CyberCoveLLC/threshold
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Worker Connectivity Health Check (#3650)

Browse Source 
* feat: Worker Connectivity Health Check

Add worker upstream connection state. API calls to the worker's ops `/heath?worker_info=1` endpoint should now return a new filed called `upstream_connection_state`

```
{
   "worker_process_info":{
      "state":"active",
      "active_session_count":0,
      "upstream_connection_state":"READY"
   }
}
```

The states returned will be GRPC connection states: https://grpc.github.io/grpc/core/md_doc_connectivity-semantics-and-api.html

There could be multiple upstream addresses but GRPC manual resolver determines the state of the connection.
This commit is contained in:
Elim Tsiagbey authored and GitHub committed 2023-09-01 13:42:41 -04:00
1 parent c55df52d44
commit fabd4093de
8 files changed
+272 -59
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

internal/cmd/ops/server_test.go
+7 -4
View File
@@ -38,6 +38,7 @@ import (
"github.com/mitchellh/cli"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"google.golang.org/grpc/connectivity"
"google.golang.org/protobuf/testing/protocmp"
"google.golang.org/protobuf/types/known/wrapperspb"
)
@@ -723,8 +724,9 @@ func TestCreateOpsHandler(t *testing.T) {
pbResp := &pbs.GetHealthResponse{}
require.NoError(t, jsonpb.Unmarshal(rsp.Body, pbResp))
want := &pbs.GetHealthResponse{WorkerProcessInfo: &pbhealth.HealthInfo{
State: server.ActiveOperationalState.String(),
ActiveSessionCount: wrapperspb.UInt32(0),
State: server.ActiveOperationalState.String(),
ActiveSessionCount: wrapperspb.UInt32(0),
UpstreamConnectionState: connectivity.TransientFailure.String(),
}}
assert.Empty(t, cmp.Diff(want, pbResp, protocmp.Transform()))
},
@@ -747,8 +749,9 @@ func TestCreateOpsHandler(t *testing.T) {
pbResp := &pbs.GetHealthResponse{}
require.NoError(t, jsonpb.Unmarshal(rsp.Body, pbResp))
want := &pbs.GetHealthResponse{WorkerProcessInfo: &pbhealth.HealthInfo{
State: server.ActiveOperationalState.String(),
ActiveSessionCount: wrapperspb.UInt32(0),
State: server.ActiveOperationalState.String(),
ActiveSessionCount: wrapperspb.UInt32(0),
UpstreamConnectionState: connectivity.TransientFailure.String(),
}}
assert.Empty(t, cmp.Diff(want, pbResp, protocmp.Transform()))
},