Commit Graph
101 Commits
Author SHA1 Message Date
Johan Brandhorst-Satzkorn e0700570a2 bsr: add endpoint to session meta
The endpoint stores the address of the host
the egress worker proxies to.
2023-05-31 12:18:24 +00:00
Timothy Messier 9f2c83f3b0 refact(bsr): Rename const to denote when interpolation is needed 2023-05-31 12:18:23 +00:00
Timothy Messier b88b4cce75 feat(bsr/ssh): Add file transfer direction to channel summary
When an SSH channel is used for a file transfer, the summary will report
the detected direction of the file transfer as either an upload or
download, relative to the end client.
2023-05-31 12:18:23 +00:00
Timothy Messier 78ea1a9f3a refact(bsr): Use JSON for session meta and summary data
The session metadata is provided all at once when a new bsr is opened.
While the session recording metadata records metadata about a recording
while the recording is happening, notably it records each time a new
connection occurs.

The nested structure of the session metadata does not lend itself to the
append-only syntax of the recording meta file. In addition we do not
want to mix different kinds of metadata together. Thus the session
metadata is now written to a separate file in the BSR, and is written as
JSON which better handles the nested structure and allows for simpler
encoding/decoding.

Similarly the summary files are intended to be written to as containers
close and have been switched to JSON.
2023-05-31 12:18:23 +00:00
irenarindos b57692f8fc chore(bsr): check index bounds for meta unmarshal 2023-05-31 12:18:22 +00:00
Timothy Messier b9de6cf00b fixup:(bsr): Correctly reference subcontainers when opening
This is a fixup for 60785b5a8
2023-05-31 12:18:21 +00:00
Timothy Messier 738a6c6472 feat(bsr/convert): Implement ToAsciicast
This adds support for converting a channel in a SSH BSR to an asciicast.
2023-05-31 12:18:21 +00:00
Timothy Messier 9227fcd879 feat(bsr/convert): Add logic to convert ssh channel to asciicast 2023-05-31 12:18:21 +00:00
Timothy Messier ed6882a457 feat(bsr/convert): Add asciicast package
This provides a new internal package for constructing and serializing an
asciicast in the v2 asciicast file fromat.

See: https://github.com/asciinema/asciinema/blob/develop/doc/asciicast-v2.md
2023-05-31 12:18:21 +00:00
Timothy Messier 916a919bf0 feat(bsr): Remove ascii format of protocol
This was originally introduced to ensure that the meta files only
included ascii characters. However, this makes reading the protocol from
meta file less reliable since it does not exactly match the protocol
string. Instead protocol strings will be recorded in the meta files as
is. Generally protocol strings should include readable characters.

This also updates the ssh.Protocol to be 4 readable characters.
2023-05-31 12:18:21 +00:00
Timothy Messier 4112c4e8fe feat(bsr): Add methods for opening ChunkScanners for data files 2023-05-31 12:18:21 +00:00
Timothy Messier e37fa39c95 feat(bsr/fstest): Add localfs implementation
This adds another test storage.FS implemenation. This can be used to
write/read containers and files from a local filesystem via calls to the
os pacakge.
2023-05-31 12:18:21 +00:00
Timothy Messier 713d961152 feat(bsr/fstest): Add test TempFile implementation 2023-05-31 12:18:21 +00:00
Timothy Messier b4ce802572 refact(bsr/convert): Use new storage.TempFile interface
This changes the ToAscinema function to take a storage.TempFile instead
of a storage.FS. It was noticed that when this was being used a
storage.FS was being created just prior to calling this just so that it
could then create a temporary file. It seemed to make more sense to just
pass the temporary file in, instead of needing to expand the storage.FS
interface.

This also renames this method to align with the file format that the
returned io.Reader will contain.
2023-05-31 12:18:21 +00:00
irenarindos 6705f6128a feat(bsr):unmarshal updated meta 2023-05-31 12:18:20 +00:00
irenarindos 304d15021f chore(bsr): remove references to ssh protocol 2023-05-31 12:18:20 +00:00
irenarindos e567bb97e6 feat(bsr): instantiate BSR session for playback 2023-05-31 12:18:20 +00:00
Timothy Messier 46e1752283 feat(bsr/fstest): Support read mode
This updates the fstest implementation of a storage.FS to support
opening sub containers and files in read-only mode.

Now when calling `SubContainer`:

- If using `storage.WithCreateFile`, an error is returned if the
  container already exists.
- If not using `storage.WithCreateFile` and error is returned if the
  container does *not* exist.

When calling `OpenFile`:

- on a container that was created with `storage.ReadOnly`, an error is
  returned if attempting to open the file wth `storage.ReadWrite`.
- with `storage.WithCreateFile`, an existing file is replaced with a new
  empty file, (ie "truncated").
- without `storage.WithCreateFile`, an error is returned if the file
  does not exist.
2023-05-31 12:18:20 +00:00
Timothy Messier e9c7e373e9 feat(bsr/fstest): Implement Open method 2023-05-31 12:18:20 +00:00
irenarindos d06c9b2933 feat(bsr): populate container meta from meta files 2023-05-31 12:18:20 +00:00
irenarindos 4e5401da7d bsr(chunk decode): fix intermittent error message inconsistency 2023-05-31 12:18:20 +00:00
Timothy Messier 5d0a9b706e feat(bsr): Add ChunkScanner
The scanner can be used to read any BSR data file. When creating a new
scanner, it will first verify the BSR magic string. Individual Chunks
can then be read back by calling Scan. The scanner will compute a
sha256sum while it is reading chunks. If the scanner was created using
the WithSha256Sum option, it will compare the computed checksum with the
one provided via the option once the scanner encounters an end chunk
(DONE) or an io.EOF error from the reader. If the checksums do not
match, and error will be returned from Scan. The current sha256sum can
be returned from the scanner using the Sum method.

There is also a convenience function, ChunkWalk that can be used to call
a function on each Chunk returned by a scanner.
2023-05-31 12:18:20 +00:00
Timothy Messier f983a05a3e feat(bsr/ssh): Support decoding chunks
This adds a function for decoding all of the SSH Chunk types and
registers the function with the chunk types so that the bsr.ChunkDecode
can be used to decode SSH Chunks.
2023-05-31 12:18:20 +00:00
Timothy Messier fe870fc989 feat(bsr): Add ChunkDecoder
A ChunkDecoder can be used to BSR Chunks. Each call to Decode will read
the next chunk from the decoder's io.Reader. It handles decompressing
any compressed chunk data, and verifying the CRCs of each chunk.
2023-05-31 12:18:20 +00:00
Timothy Messier 648842f031 feat(bsr/checksum): Add function to load SHA256SUM
This provides a Sha256Sums type to store the information from a
SHA256SUM file in memory for easy lookup of the sums of files. It can be
used when checking the checksums of individual files without needing to
go back to disk.
2023-05-31 12:18:19 +00:00
Johan Brandhorst-Satzkorn 74d10da7b4 bsr: remove duplicate target scope info
We were storing the project ID both as part of the scope
and as a field directly on the target. Remove the extra field.
2023-05-31 12:18:19 +00:00
irenarindos 92df5cf8c7 feat(bsr): add worker information to session meta 2023-05-31 12:18:19 +00:00
irenarindos 200e7dca5a feat(bsr): add credentials to session meta 2023-05-31 12:18:19 +00:00
irenarindos 2ba780703f feat(bsr): add summary unavailable error 2023-05-31 12:18:17 +00:00
Jim 46348d7b49 fix (bsr/kms): use const time compare for sig 2023-05-31 12:18:16 +00:00
irenarindos 3e9a4bf03e feat(bsr): add VerifySignatureWithPubKey to kms methods 2023-05-31 12:18:16 +00:00
Timothy Messier 29c739e8b8 feat(bsr): Add function for generating a channel id 2023-05-31 12:18:16 +00:00
irenarindos 718536783c feat(bsr): expand session metadata 2023-05-31 12:18:16 +00:00
irenarindos 93afd24327 feat(bsr): add channel request types to chan callbacks 2023-05-31 12:18:15 +00:00
irenarindos 7ac73d7ba8 feat(bsr): Define types for summary data 2023-05-31 12:18:15 +00:00
irenarindos e29b844131 feat(bsr): improve error handling 2023-05-31 12:18:14 +00:00
irenarindos 47477e8fd6 feat(bsr): add BSR playback/conversion interfaces 2023-05-31 12:18:13 +00:00
irenarindos 444af9151c feat(bsr): Sync BSR keys 2023-05-31 12:18:13 +00:00
Timothy Messier 51e6e64d09 fix(bsr/checksum): Close after calling Stat 2023-05-31 12:18:12 +00:00
Timothy Messier 96f3c71cda test(bsr): Properly handle closed state for test fs implementation
The test implementation will now correctly mimic a real fs's Close
behavior. It will prevent reads and writes to a closed file or
container.
2023-05-31 12:18:12 +00:00
irenarindos b43830e66e feat(bsr): add protocol to sessionMeta 2023-05-31 12:18:12 +00:00
irenarindos 431b9d25a2 feat(bsr): use go 1.20 multierror 2023-05-31 12:18:11 +00:00
irenarindos ba476ca73b feat(bsr/ssh): Add SSH Request type chunks 2023-05-31 12:18:11 +00:00
Timothy Messier 26bd86fb07 feat(bsr): Add chunk encoder 2023-05-31 12:18:11 +00:00
Timothy MessierandIrena Rindos 2a31ddd881 feat(bsr): Define container and the required fs interfaces
Co-authored-by: Irena Rindos <irenarindos@users.noreply.github.com>
2023-05-31 12:18:11 +00:00
Jim ff9b3b0873 fix(bsr/kms): VerifySignatureWithBsrKey: handle invalid signature 2023-05-31 12:18:10 +00:00
Jim 9543af8765 feature (bsr/kms): add VerifySignatureWithBsrKey(...) 2023-05-31 12:18:10 +00:00
Timothy Messier 397a697731 feat(bsr): Define stubs for meta structures
These are used to provide the metadata about a session or connection
when writing a BSR and will be written to the corresponding .meta files
within a BSR.
2023-05-31 12:18:09 +00:00
Timothy Messier d15d2e3bc0 feat(bsr): Define required interface needed by storage 2023-05-31 12:18:09 +00:00
Jim aa10d1dbfe feat(KMS): add BsrKeys type and req. receiver functions 2023-05-31 12:18:09 +00:00