Commit Graph
32 Commits
Author SHA1 Message Date
Jeff Mitchell 31ef4a65ac Bump deps and make gen 2020-05-29 11:21:12 -04:00
0c7f6da936 Static Host Catalog (#51)
* database: add wt_public_id domain type

* static-hosts: add migrations

* Rename migrations

* migrations: add description column and fix foreign keys

* Add protobuf for static host

* Add HostCatalog and HostSet

* Add Options

* Add foreign key constraint

* Add package doc

* Add copied scanner code

* Add NewHostCatalog with tests

* Override default table name for static host catalog

* WIP: save static host catalog to database

* WIP: Print db url for debugging

* Add iam test helper function

* Make gorm default to null for names and descriptions

* Add foreign key constraint from host catalog to scope

* Names are not needed for the test org and project

* Avoid an ordering dependency in down migrations

* Use wt_timestamp

* Add check constraint on host address column

* Add Host

* Add HostSet

* Add HostSetMember

* Test a HostSetMember can be written to the database

* Move the getters and setters for table name to a new file

The TableName methods are needed by gorm and the SetTableName methods
are for oplog. Moving them to a new file so related functionality is in
the same file.

* Add repository

* Add NewRepository

* Close connection before cleanup of database

* Add comments to protobuf file and regenerate

* Define repository methods for HostCatalog

* Assign PublicId when resource is inserted into the database

* Wrap a nil parameter error

* Add oplog tickets for static host types

* WIP: first cut at CreateCatalog

* CreateHostCatalog: more tests

* Detect unique constraint violation

* Use const for PublicId prefixes

* Use PostgreSQL error name instead of error code

* Add TODO comments

* Fix comments

* Use PublicId not PublicID

* Refactor: move error variables from host/static to db

The error variables defined in host/static are generic and can be reused
by other packages.

* Create a new repository for each test

* Validate CreateCatalog returns a new HostCatalog

* Do not modify the HostCatalog passed in

* Implement UpdateCatalog

* Minor cleanup to make folds work

* Implement LookupCatalog

* Refactor: extract allocCatalog function

* Add comment

* Refactor: extract newCatalogMetadata function

* Make error strings more consistent

* Move database specific error check to internal/db package

* Change signature of DeleteCatalog and update docs

* Implement DeleteCatalog

* Make error strings more consistent

* Add examples

* Update docs

* Add package level doc and example for repository

* No need for using reflect.DeepEqual

* Remove redundant comment

* Clarify ErrMultipleRecords results in the transaction being rolled back

* Remove redundant error checks

* Remove unnecessary check

* Return early instead of using nested 'else' statements

* Update and Delete must return a row count

* catch err on defers (#82)

Co-authored-by: Jim Lambert <jimlambrt@Jims-MBP-3.home>

* Remove 'if assert' pattern

* Use assert package in test helper function

* Add test for updating a non existent catalog

* Use a single assert when a specific error is expected

* Use assert package when it clarifies the check

* Catch errors in cleanup functions

* Update comments in protobuf file

* Update doc for UpdateCatalog to reflect the new behavior

* Add common error values for field mask errors

* Require a field mask when calling update methods

* Add TODO for removing direct call to gorm

* Export Public ID prefixes

* Add db timestamp guards

* Remove unused option

Co-authored-by: Jim <jlambert@hashicorp.com>
Co-authored-by: Jim Lambert <jimlambrt@Jims-MBP-3.home>
2020-05-27 18:16:43 -04:00
Jim aa64579a8c db timestamp guards (#70) 2020-05-27 17:35:04 -04:00
Jeff Mitchell b718091419 Update signatures of generated files to match our tool versions 2020-05-26 13:29:33 -04:00
Jim Lambert a7d8bda7c0 add missing parameter 2020-05-21 09:30:52 -04:00
Jim b75feb957d require fieldMaskPaths in Update and refactor errors (#71) 2020-05-21 09:24:37 -04:00
Michael Gaffney c9ad86c7ba Refactor: move error variables from host/static to db (#68)
The error variables defined in host/static are generic and can be reused
by other packages.
2020-05-20 16:23:22 -04:00
7f0dfa2344 Fix Update Project Logic (#54)
* carve out scope into a separate branch for a PR

* refactor getOpts to be private

* narrowed to just the scope proto

* wrap in a transaction

* remove user from scopes PR

* make scope type immutable

* refactored with more referential integrity (thanks mike)

* refactor based on pair programming with Mike

* better VetForWrite that supports db.Options but that requires GetOpts and Options to be exported.

* stop printing out all the db logs

* refactor withScope to be private

* refactor the VetForWrite a bit based on Todd's PR comment

* fix typo

Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>

* fix ctx propagation and Scope returns

* fix ctx propagation and use a const for std retry count

* change Std to Crud

* fix ctx propagation

* make sure update with field masks doesn't update a field not in the mask and force a read after update.

* add description

* remove comment about future

* clean error strings

* clean up enum string rep

* stdMeta

* fix return value to be just nil

* fix returns and errors

* update test to match new error string

* change len of domain to 24

* provide common function for new public ids

* use common db.NewPublicId() and provide the correct prefix

* move list out of CrudActions and provide a CrudlActions shortcut as well

* some comments about the purpose of VetForWrite()

* move to random 10

* Adding project API interface.

* Creation of Project handler and Tests

* Move to using the iam objects.

* Setting up testify framework for mocking and assertions.

* Fix iam options typo references.

* Adding tests for CreateProject and UpdateProject.

This also adds some additional argument validation.

* Merging in PR 40

* Renamed fakeRepo to mockRepo

* Changing around TODOs to capture current plans with the code.

* Remove the repo interface and use a real DB for tests.

* Fixed a few tests.

* Adding checks for properly formatted identifiers.

* Updating code to account for recent merges.  Relaxing some restrictions on what an id can look like.

* Added small comment explaining why RegisterGrpcGatewayer interface exists.

* Move project service up a call level.

Long term I think it would make sense to define these services even higher up the call stack, potentially at the initialization of the controller server, near where the DB is initialized.

* Making these fields align with those in the resources under hosts/...

* Hard code the service initialization in the handler registration code.

* Regenerate project related code proto code.

* Removing type field for project.

* Adds basic project handling in CLI/SDK (#47)

*    Adds SDK for creating and reading projects
*    Reorganizes a bit
*    Adds CLI for creating and reading projects
*    Fixes some some generation bugs
*    Adds redirect support for default org ID

* Fix protobuf import references being moved.

* Update client test to look for "orgs" and "projects" instead of "org" and "project"

* Update expected segments test

* Fix setAddr handling when default organization is in use

* Set client in target after create or read

* Adding functionality and tests for update masks.

* Additional tests for update.  Disable update non existant project test.

* Make not providing an update mask be an invalid argument.

* Make empty update mask paths and no mutable fields in the path an InvalidArgument error.

* Fix get Project test.

* Adding back in the Update a non existant project test case.

* Adding seperate path structured tests.

* Attach an issue id to field mask validation TODO.

* Generate Project Update SDK code (#58)

* Created the UpdateFunction method.

* Regenerate protobuf derived code since pull #50 updated dependencies.

* Special case IDs to not be passed for updates.

* Updated the tools file since imports reordered it.

* Add spaces between each tools import/comment grouping.

Co-authored-by: Jim Lambert <jimlambrt@Jims-MBP-3.home>
Co-authored-by: Jim <jlambert@hashicorp.com>
Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>
Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
2020-05-19 12:10:39 -07:00
Jim 980a55a4e3 Added DeleteScope() with tests and TestVerifyOplog() for easier testing (#49) 2020-05-18 17:51:09 -04:00
Jim d1acc1437b fix multierror 2020-05-18 14:38:32 -04:00
Jim ee46266893 ErrRecordNotFound from LookupWhere() when no rows found (#59) 2020-05-18 12:57:45 -04:00
Michael Gaffney fad302a36d Run make migrations (#61) 2020-05-16 19:53:13 -04:00
Michael Gaffney 3d614dd732 Run migrations in a transaction (#60) 2020-05-16 13:26:57 -04:00
Jeff Mitchell f8b456f3a7 Add TestController (#50)
This can be used in tests to get access to a controller that behaves
just like a dev controller but doesn't require the normal Command bits
(UI, etc).

This can be enhanced over time as needed; for now it's pretty simple but
it's good enough to be used for API tests. A simple project create/read
API test is added as well, which serves to show the programmatic model
(as well as test those functions at the API level).
2020-05-15 10:19:55 -04:00
2aca5fff03 fix transaction rollbacks in DoTx (#55)
* fix transaction rollbacks in DoTx

* better assertion

* Update internal/db/read_writer_test.go

Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>

Co-authored-by: Jim Lambert <jimlambrt@Jims-MBP-3.home>
Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>
2020-05-15 08:38:36 -04:00
Jim 4a88d10ecc fix a minor defect in fakeFile Readdir() and unit tests (#53) 2020-05-14 13:30:45 -04:00
Michael Gaffney 477a47897e Add wt_timestamp type to database (#52)
* Add wt_timestamp type to database

* Swap the order of domains and oplog migrations

All migrations should use the new 'wt_timestamp' domain type for
timestamps.

* Remove dead code

* Add down migration for wt_timestamp
2020-05-13 10:40:57 -04:00
Michael Gaffney 8d7f24268c Return url of test database (#48)
It is helpful to connect to a database when tests have failed.
2020-05-12 15:49:37 -04:00
Jeff Mitchell 7d8661be31 Remove migrations tag, not needed anymore 2020-05-12 12:31:40 -04:00
Jim c04c73453f refactor Update/Delete to return the number of rows affected. (#46) 2020-05-12 11:57:28 -04:00
Jim 61ca88dac9 refactor to use db.New() and clean up error msgs (#45) 2020-05-12 06:51:02 -04:00
Jeff Mitchell ad8d4e9c80 Rename Db's Tx to underlying 2020-05-11 21:05:11 -04:00
Jeff Mitchell 670abaed88 Rename GormReadWriter -> Db 2020-05-11 16:54:05 -04:00
Jim ee4a0d3f45 refactor lookups so we can easily check for not found and do the "right" thing in the repos (#42) 2020-05-11 16:21:09 -04:00
Jeff Mitchell 7f2aa5ca8f Add httpfs DB migrations and generate the input (#35) 2020-05-11 12:42:10 -04:00
Jim bd5ddea6e4 refactor clonable to not rely on the resource interface (#36) 2020-05-11 10:31:15 -04:00
Jim 0676ea51f1 refactor oplog to use db/internal migrations, remove InitTicket() for more reliability and simplify test step up and tear down (#37) 2020-05-11 10:30:44 -04:00
Jim c43e05be2e provide db.Open() and db.Migrate() as part of icu-64 (#34) 2020-05-08 17:21:22 -04:00
Jim 8a5197e39e implement schema, store and repository for IAM scopes of organization and project 2020-05-08 11:36:28 -04:00
Michael Gaffney 8824a6f7f9 database: add wt_public_id domain type (#30) 2020-05-06 08:29:13 -04:00
Jim 725a777baf refactor friendly name to just be name (#29) 2020-05-05 16:35:24 -04:00
Jim 27b4d7a034 basic internal db interfaces and Gorm implementation
* db reader and writer interfaces which should allow us to encapsulate the vast majority of database code in this package.

* GormReadWriter implementation of the Reader and Writer interfaces
2020-05-05 13:29:47 -04:00