Commit Graph
60 Commits
Author SHA1 Message Date
Jim 8d8bb42418 add CreateItems and DeleteItems to writer interface (#156) 2020-06-29 19:28:15 -04:00
Jim 3e3144117e iam and auth integration (#144) 2020-06-29 08:18:57 -04:00
Jim 9a87df38c7 add CreateItems() and DeleteItems() (#146) 2020-06-29 06:31:23 -04:00
Michael Gaffney d69588131c Fix for panic in db.setFieldsToNil (#155) 2020-06-28 16:55:57 -04:00
Jim 6641244734 add support for WithVersion option to updates (#126) 2020-06-24 12:33:49 -04:00
Michael Gaffney c110c86ee5 Add function to check if db error is a check constraint violation (#142) 2020-06-23 18:00:50 -04:00
Todd Knight 0c8385b423 Updating TxHandler to accept a reader as well in order to support reads inside the same transactions as writes. (#139) 2020-06-22 13:27:26 -07:00
Michael Gaffney 6908f7a7a2 Use standard column names (#138) 2020-06-22 13:54:55 -04:00
Michael Gaffney 1023c90729 Enforce auth_account to iam_user plus iam_scope constraints (#137) 2020-06-21 13:16:03 -04:00
Michael Gaffney 0ebb38bbe5 auth method schema (#131)
* Rename iam migrations to 06

* Add base tables for auth methods and auth accounts

* Add triggers for auth_method and auth_account subtypes
2020-06-19 13:10:05 -04:00
Jim c413e013bd get oplog Ticket before writing data (#127) 2020-06-18 21:24:55 -04:00
Jim d9073a03d5 listing users (#118) 2020-06-16 18:59:30 -04:00
Jim ffddf24f86 iam roles (#116) 2020-06-15 12:42:08 -04:00
Michael Gaffney 2d650b241d Fix down migrations (#120)
* Fix iam down migrations

* Fix '03_db' down migrations

* Fix oplog down migrations

* Fix domain down migrations

* Run make gen
2020-06-13 07:20:40 -04:00
Jim 5c2cff3ad2 iam groups (#112) 2020-06-11 16:36:47 -04:00
Michael Gaffney 4f30983c14 Bump deps and run make gen (#115) 2020-06-11 10:08:54 -04:00
Jim 2d18177c9d oplog support for null field updates (#107) 2020-06-10 06:56:46 -04:00
Jim c70847fbac refactor delete test to be table driven. (#102) 2020-06-09 21:22:15 -04:00
Jeff Mitchell 5333482ff8 Fix gorm newlines issue (#108)
Even if you made a custom formatter for gorm, it would blithely println
anything returned, including nils, resulting in lots of new lines when
we were ignoring output. This also creates a custom logger that doesn't
do anything when newlines are found.
2020-06-08 21:12:00 -04:00
Jim 7d4993fb61 iam users (#96) 2020-06-04 20:05:08 -04:00
JimandJim Lambert e3fdb493b8 prevent primary key updates (#95)
* prevent primary key updates

* better error to return

Co-authored-by: Jim Lambert <jimlambrt@Jims-MBP-3.home>
2020-06-03 09:51:34 -04:00
Jim 1048673171 update dbassert dep (#100) 2020-06-02 12:07:00 -04:00
Jim 9feb280536 wrapper for dbasserts (#94) 2020-06-02 08:59:40 -04:00
Jeff Mitchell c175fcde8d Add a gorm log formatter to adapt to hclog (#93)
This is also added in a test that otherwise output to stdout, which
validates functionality.
2020-06-01 17:32:48 -04:00
Michael Gaffney 5bdd8a46ea Extract timestamp scanners into reusable package (#91)
* Refactor: move Timestamp from static/store to db/timestamp

* Refactor: convert remaining Timestamps to timestamp.Timestamp

* Use new db timestamp
2020-06-01 11:46:50 -04:00
Michael Gaffney 9eaf8afb6f Rename db.IsUnique to db.IsUniqueError (#90) 2020-06-01 09:54:02 -04:00
Jim ef6edbd515 Null updates (#78) 2020-05-31 19:55:56 -04:00
Michael Gaffney 6d123d91bb Fix comment for trigger (#92) 2020-05-29 14:51:22 -04:00
Jeff Mitchell 31ef4a65ac Bump deps and make gen 2020-05-29 11:21:12 -04:00
0c7f6da936 Static Host Catalog (#51)
* database: add wt_public_id domain type

* static-hosts: add migrations

* Rename migrations

* migrations: add description column and fix foreign keys

* Add protobuf for static host

* Add HostCatalog and HostSet

* Add Options

* Add foreign key constraint

* Add package doc

* Add copied scanner code

* Add NewHostCatalog with tests

* Override default table name for static host catalog

* WIP: save static host catalog to database

* WIP: Print db url for debugging

* Add iam test helper function

* Make gorm default to null for names and descriptions

* Add foreign key constraint from host catalog to scope

* Names are not needed for the test org and project

* Avoid an ordering dependency in down migrations

* Use wt_timestamp

* Add check constraint on host address column

* Add Host

* Add HostSet

* Add HostSetMember

* Test a HostSetMember can be written to the database

* Move the getters and setters for table name to a new file

The TableName methods are needed by gorm and the SetTableName methods
are for oplog. Moving them to a new file so related functionality is in
the same file.

* Add repository

* Add NewRepository

* Close connection before cleanup of database

* Add comments to protobuf file and regenerate

* Define repository methods for HostCatalog

* Assign PublicId when resource is inserted into the database

* Wrap a nil parameter error

* Add oplog tickets for static host types

* WIP: first cut at CreateCatalog

* CreateHostCatalog: more tests

* Detect unique constraint violation

* Use const for PublicId prefixes

* Use PostgreSQL error name instead of error code

* Add TODO comments

* Fix comments

* Use PublicId not PublicID

* Refactor: move error variables from host/static to db

The error variables defined in host/static are generic and can be reused
by other packages.

* Create a new repository for each test

* Validate CreateCatalog returns a new HostCatalog

* Do not modify the HostCatalog passed in

* Implement UpdateCatalog

* Minor cleanup to make folds work

* Implement LookupCatalog

* Refactor: extract allocCatalog function

* Add comment

* Refactor: extract newCatalogMetadata function

* Make error strings more consistent

* Move database specific error check to internal/db package

* Change signature of DeleteCatalog and update docs

* Implement DeleteCatalog

* Make error strings more consistent

* Add examples

* Update docs

* Add package level doc and example for repository

* No need for using reflect.DeepEqual

* Remove redundant comment

* Clarify ErrMultipleRecords results in the transaction being rolled back

* Remove redundant error checks

* Remove unnecessary check

* Return early instead of using nested 'else' statements

* Update and Delete must return a row count

* catch err on defers (#82)

Co-authored-by: Jim Lambert <jimlambrt@Jims-MBP-3.home>

* Remove 'if assert' pattern

* Use assert package in test helper function

* Add test for updating a non existent catalog

* Use a single assert when a specific error is expected

* Use assert package when it clarifies the check

* Catch errors in cleanup functions

* Update comments in protobuf file

* Update doc for UpdateCatalog to reflect the new behavior

* Add common error values for field mask errors

* Require a field mask when calling update methods

* Add TODO for removing direct call to gorm

* Export Public ID prefixes

* Add db timestamp guards

* Remove unused option

Co-authored-by: Jim <jlambert@hashicorp.com>
Co-authored-by: Jim Lambert <jimlambrt@Jims-MBP-3.home>
2020-05-27 18:16:43 -04:00
Jim aa64579a8c db timestamp guards (#70) 2020-05-27 17:35:04 -04:00
Jeff Mitchell b718091419 Update signatures of generated files to match our tool versions 2020-05-26 13:29:33 -04:00
Jim Lambert a7d8bda7c0 add missing parameter 2020-05-21 09:30:52 -04:00
Jim b75feb957d require fieldMaskPaths in Update and refactor errors (#71) 2020-05-21 09:24:37 -04:00
Michael Gaffney c9ad86c7ba Refactor: move error variables from host/static to db (#68)
The error variables defined in host/static are generic and can be reused
by other packages.
2020-05-20 16:23:22 -04:00
7f0dfa2344 Fix Update Project Logic (#54)
* carve out scope into a separate branch for a PR

* refactor getOpts to be private

* narrowed to just the scope proto

* wrap in a transaction

* remove user from scopes PR

* make scope type immutable

* refactored with more referential integrity (thanks mike)

* refactor based on pair programming with Mike

* better VetForWrite that supports db.Options but that requires GetOpts and Options to be exported.

* stop printing out all the db logs

* refactor withScope to be private

* refactor the VetForWrite a bit based on Todd's PR comment

* fix typo

Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>

* fix ctx propagation and Scope returns

* fix ctx propagation and use a const for std retry count

* change Std to Crud

* fix ctx propagation

* make sure update with field masks doesn't update a field not in the mask and force a read after update.

* add description

* remove comment about future

* clean error strings

* clean up enum string rep

* stdMeta

* fix return value to be just nil

* fix returns and errors

* update test to match new error string

* change len of domain to 24

* provide common function for new public ids

* use common db.NewPublicId() and provide the correct prefix

* move list out of CrudActions and provide a CrudlActions shortcut as well

* some comments about the purpose of VetForWrite()

* move to random 10

* Adding project API interface.

* Creation of Project handler and Tests

* Move to using the iam objects.

* Setting up testify framework for mocking and assertions.

* Fix iam options typo references.

* Adding tests for CreateProject and UpdateProject.

This also adds some additional argument validation.

* Merging in PR 40

* Renamed fakeRepo to mockRepo

* Changing around TODOs to capture current plans with the code.

* Remove the repo interface and use a real DB for tests.

* Fixed a few tests.

* Adding checks for properly formatted identifiers.

* Updating code to account for recent merges.  Relaxing some restrictions on what an id can look like.

* Added small comment explaining why RegisterGrpcGatewayer interface exists.

* Move project service up a call level.

Long term I think it would make sense to define these services even higher up the call stack, potentially at the initialization of the controller server, near where the DB is initialized.

* Making these fields align with those in the resources under hosts/...

* Hard code the service initialization in the handler registration code.

* Regenerate project related code proto code.

* Removing type field for project.

* Adds basic project handling in CLI/SDK (#47)

*    Adds SDK for creating and reading projects
*    Reorganizes a bit
*    Adds CLI for creating and reading projects
*    Fixes some some generation bugs
*    Adds redirect support for default org ID

* Fix protobuf import references being moved.

* Update client test to look for "orgs" and "projects" instead of "org" and "project"

* Update expected segments test

* Fix setAddr handling when default organization is in use

* Set client in target after create or read

* Adding functionality and tests for update masks.

* Additional tests for update.  Disable update non existant project test.

* Make not providing an update mask be an invalid argument.

* Make empty update mask paths and no mutable fields in the path an InvalidArgument error.

* Fix get Project test.

* Adding back in the Update a non existant project test case.

* Adding seperate path structured tests.

* Attach an issue id to field mask validation TODO.

* Generate Project Update SDK code (#58)

* Created the UpdateFunction method.

* Regenerate protobuf derived code since pull #50 updated dependencies.

* Special case IDs to not be passed for updates.

* Updated the tools file since imports reordered it.

* Add spaces between each tools import/comment grouping.

Co-authored-by: Jim Lambert <jimlambrt@Jims-MBP-3.home>
Co-authored-by: Jim <jlambert@hashicorp.com>
Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>
Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
2020-05-19 12:10:39 -07:00
Jim 980a55a4e3 Added DeleteScope() with tests and TestVerifyOplog() for easier testing (#49) 2020-05-18 17:51:09 -04:00
Jim d1acc1437b fix multierror 2020-05-18 14:38:32 -04:00
Jim ee46266893 ErrRecordNotFound from LookupWhere() when no rows found (#59) 2020-05-18 12:57:45 -04:00
Michael Gaffney fad302a36d Run make migrations (#61) 2020-05-16 19:53:13 -04:00
Michael Gaffney 3d614dd732 Run migrations in a transaction (#60) 2020-05-16 13:26:57 -04:00
Jeff Mitchell f8b456f3a7 Add TestController (#50)
This can be used in tests to get access to a controller that behaves
just like a dev controller but doesn't require the normal Command bits
(UI, etc).

This can be enhanced over time as needed; for now it's pretty simple but
it's good enough to be used for API tests. A simple project create/read
API test is added as well, which serves to show the programmatic model
(as well as test those functions at the API level).
2020-05-15 10:19:55 -04:00
2aca5fff03 fix transaction rollbacks in DoTx (#55)
* fix transaction rollbacks in DoTx

* better assertion

* Update internal/db/read_writer_test.go

Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>

Co-authored-by: Jim Lambert <jimlambrt@Jims-MBP-3.home>
Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>
2020-05-15 08:38:36 -04:00
Jim 4a88d10ecc fix a minor defect in fakeFile Readdir() and unit tests (#53) 2020-05-14 13:30:45 -04:00
Michael Gaffney 477a47897e Add wt_timestamp type to database (#52)
* Add wt_timestamp type to database

* Swap the order of domains and oplog migrations

All migrations should use the new 'wt_timestamp' domain type for
timestamps.

* Remove dead code

* Add down migration for wt_timestamp
2020-05-13 10:40:57 -04:00
Michael Gaffney 8d7f24268c Return url of test database (#48)
It is helpful to connect to a database when tests have failed.
2020-05-12 15:49:37 -04:00
Jeff Mitchell 7d8661be31 Remove migrations tag, not needed anymore 2020-05-12 12:31:40 -04:00
Jim c04c73453f refactor Update/Delete to return the number of rows affected. (#46) 2020-05-12 11:57:28 -04:00
Jim 61ca88dac9 refactor to use db.New() and clean up error msgs (#45) 2020-05-12 06:51:02 -04:00
Jeff Mitchell ad8d4e9c80 Rename Db's Tx to underlying 2020-05-11 21:05:11 -04:00