feat(scheduler): Use job table to select jobs to run
This commit changes the logic around selecting jobs to run. Before this, Boundary used a view "job_jobs_to_run" which was a collation of all jobs that were not running and scheduled to run. When jobs were requested to run, Boundary would look at this view, grab a certain amount of jobs and insert them into the "job_run" table. Importantly, while the amount of jobs to be scheduled at once defaulted to 1 in the scheduler package, this was entirely overriden in the controller package to -1 (no limit, run all jobs that the query would return). With this commit, Boundary now looks at the "job" table directly for jobs to run, regardless of job run status. This was done to improve DB performance as the view (which queried all "job_run" rows) was causing significant DB contention. Additionally, the option to limit job runs is now removed since it was already hardcoded to -1 and not user-configurable anyways. Should the need arise to implement this limit again, this will have to be done in a different way since the new query does not account for running jobs and limiting the rows returned after jobs are already running would yield a no-op (the new query inserts nothing into "job_run" if any given job is already running).
This commit is contained in:
1 parent
27c432a748
commit
f227020b67
13 files changed
+46
-334
No files matched your search
@@ -396,8 +396,8 @@ func New(ctx context.Context, conf *Config) (*Controller, error) {
|
||||
jobRepoFn := func() (*job.Repository, error) {
|
||||
return job.NewRepository(ctx, dbase, dbase, c.kms)
|
||||
}
|
||||
// TODO: Allow setting run jobs limit from config
|
||||
schedulerOpts := []scheduler.Option{scheduler.WithRunJobsLimit(-1)}
|
||||
|
||||
schedulerOpts := []scheduler.Option{}
|
||||
if c.conf.RawConfig.Controller.Scheduler.JobRunIntervalDuration > 0 {
|
||||
schedulerOpts = append(schedulerOpts, scheduler.WithRunJobsInterval(c.conf.RawConfig.Controller.Scheduler.JobRunIntervalDuration))
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user