Skip to content

Renaming jobs

River's design makes it safe to rename the Go types of job args and workers, but changing job Kind strings takes a few extra steps to perform safely.


Job kinds as strings

The Kind function of a job args class produces an identifier under which jobs of that type are stored to the database:

type MyArgs struct {}
func (MyArgs) Kind() string { return "stable_kind" }

Inserting MyArgs would create a record in the river_job table with kind set to "stable_kind".

When working a job, River sends it to the worker registered for the kind. Renaming the Go types MyArgs and MyWorker is always safe because regardless of the new name they're assigned, their Kind string stays the same, so River still knows where to send jobs.

Existing jobs make renames unsafe

It's the presence of existing data that makes renaming potentially unsafe. If we inserted an instance of MyArgs, stopped River, changed MyArgs's Kind return string from "stable_kind" to "UNSTABLE_kind", and deployed, River wouldn't know how to work the previously inserted job because there's no longer a worker registered for it.

Conversely, with no existing data (i.e. the river_job table is completely empty), renaming is always safe. Go job args and workers can not only be renamed at will, but their Kind strings changed liberally too.

Safely renaming job kinds

Job kinds can be renamed with existing jobs outstanding, but care must be taken to avoid accidental data loss.

Renaming a kind is a three step process involving the JobArgsWithKindAliases interface:

  1. Start with a job args struct with its original name:

    type jobArgsBeingRenamed struct{}
    func (a jobArgsBeingRenamed) Kind() string { return "old_name" }
  2. Change the job's Kind to its new target name, a KindAliases implementation which retains the old name, and deploy:

    type jobArgsBeingRenamed struct{}
    func (a jobArgsBeingRenamed) Kind() string { return "new_name" }
    func (a jobArgsBeingRenamed) KindAliases() []string { return []string{"old_name"} }

    New jobs are inserted as "new_name". Existing jobs still identify as "old_name". River happily dispatches either to the appropriate worker.

  3. After all jobs inserted under "old_name" are complete, remove KindAliases, and deploy:

    type jobArgsBeingRenamed struct{}
    func (a jobArgsBeingRenamed) Kind() string { return "new_name" }

Jobs under "old_name" may have previously errored and still be queued for eventual retry. Under the default retry policy, it'd take a full three weeks for a chronically failing job to be fully moved to discarded, and during that time they may run with "old_name".

Querying or updating kind aliases

To confirm whether or not it's safe to remove an old alias, query for jobs of a particular kind that are still eligible to be worked or retried:

SELECT count(*)
FROM river_job
WHERE kind = 'old_name' AND finalized_at IS NULL;

Alternatively, the process of waiting for an old kind to turn over can be expedited by updating the kind of all non-finalized (i.e those not completed or discarded) job rows:

UPDATE river_job
SET kind = 'new_name'
WHERE kind = 'old_name' AND finalized_at IS NULL;

The usual caveat applies in that this may be an expensive operation for enormous jobs tables, so run it with care. Depending on size, it may be advisable to update in limited batches to avoid long running queries:

WITH rename_batch AS (
SELECT *
FROM river_job
WHERE kind = 'old_name' AND finalized_at IS NULL
LIMIT 1000
)
UPDATE river_job
SET kind = 'new_name'
WHERE id IN (
SELECT id
FROM rename_batch
);