River's design makes it safe to rename the Go types of job args and workers, but changing job Kind
strings takes a few extra steps to perform safely.
Job kinds as strings
The Kind
function of a job args class produces an identifier under which jobs of that type are stored to the database:
type MyArgs struct {}
func (MyArgs) Kind() string { return "stable_kind" }
Inserting MyArgs
would create a record in the river_job
table with kind
set to "stable_kind"
.
When working a job, River sends it to the worker registered for the kind. Renaming the Go types MyArgs
and MyWorker
is always safe because regardless of the new name they're assigned, their Kind
string stays the same, so River still knows where to send jobs.
Existing jobs make renames unsafe
It's the presence of existing data that makes renaming potentially unsafe. If we inserted an instance of MyArgs
, stopped River, changed MyArgs
's Kind
return string from "stable_kind"
to "UNSTABLE_kind"
, and deployed, River wouldn't know how to work the previously inserted job because there's no longer a worker registered for it.
Conversely, with no existing data (i.e. the river_job
table is completely empty), renaming is always safe. Go job args and workers can not only be renamed at will, but their Kind
strings changed liberally too.
Safely renaming job kinds
Job kinds can be renamed with existing jobs outstanding, but care must be taken to avoid accidental data loss.
Renaming a kind is a three step process involving the JobArgsWithKindAliases
interface:
Start with a job args struct with its original name:
type jobArgsBeingRenamed struct{}func (a jobArgsBeingRenamed) Kind() string { return "old_name" }Change the job's
Kind
to its new target name, aKindAliases
implementation which retains the old name, and deploy:type jobArgsBeingRenamed struct{}func (a jobArgsBeingRenamed) Kind() string { return "new_name" }func (a jobArgsBeingRenamed) KindAliases() []string { return []string{"old_name"} }New jobs are inserted as
"new_name"
. Existing jobs still identify as"old_name"
. River happily dispatches either to the appropriate worker.After all jobs inserted under
"old_name"
are complete, removeKindAliases
, and deploy:type jobArgsBeingRenamed struct{}func (a jobArgsBeingRenamed) Kind() string { return "new_name" }
Jobs under "old_name"
may have previously errored and still be queued for eventual retry. Under the default retry policy, it'd take a full three weeks for a chronically failing job to be fully moved to discarded
, and during that time they may run with "old_name"
.
Querying or updating kind aliases
To confirm whether or not it's safe to remove an old alias, query for jobs of a particular kind
that are still eligible to be worked or retried:
SELECT count(*)FROM river_jobWHERE kind = 'old_name' AND finalized_at IS NULL;
Alternatively, the process of waiting for an old kind to turn over can be expedited by updating the kind
of all non-finalized (i.e those not completed
or discarded
) job rows:
UPDATE river_jobSET kind = 'new_name'WHERE kind = 'old_name' AND finalized_at IS NULL;
The usual caveat applies in that this may be an expensive operation for enormous jobs tables, so run it with care. Depending on size, it may be advisable to update in limited batches to avoid long running queries:
WITH rename_batch AS ( SELECT * FROM river_job WHERE kind = 'old_name' AND finalized_at IS NULL LIMIT 1000)UPDATE river_jobSET kind = 'new_name'WHERE id IN ( SELECT id FROM rename_batch);