Cancelling jobs

Jobs can be cancelled permanently in order to prevent them from running again or to cancel an ongoing execution.

If a worker recognizes that a job will never succeed, it can be cancelled permanently by returning the result of JobCancel. Compared to returning another error, JobCancel saves resources by preventing further retries.

Under normal circumstances, jobs that return an error are scheduled to be retried. This is done under the assumption that the problem they ran into was intermittent, or can be corrected by a code deploy that fixes a worker bug.

But there are times when a worker might realize that a job it's working will never succeed and should be abandoned immediately instead of wasting worker resources with continuous retries. This is where the JobCancel function is useful, which generates an error that River will recognize as a signal to permanently cancel a job:

func (w *CancellingWorker) Work(ctx context.Context, j *river.Job[CancellingArgs]) error {
    if thisJobWillNeverSucceed {
        return river.JobCancel(
            fmt.Errorf("this wrapped error message will be persisted to DB"))
    }

    return nil
}

See the JobCancel example for complete code.

JobCancel takes one argument — an error that should contain information on why the job was cancelled. This is purely for the benefit of operators — the error is persisted to the database and can be retrieved from the job's record to help understand the reason for its cancellation.

Jobs can also be cancelled from the River client using the Client.JobCancel method, whether or not the job is currently running:

if _, err := riverClient.JobCancel(ctx, jobID); err != nil {
    // handle error
}

See the JobCancelFromClient example for complete code.

This operation is inherently prone to certain race conditions, the details of which depend on the job's current state.

If the job is currently running, it is not immediately cancelled, but is instead marked for cancellation. The client running the job will also be notified (via LISTEN/NOTIFY) to cancel the running job's context. Although the job's context will be cancelled, since Go does not provide a mechanism to interrupt a running goroutine the job will continue running until it returns. As always, it is important for workers to respect context cancellation and return promptly when the job context is done.

Once the cancellation signal is received by the client running the job, any error returned by that job will result in it being cancelled permanently and not retried. However if the job returns no error, it will be completed as usual.

In the event the running job finishes executing before the cancellation signal is received but after this update was made, the behavior depends on which state the job is being transitioned into (based on its return value):

• If the job was due to be finalized because it completed successfully, was cancelled from within, or was discarded due to exceeding its max attempts, the job will be updated as usual.
• If the job was snoozed to run again later or encountered a retryable error, the job will be marked as cancelled and will not be attempted again.

For jobs that are still in the queue (available, scheduled, or retryable), cancellation is straightforward. The job is immediately and atomically marked as cancelled so that no client will fetch it from the queue and it will not be attempted again.

If the job is already finalized (cancelled, completed, or discarded), no changes are made.