Skip to content
River Docs

Per-queue job retention

Normally River's job cleaner uses global settings that determine how long it should retain cancelled, completed, and discarded jobs, but additional retention granularity is also available on a per-queue basis. This is useful in cases like where a hyper-frequent high-volume job should only be retained for a minimal amount of time to help keep the database's size down, but a more important job related to account billing should be retained for an extended period for audibility.

Per-queue job retention is a feature of River Pro ✨. If you haven't yet, install River Pro.

Added in River Pro v0.16.0.

Global retention settings

River's standard (i.e. non-pro) behavior is that all jobs are cleaned according to global settings configurable through properties on River.Config:

  • Cancelled: CancelledJobRetentionPeriod, defaults to 24 hours.
  • Completed: CompletedJobRetentionPeriod, defaults to 24 hours.
  • Discarded: DiscardedJobRetentionPeriod, defaults to 7 days.

An example of how to configure these retention periods:

riverClient, err = river.NewClient(riverpropgxv5.New(dbPool), &river.Config{
    CancelledJobRetentionPeriod: 24 * time.Hour,
    CompletedJobRetentionPeriod: 24 * time.Hour,
    DiscardedJobRetentionPeriod: 7 * 24 * time.Hour,
    Queues: map[string]river.QueueConfig{
        river.QueueDefault: {
            MaxWorkers: 100,
        },
    },
    Workers:  workers,
})
if err != nil {
    // handle error
}

Configuring per-queue retention

Per-queue retention requires a River Pro client. Retention settings have the same names as the global configuration, and are settable on each entry in the riverpro.Config.ProQueues map:

riverClient, err = riverpro.NewClient(riverpropgxv5.New(dbPool), &riverpro.Config{
    Config: river.Config{
        Workers:  workers,
    },
    ProQueues: map[string]riverpro.QueueConfig{
        "queue-hyper-frequent": { // retain a short time
            CancelledJobRetentionPeriod: 10 * time.Minute,
            CompletedJobRetentionPeriod: 10 * time.Minute,
            DiscardedJobRetentionPeriod: 24 * time.Hour,
            MaxWorkers:                  100,
        },
        "queue-long-term-retention": { // retain a long time
            CancelledJobRetentionPeriod: 7 * 24 * time.Hour,
            CompletedJobRetentionPeriod: 7 * 24 * time.Hour,
            DiscardedJobRetentionPeriod: 30 * 24 * time.Hour,
            MaxWorkers:                  100,
        },
    },
})
if err != nil {
    // handle error
}

Queues that aren't present in ProQueues, or which don't have custom retention settings (i.e. left unset and default to their zero values), will be cleaned according to global configuration the same way as any other job.