In-memory queue

River can be run as a fully in-memory queue by combining the SQLite driver with SQLite's :memory: mode. This is useful for development, testing, and ephemeral workloads where persistence isn't needed.

An in-memory queue behaves identically to a persistent one — jobs are inserted, worked, retried, and are even transactional — but all state is lost when the process exits. There's no disk I/O, no cleanup, and no external dependencies.

Sample code

Open a database/sql connection with the :memory: DSN and pass it to River's SQLite driver:

import (
    "database/sql"

    _ "modernc.org/sqlite"

    "github.com/riverqueue/river"
    "github.com/riverqueue/river/riverdriver/riversqlite"
    "github.com/riverqueue/river/rivermigrate"
)

dbPool, err := sql.Open("sqlite", ":memory:")
if err != nil {
    panic(err)
}
defer dbPool.Close()

dbPool.SetMaxOpenConns(1)

migrator, err := rivermigrate.New(riversqlite.New(dbPool), nil)
if err != nil {
    panic(err)
}

_, err = migrator.Migrate(ctx, rivermigrate.DirectionUp, nil)
if err != nil {
    panic(err)
}

workers := river.NewWorkers()
river.AddWorker(workers, &MyWorker{})

riverClient, err := river.NewClient(riversqlite.New(dbPool), &river.Config{
    Queues: map[string]river.QueueConfig{
        river.QueueDefault: {MaxWorkers: 100},
    },
    Workers: workers,
})
if err != nil {
    panic(err)
}

Since an in-memory database starts empty every time, it's necessary to run migrations before starting the client. The CLI can't be used in this instance because it would open its own connection to a separate in-memory database.

No cleanup is necessary after this code runs. The in-memory database is discarded when the connection closes.

SQLite only

In-memory queues are SQLite only, and as such the same concurrency considerations as file-backed SQLite apply. Use a pool with a maximum of one connection (SetMaxOpenConns(1)) and keep transactions short to avoid blocking.