River supports batch inserts, wherein many jobs are inserted at once for fewer round trips and optimal performance.
Insert many
Batch inserts are executed with Client.InsertMany and Client.InsertManyTx. Both take a slice of InsertManyParams structs, which like a call to a normal non-batch insert function, take job args and optional InsertOpts.
results, err := riverClient.InsertMany(ctx, []river.InsertManyParams{
{Args: BatchInsertArgs{}},
{Args: BatchInsertArgs{}},
{Args: BatchInsertArgs{}},
{Args: BatchInsertArgs{}, InsertOpts: &river.InsertOpts{Priority: 3}},
{Args: BatchInsertArgs{}, InsertOpts: &river.InsertOpts{Priority: 4}},
})
if err != nil {
panic(err)
}
fmt.Printf("Inserted %d jobs\n", len(results))
See the BatchInsert example for complete code.
InsertManyTx takes a transaction, and like InsertTx, all the normal transactional enqueuing benefits apply, like that jobs aren't worked until the transaction commits, and are removed if it rolls back.
results, err := riverClient.InsertManyTx(ctx, tx, []river.InsertManyParams{
...
}
An even faster variant
Normal job insertions are quite fast so it's usually not necessary to resort to batch job insertion, but it may be desireable in situations where multiple jobs are being inserted at once because batch insertion requires fewer round trips.
A third option exists for cases where thousands of jobs are being inserted at once: InsertManyFast / InsertManyFastTx. Under the hood these methods use Postgres COPY FROM (only in the riverpgxv5 driver), has a few further performance benefits like reduced logging overhead. This method has some limitations, however, such as its inability to return inserted data or to detect and handle unique conflicts cleanly without rolling back the entire transaction.