mirror of
https://github.com/hibiken/asynq.git
synced 2026-06-22 01:17:49 +08:00
Document atomicity guarantees and fix NOSCRIPT bug in BatchEnqueue
Add comprehensive doc comments to BatchEnqueueContext, the Broker interface method, and the RDB implementation explaining that the batch uses a Redis pipeline (not MULTI/EXEC), so partial success is possible and individual Lua scripts are atomic but the batch is not. Fix a bug where Script.Run inside a pipeline only sends EVALSHA without the automatic EVAL fallback that non-pipeline calls get. On a fresh Redis (or after SCRIPT FLUSH), this caused NOSCRIPT errors for every pipeline-batched script invocation. The fix preloads the required Lua scripts before building the pipeline. Also roll back the in-memory queuesPublished cache when the pipeline fails, preventing stale entries from suppressing future SADD calls.
This commit is contained in:
37
client.go
37
client.go
@@ -426,10 +426,39 @@ type BatchEnqueueResult struct {
|
||||
Err error
|
||||
}
|
||||
|
||||
// BatchEnqueueContext enqueues all given tasks using a single Redis pipeline round-trip.
|
||||
// Each task gets its own result so callers can handle partial success.
|
||||
// Immediate and scheduled tasks are supported; unique and group tasks are
|
||||
// rejected with an error in the corresponding BatchEnqueueResult.
|
||||
// BatchEnqueueContext enqueues multiple tasks in a single Redis pipeline round-trip,
|
||||
// returning a per-task result slice aligned with the input tasks slice.
|
||||
//
|
||||
// # Atomicity Guarantees
|
||||
//
|
||||
// There is no all-or-nothing guarantee across the batch. Each task is executed as
|
||||
// an independent Lua script inside a Redis pipeline. Individual scripts are atomic
|
||||
// (the existence check, hash write, and list/sorted-set push for one task cannot
|
||||
// be partially applied), but the pipeline as a whole is not wrapped in a
|
||||
// MULTI/EXEC transaction. This means:
|
||||
//
|
||||
// - Partial success is possible: some tasks may be enqueued while others are not.
|
||||
// - A task whose ID already exists in Redis is silently skipped (treated as a
|
||||
// no-op by the Lua script), and its result will still show success.
|
||||
// - If the Redis pipeline call itself fails (e.g. connection lost, context
|
||||
// cancelled), every task that passed client-side validation receives that
|
||||
// error — none of them can be assumed to have been enqueued.
|
||||
//
|
||||
// # Validation Errors (pre-pipeline)
|
||||
//
|
||||
// The following are caught before any Redis call and rejected in the
|
||||
// corresponding BatchEnqueueResult.Err without affecting other tasks:
|
||||
//
|
||||
// - nil task
|
||||
// - empty task type name
|
||||
// - invalid options
|
||||
// - group tasks (not supported in batch mode)
|
||||
// - unique tasks (not supported in batch mode)
|
||||
//
|
||||
// # Supported Task Types
|
||||
//
|
||||
// Immediate and scheduled (via [ProcessAt] or [ProcessIn]) tasks are supported.
|
||||
// Group and unique tasks are rejected as described above.
|
||||
func (c *Client) BatchEnqueueContext(ctx context.Context, tasks []*Task, opts ...Option) []BatchEnqueueResult {
|
||||
results := make([]BatchEnqueueResult, len(tasks))
|
||||
if len(tasks) == 0 {
|
||||
|
||||
Reference in New Issue
Block a user