2
0
mirror of https://github.com/hibiken/asynq.git synced 2026-06-22 01:17:49 +08:00

Document atomicity guarantees and fix NOSCRIPT bug in BatchEnqueue

Add comprehensive doc comments to BatchEnqueueContext, the Broker
interface method, and the RDB implementation explaining that the batch
uses a Redis pipeline (not MULTI/EXEC), so partial success is possible
and individual Lua scripts are atomic but the batch is not.

Fix a bug where Script.Run inside a pipeline only sends EVALSHA without
the automatic EVAL fallback that non-pipeline calls get. On a fresh
Redis (or after SCRIPT FLUSH), this caused NOSCRIPT errors for every
pipeline-batched script invocation. The fix preloads the required Lua
scripts before building the pipeline.

Also roll back the in-memory queuesPublished cache when the pipeline
fails, preventing stale entries from suppressing future SADD calls.
This commit is contained in:
Erik Nilsen
2026-04-09 11:35:11 -07:00
parent 7ae0b3fe22
commit 06a06970d6
3 changed files with 91 additions and 14 deletions

View File

@@ -426,10 +426,39 @@ type BatchEnqueueResult struct {
Err error
}
// BatchEnqueueContext enqueues all given tasks using a single Redis pipeline round-trip.
// Each task gets its own result so callers can handle partial success.
// Immediate and scheduled tasks are supported; unique and group tasks are
// rejected with an error in the corresponding BatchEnqueueResult.
// BatchEnqueueContext enqueues multiple tasks in a single Redis pipeline round-trip,
// returning a per-task result slice aligned with the input tasks slice.
//
// # Atomicity Guarantees
//
// There is no all-or-nothing guarantee across the batch. Each task is executed as
// an independent Lua script inside a Redis pipeline. Individual scripts are atomic
// (the existence check, hash write, and list/sorted-set push for one task cannot
// be partially applied), but the pipeline as a whole is not wrapped in a
// MULTI/EXEC transaction. This means:
//
// - Partial success is possible: some tasks may be enqueued while others are not.
// - A task whose ID already exists in Redis is silently skipped (treated as a
// no-op by the Lua script), and its result will still show success.
// - If the Redis pipeline call itself fails (e.g. connection lost, context
// cancelled), every task that passed client-side validation receives that
// error — none of them can be assumed to have been enqueued.
//
// # Validation Errors (pre-pipeline)
//
// The following are caught before any Redis call and rejected in the
// corresponding BatchEnqueueResult.Err without affecting other tasks:
//
// - nil task
// - empty task type name
// - invalid options
// - group tasks (not supported in batch mode)
// - unique tasks (not supported in batch mode)
//
// # Supported Task Types
//
// Immediate and scheduled (via [ProcessAt] or [ProcessIn]) tasks are supported.
// Group and unique tasks are rejected as described above.
func (c *Client) BatchEnqueueContext(ctx context.Context, tasks []*Task, opts ...Option) []BatchEnqueueResult {
results := make([]BatchEnqueueResult, len(tasks))
if len(tasks) == 0 {