2
0
mirror of https://github.com/hibiken/asynq.git synced 2026-06-16 02:56:17 +08:00
Commit Graph

83 Commits

Author SHA1 Message Date
Mohamed Sohail
785bb7208c Merge pull request #1094 from fanatics-live/batch-enqueue
Add BatchEnqueue for pipelined multi-task enqueue
2026-05-14 15:34:54 +03:00
kayoch1n
1af926b147 chore: Add docs string to asynq.Header and remove asynq.Headers 2026-04-15 20:02:30 +08:00
Erik Nilsen
06a06970d6 Document atomicity guarantees and fix NOSCRIPT bug in BatchEnqueue
Add comprehensive doc comments to BatchEnqueueContext, the Broker
interface method, and the RDB implementation explaining that the batch
uses a Redis pipeline (not MULTI/EXEC), so partial success is possible
and individual Lua scripts are atomic but the batch is not.

Fix a bug where Script.Run inside a pipeline only sends EVALSHA without
the automatic EVAL fallback that non-pipeline calls get. On a fresh
Redis (or after SCRIPT FLUSH), this caused NOSCRIPT errors for every
pipeline-batched script invocation. The fix preloads the required Lua
scripts before building the pipeline.

Also roll back the in-memory queuesPublished cache when the pipeline
fails, preventing stale entries from suppressing future SADD calls.
2026-04-09 11:35:11 -07:00
kayoch1n
5216f1c3be feat: Add an option to create headers from a map 2026-03-17 14:47:34 +08:00
kayoch1n
5c391f3ffb feat: Add an option to specify headers 2026-03-17 12:29:58 +08:00
Erik Nilsen
71ebcfa129 Fix BatchEnqueueContext time comparison and add scheduled task support
BatchEnqueueContext had a time comparison bug where `now` was captured
before the loop but `processAt` was set to time.Now() inside
composeOptions during each iteration, causing all immediate tasks to be
incorrectly classified as scheduled and rejected.

Fix: move `now` capture inside the loop, after composeOptions.

Additionally, extend BatchEnqueueContext to support scheduled tasks in
the same pipeline. Tasks with a future ProcessAt are now routed to
scheduleCmd (ZADD to scheduled set) instead of being rejected. Only
unique and group tasks remain unsupported.

Changes:
- Add BatchEnqueueItem type pairing TaskMessage with optional ProcessAt
- Update Broker interface, RDB, and testbroker to use BatchEnqueueItem
- Route immediate tasks to enqueueCmd, scheduled tasks to scheduleCmd
- Return correct TaskState (Pending vs Scheduled) in results
- Add tests for immediate, scheduled, and mixed batch scenarios
2026-02-25 09:42:30 -08:00
Erik Nilsen
f919a605d5 Add BatchEnqueue for pipelined multi-task enqueue
Adds BatchEnqueue to the Broker interface and RDB implementation that
sends multiple enqueueCmd Lua script invocations in a single Redis
pipeline round-trip. Also adds BatchEnqueueContext to the Client as
the public API, returning per-task results for partial-success handling.

Ref: hibiken/asynq#1069
2026-02-23 16:25:49 -08:00
Joe
5de9b1faf0 Feature: Add Headers Support to Tasks (#1070)
* feat(task): Add headers support to tasks

* fix: cleanup copy map code

* fix: Add tests
2025-11-04 20:07:59 +03:00
Pior Bastida
03f4799712 Run golangci-lint in CI (#927)
* Setup golangci-lint in CI and local-dev

* Fix linting error or locally disable linter
2024-10-26 08:48:12 +03:00
Patrick Barnum
b1e13893ff [RFC] Adds Ping() to client/scheduler/server (#585)
* [RFC] Adds Ping() to client/scheduler/server

* Checks for scheduler state closed
2024-10-19 09:44:06 +03:00
Mohammed Sohail
461d922616 docs: apply recommendaded updates
* additionally, we log an erro in the case the redis client cannot shutdown in the scheduler
2024-10-19 09:05:17 +03:00
Jeroen Bobbeldijk
9e548fc097 Implement reusing redis client 2023-09-19 11:20:32 +02:00
Emanuel Bennici
0275df8df4 Update redis/go-redis to v9
Version v9 implements the support for Redis v7 and has some
other improvements.
2023-04-17 22:30:33 -07:00
Jeffrey Lo
9116c096ec docs: correct typo (deafult => default) 2022-05-04 19:10:09 -07:00
Ken Hibino
8b2a787759 Rename variables for public API documentation 2022-04-11 16:55:43 -07:00
Ken Hibino
562506c7ba Fix client to return error when nil task is passed 2022-04-11 16:55:43 -07:00
Ken Hibino
ab28234767 Update client dependency to base.Broker 2022-04-11 16:55:43 -07:00
Ken Hibino
4e8885276c Update client to store groupKey under TaskMessage 2022-04-11 16:55:43 -07:00
Ken Hibino
f17c157b0f Update Client to add task to group if Group option is specified 2022-04-11 16:55:43 -07:00
Ken Hibino
2ce71e83b0 Add Group task option 2022-04-11 16:55:43 -07:00
Ken Hibino
9f2c321e98 Add EnqueueContext method to Client 2021-11-15 16:34:26 -08:00
Ken Hibino
e2b61c9056 Return error if Unique TTL is less than 1s 2021-11-09 16:37:02 -08:00
Ken Hibino
f4ddac4dcc Introduce Task Results
* Added Retention Option to specify retention TTL for tasks
* Added ResultWriter as a client interface to write result data for the associated task
2021-11-06 15:18:42 -07:00
Ken Hibino
9e2f88c00d Add TaskID option to allow user to specify task id 2021-11-06 15:18:42 -07:00
Ken Hibino
2261c7c9a0 Change TaskMessage.ID type from uuid.UUID to string 2021-11-06 15:18:42 -07:00
Ken Hibino
83cae4bb24 Update NewTask function to take Option as varargs 2021-11-06 15:18:42 -07:00
Jason White
b3ef9e91a9 Upgrade go-redis/redis to version 8 2021-09-02 05:56:02 -07:00
Luqqk
95c90a5cb8 Add changelog entry, add additional test case 2021-08-02 20:20:09 -07:00
Luqqk
6817af366a Adjust error message, use TrimSpace for more robust empty typename check 2021-08-02 20:20:09 -07:00
Luqqk
4bce28d677 client.Enqueue - prevent empty task's typename 2021-08-02 20:20:09 -07:00
Ken Hibino
46b23d6495 Allow upper case characters in queue name 2021-07-15 06:55:47 -07:00
Ken Hibino
a9feec5967 Change TaskInfo to use public fields instead of methods 2021-06-29 16:34:21 -07:00
Ken Hibino
b835090ad8 Update Client.Enqueue to return TaskInfo 2021-06-29 16:34:21 -07:00
Ken Hibino
1812d05d21 Fix build 2021-06-29 16:34:21 -07:00
Ken Hibino
476812475e Change payload to byte slice 2021-06-29 16:34:21 -07:00
Ken Hibino
97b5516183 Update RedisConnOpt interface 2021-01-31 06:09:40 -08:00
Ken Hibino
e13122723a Move all inspector related code to subpackage inspeq 2021-01-31 06:09:40 -08:00
Ken Hibino
b604d25937 Add helper function to parse Option string 2021-01-14 06:43:44 -08:00
Ken Hibino
8312515e64 Update Option interface
- Added `String()`, `Type()`, and `Value()` methods to the interface to
  aid with debugging and error handling.
2020-10-13 06:31:47 -07:00
Ken Hibino
50e7f38365 Add Scheduler
- Renamed previously called scheduler to forwarder to resolve name
  conflicts
2020-10-13 06:31:47 -07:00
Ken Hibino
a293efcdab Add Close to Inspector 2020-09-12 12:59:03 -07:00
Ken Hibino
cca680a7fd Change Client.Enqueue to take ProcessAt and ProcessIn as Option 2020-09-12 12:59:03 -07:00
Ken Hibino
dab8295883 Validate queue name in Inspector 2020-09-12 12:59:03 -07:00
Ken Hibino
131ac823fd Return error if queue name is empty when enqueueing 2020-09-12 12:59:03 -07:00
Ken Hibino
50b6034bf9 Move unique key generator function to base 2020-09-12 12:59:03 -07:00
Ken Hibino
34b90ecc8a Return Result struct to caller of Enqueue 2020-07-06 05:48:31 -07:00
Ken Hibino
8b60e6a268 Replace github.com/rs/xid with github.com/google/uuid 2020-07-06 05:48:31 -07:00
Ken Hibino
2e2c9b9f6b Update docs 2020-07-06 05:48:31 -07:00
Ken Hibino
7e942ec241 Use int64 type for Timeout and Deadline in TaskMessage 2020-07-06 05:48:31 -07:00
Ken Hibino
716d3d987e Use default timeout of 30mins if both timeout and deadline are not
provided
2020-07-06 05:48:31 -07:00