2
0
mirror of https://github.com/hibiken/asynq.git synced 2026-06-12 14:26:15 +08:00
Commit Graph

62 Commits

Author SHA1 Message Date
Mohamed Sohail
785bb7208c Merge pull request #1094 from fanatics-live/batch-enqueue
Add BatchEnqueue for pipelined multi-task enqueue
2026-05-14 15:34:54 +03:00
Erik Nilsen
a32ac05d09 test: add coverage for BatchEnqueue error paths
Add tests for all uncovered BatchEnqueueContext error paths: nil task,
empty/blank typename, invalid options, group rejection, unique rejection,
and broker-down (sleeping broker). Add pipeline error test for rdb.BatchEnqueue
via cancelled context, and TestTaskOptions/TestTaskOptionsNil for the
new Options() accessor on Task.
2026-04-10 10:14:01 -07:00
kayoch1n
dbfdfbac5a chore: Update test cases 2026-03-17 12:56:12 +08:00
kayoch1n
5c391f3ffb feat: Add an option to specify headers 2026-03-17 12:29:58 +08:00
Erik Nilsen
71ebcfa129 Fix BatchEnqueueContext time comparison and add scheduled task support
BatchEnqueueContext had a time comparison bug where `now` was captured
before the loop but `processAt` was set to time.Now() inside
composeOptions during each iteration, causing all immediate tasks to be
incorrectly classified as scheduled and rejected.

Fix: move `now` capture inside the loop, after composeOptions.

Additionally, extend BatchEnqueueContext to support scheduled tasks in
the same pipeline. Tasks with a future ProcessAt are now routed to
scheduleCmd (ZADD to scheduled set) instead of being rejected. Only
unique and group tasks remain unsupported.

Changes:
- Add BatchEnqueueItem type pairing TaskMessage with optional ProcessAt
- Update Broker interface, RDB, and testbroker to use BatchEnqueueItem
- Route immediate tasks to enqueueCmd, scheduled tasks to scheduleCmd
- Return correct TaskState (Pending vs Scheduled) in results
- Add tests for immediate, scheduled, and mixed batch scenarios
2026-02-25 09:42:30 -08:00
Joe
5de9b1faf0 Feature: Add Headers Support to Tasks (#1070)
* feat(task): Add headers support to tasks

* fix: cleanup copy map code

* fix: Add tests
2025-11-04 20:07:59 +03:00
Pior Bastida
03f4799712 Run golangci-lint in CI (#927)
* Setup golangci-lint in CI and local-dev

* Fix linting error or locally disable linter
2024-10-26 08:48:12 +03:00
kanzihuang
ff7ef48463 fix: possible inconsistent scores between ProcessIn and ProcessAt (#876) 2024-10-19 09:45:18 +03:00
Jeroen Bobbeldijk
9e548fc097 Implement reusing redis client 2023-09-19 11:20:32 +02:00
Ken Hibino
e939b5d166 Rename asynqtest package to testutil 2022-04-11 16:55:43 -07:00
Ken Hibino
45ed560708 Add Group field to TaskInfo struct 2022-04-11 16:55:43 -07:00
Ken Hibino
562506c7ba Fix client to return error when nil task is passed 2022-04-11 16:55:43 -07:00
Ken Hibino
4e8885276c Update client to store groupKey under TaskMessage 2022-04-11 16:55:43 -07:00
Ken Hibino
f17c157b0f Update Client to add task to group if Group option is specified 2022-04-11 16:55:43 -07:00
Ken Hibino
e2b61c9056 Return error if Unique TTL is less than 1s 2021-11-09 16:37:02 -08:00
Ken Hibino
f4ddac4dcc Introduce Task Results
* Added Retention Option to specify retention TTL for tasks
* Added ResultWriter as a client interface to write result data for the associated task
2021-11-06 15:18:42 -07:00
Ken Hibino
9e2f88c00d Add TaskID option to allow user to specify task id 2021-11-06 15:18:42 -07:00
Ken Hibino
83cae4bb24 Update NewTask function to take Option as varargs 2021-11-06 15:18:42 -07:00
Jason White
b3ef9e91a9 Upgrade go-redis/redis to version 8 2021-09-02 05:56:02 -07:00
Luqqk
95c90a5cb8 Add changelog entry, add additional test case 2021-08-02 20:20:09 -07:00
Luqqk
4bce28d677 client.Enqueue - prevent empty task's typename 2021-08-02 20:20:09 -07:00
Ken Hibino
46b23d6495 Allow upper case characters in queue name 2021-07-15 06:55:47 -07:00
Ken Hibino
a9feec5967 Change TaskInfo to use public fields instead of methods 2021-06-29 16:34:21 -07:00
Ken Hibino
b835090ad8 Update Client.Enqueue to return TaskInfo 2021-06-29 16:34:21 -07:00
Ken Hibino
476812475e Change payload to byte slice 2021-06-29 16:34:21 -07:00
Ken Hibino
7af3981929 Refactor redis keys and store messages in protobuf
Changes:
- Task messages are stored under "asynq:{<qname>}:t:<task_id>" key in redis, value is a HASH type and message are stored under "msg" key in the hash. The hash also stores "deadline", "timeout".
- Redis LIST and ZSET stores task message IDs
- Task messages are serialized using protocol buffer
2021-06-29 16:34:21 -07:00
Ken Hibino
e13122723a Move all inspector related code to subpackage inspeq 2021-01-31 06:09:40 -08:00
Ken Hibino
b604d25937 Add helper function to parse Option string 2021-01-14 06:43:44 -08:00
Ken Hibino
50e7f38365 Add Scheduler
- Renamed previously called scheduler to forwarder to resolve name
  conflicts
2020-10-13 06:31:47 -07:00
Ken Hibino
69d7ec725a Close redis client after each test run 2020-09-12 12:59:03 -07:00
Ken Hibino
c8c47fcbf0 Rename Enqueued to Pending 2020-09-12 12:59:03 -07:00
Ken Hibino
cca680a7fd Change Client.Enqueue to take ProcessAt and ProcessIn as Option 2020-09-12 12:59:03 -07:00
Ken Hibino
131ac823fd Return error if queue name is empty when enqueueing 2020-09-12 12:59:03 -07:00
Ken Hibino
6b96459881 Add test flags to run tests using redis cluster 2020-09-12 12:59:03 -07:00
Ken Hibino
4037b41479 Fix client tests 2020-09-12 12:59:03 -07:00
Ken Hibino
3f26122ac0 Fix more build errors 2020-09-12 12:59:03 -07:00
Ken Hibino
50b6034bf9 Move unique key generator function to base 2020-09-12 12:59:03 -07:00
Ken Hibino
a28f61f313 Add Inspector type 2020-07-28 22:45:57 -07:00
Ken Hibino
34b90ecc8a Return Result struct to caller of Enqueue 2020-07-06 05:48:31 -07:00
Ken Hibino
7e942ec241 Use int64 type for Timeout and Deadline in TaskMessage 2020-07-06 05:48:31 -07:00
Ken Hibino
716d3d987e Use default timeout of 30mins if both timeout and deadline are not
provided
2020-07-06 05:48:31 -07:00
Ken Hibino
e33d297d8e Add SetDefaultOptions method to Client 2020-04-27 06:45:13 -07:00
Ken Hibino
c33dd447ac Allow client to enqueue a task with unique option
Changes:

- Added Unique option for clients
- Require go v.13 or above (to use new errors wrapping functions)
- Fixed adding queue key to all-queues set (asynq:queues) when scheduling.
2020-03-21 11:40:40 -07:00
Ken Hibino
9c79a7d507 Simplify code with gofmt -s 2020-03-13 14:24:24 -07:00
Ken Hibino
3c722386b0 Add Deadline option when enqueuing tasks
Deadline option sets the deadline for the given task's context deadline.
2020-03-08 17:12:42 -07:00
Ken Hibino
49c117f4d1 Change Client APIs
Use `Enqueue`, `EnqueueAt`, and `EnqueueIn` to enqueue and schedule
tasks.
`Schedule` method was removed.
2020-02-23 20:40:40 -08:00
Ken Hibino
e21fe3bd79 Add ListWorkers to RDB 2020-02-22 21:21:55 -08:00
Ken Hibino
39459b4412 Allow user to specify timeout per task 2020-02-13 06:23:51 -08:00
Ken Hibino
f66a65d6ca Fix tests 2020-01-16 21:04:46 -08:00
Ken Hibino
97316d6766 Fix flaky tests
Some tests were failing due to mismatch in Score in ZSetEntry.
Changed ZSetEntry Score to float64 type so that we can use
cmpopts.EquateApprox to allow for margin when comparing.
2020-01-11 10:09:15 -08:00