2
0
mirror of https://github.com/hibiken/asynq.git synced 2026-06-15 10:38:01 +08:00
Commit Graph

971 Commits

Author SHA1 Message Date
dependabot[bot]
1fad76c014 build(deps): bump github.com/redis/go-redis/v9 from 9.14.1 to 9.20.0
Bumps [github.com/redis/go-redis/v9](https://github.com/redis/go-redis) from 9.14.1 to 9.20.0.
- [Release notes](https://github.com/redis/go-redis/releases)
- [Changelog](https://github.com/redis/go-redis/blob/master/RELEASE-NOTES.md)
- [Commits](https://github.com/redis/go-redis/compare/v9.14.1...v9.20.0)

---
updated-dependencies:
- dependency-name: github.com/redis/go-redis/v9
  dependency-version: 9.20.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-03 16:01:15 +00:00
Mohamed Sohail
785bb7208c Merge pull request #1094 from fanatics-live/batch-enqueue
Add BatchEnqueue for pipelined multi-task enqueue
2026-05-14 15:34:54 +03:00
Mohamed Sohail
3a7ec97482 Merge pull request #1096 from kayoch1n/feat/header-option
feat: Add an option to specify headers
2026-05-14 15:33:28 +03:00
Mohamed Sohail
d671a4c427 Merge pull request #1100 from ljluestc/fix/669-sentinel-db-parsing
fix: parse DB number from redis-sentinel URI path
2026-05-14 15:32:15 +03:00
Mohamed Sohail
f81d452160 Merge pull request #1101 from sevico/refactor
refactor(cli): migrate commands from Run to RunE and fix group error …
2026-05-14 15:31:49 +03:00
kayoch1n
1af926b147 chore: Add docs string to asynq.Header and remove asynq.Headers 2026-04-15 20:02:30 +08:00
Mohamed Sohail
23905a286f Merge pull request #1104 from Bahtya/fix/pubsub-connection-leak
fix: close pubsub connection on Subscribe error in CancelationPubSub
2026-04-13 13:58:23 +03:00
Erik Nilsen
a32ac05d09 test: add coverage for BatchEnqueue error paths
Add tests for all uncovered BatchEnqueueContext error paths: nil task,
empty/blank typename, invalid options, group rejection, unique rejection,
and broker-down (sleeping broker). Add pipeline error test for rdb.BatchEnqueue
via cancelled context, and TestTaskOptions/TestTaskOptionsNil for the
new Options() accessor on Task.
2026-04-10 10:14:01 -07:00
Erik Nilsen
68f03688e3 docs: add WARNING line for silent duplicate skipping in BatchEnqueue 2026-04-10 09:56:35 -07:00
Erik Nilsen
06a06970d6 Document atomicity guarantees and fix NOSCRIPT bug in BatchEnqueue
Add comprehensive doc comments to BatchEnqueueContext, the Broker
interface method, and the RDB implementation explaining that the batch
uses a Redis pipeline (not MULTI/EXEC), so partial success is possible
and individual Lua scripts are atomic but the batch is not.

Fix a bug where Script.Run inside a pipeline only sends EVALSHA without
the automatic EVAL fallback that non-pipeline calls get. On a fresh
Redis (or after SCRIPT FLUSH), this caused NOSCRIPT errors for every
pipeline-batched script invocation. The fix preloads the required Lua
scripts before building the pipeline.

Also roll back the in-memory queuesPublished cache when the pipeline
fails, preventing stale entries from suppressing future SADD calls.
2026-04-09 11:35:11 -07:00
Bahtya
5586efeae7 test: add test for CancelationPubSub error path
Add TestCancelationPubSubReceiveError to verify that when
Receive() fails in CancelationPubSub(), an error is returned
and the pubsub connection is not leaked.

This provides test coverage for the pubsub.Close() fix that
was missing in the previous commit.

Bahtya
2026-04-09 20:11:53 +08:00
bahtya
dd3c923f44 fix: close pubsub connection on Subscribe error in CancelationPubSub
When redis.Subscribe succeeds but Receive() fails, the pubsub
connection was not being closed before returning the error. This caused
the subscriber goroutine to leak a Redis connection on each retry
iteration, eventually exhausting the connection pool.

Fixes #1095
2026-04-09 18:15:48 +08:00
Mohamed Sohail
f81c78e68d Merge pull request #1092 from NilPuig/fix/memory-usage-nil-guard
Fix nil panic in memoryUsageCmd Lua script
2026-04-09 10:08:50 +03:00
shiweikang
f8d6677814 refactor(cli): migrate commands from Run to RunE and fix group error handling
Migrate all CLI command handlers from cobra's Run to RunE, replacing
fmt.Println(err) + os.Exit(1) patterns with idiomatic error returns.
This improves testability and lets cobra handle errors consistently
via the existing SilenceUsage/SilenceErrors configuration.

Also refactor getDuration() and getTime() helpers to return errors
instead of calling os.Exit(1).

Fix a bug in group.go where the error from inspector.Groups() was
never checked, causing silent failures (e.g. on Redis connection
errors) to be misreported as "No groups found".
2026-04-05 01:13:41 +08:00
Jiale Lin
07898eade0 fix: parse DB number from redis-sentinel URI path (#669)
Previously, parseRedisSentinelURI ignored the /dbnumber path segment,
making it impossible to connect to any DB other than 0 via sentinel URIs.

This adds DB extraction from the URI path, consistent with how
parseRedisURI already handles it for redis:// and rediss:// schemes.

Closes #669
2026-03-21 11:33:56 -07:00
kayoch1n
5216f1c3be feat: Add an option to create headers from a map 2026-03-17 14:47:34 +08:00
kayoch1n
dbfdfbac5a chore: Update test cases 2026-03-17 12:56:12 +08:00
kayoch1n
5c391f3ffb feat: Add an option to specify headers 2026-03-17 12:29:58 +08:00
Erik Nilsen
7ae0b3fe22 Add Options() accessor on Task for external option merging
Exposes the opts field so callers can read a task's options to merge
with additional per-task options at batch time.
2026-02-25 09:42:30 -08:00
Erik Nilsen
71ebcfa129 Fix BatchEnqueueContext time comparison and add scheduled task support
BatchEnqueueContext had a time comparison bug where `now` was captured
before the loop but `processAt` was set to time.Now() inside
composeOptions during each iteration, causing all immediate tasks to be
incorrectly classified as scheduled and rejected.

Fix: move `now` capture inside the loop, after composeOptions.

Additionally, extend BatchEnqueueContext to support scheduled tasks in
the same pipeline. Tasks with a future ProcessAt are now routed to
scheduleCmd (ZADD to scheduled set) instead of being rejected. Only
unique and group tasks remain unsupported.

Changes:
- Add BatchEnqueueItem type pairing TaskMessage with optional ProcessAt
- Update Broker interface, RDB, and testbroker to use BatchEnqueueItem
- Route immediate tasks to enqueueCmd, scheduled tasks to scheduleCmd
- Return correct TaskState (Pending vs Scheduled) in results
- Add tests for immediate, scheduled, and mixed batch scenarios
2026-02-25 09:42:30 -08:00
Erik Nilsen
4e62d7e29d Add tests for BatchEnqueue: multiple tasks, empty batch, duplicate IDs 2026-02-23 16:30:35 -08:00
Erik Nilsen
f919a605d5 Add BatchEnqueue for pipelined multi-task enqueue
Adds BatchEnqueue to the Broker interface and RDB implementation that
sends multiple enqueueCmd Lua script invocations in a single Redis
pipeline round-trip. Also adds BatchEnqueueContext to the Client as
the public API, returning per-task results for partial-success handling.

Ref: hibiken/asynq#1069
2026-02-23 16:25:49 -08:00
Nil
2fd155e31d Fix nil guard for MEMORY USAGE in memoryUsageCmd Lua script
MEMORY USAGE returns nil for keys that no longer exist (e.g., expired
or deleted task keys). In Lua, nil is converted to false (a boolean).
The script then attempts arithmetic on this boolean value, causing:

  ERR user_script:30: attempt to perform arithmetic on local 'bytes'
  (a boolean value)

This breaks the /api/queues endpoint in asynqmon, showing "Could not
retrieve queues live data" in the UI.

The fix adds nil guards around all three MEMORY USAGE calls on task
keys, and a divide-by-zero guard on agg_task_sample_size.

Tested in production with Redis 7.2 and asynq v0.25.1 worker.

Fixes #728
Related to #901
2026-02-07 15:13:58 +08:00
Mohamed Sohail
d704b68a42 Prepare release (docs): v0.26.0 (#1084)
* pre-release: v0.26.0

* deps upgrades
* min go version set to 1.24.0

* feat: done add-username-cli (#1083)

* Feature: Add Headers Support to Tasks (#1070)

* feat(task): Add headers support to tasks

* fix: cleanup copy map code

* fix: Add tests

* Add --tls option to dash command (#1073)

* Add --tls option to dash command

* Switch order so it works better when both --tls and --tls_server are set

* docs: update CHANGELOG

---------

Co-authored-by: Artemii Kulikov <91570054+vlle@users.noreply.github.com>
Co-authored-by: Joe <85931983+joejoe-am@users.noreply.github.com>
Co-authored-by: Thomas Hansen <th4019@gmail.com>
v0.26.0
2026-02-03 09:36:26 +03:00
Mohammed Sohail
a8db5b5571 docs: update CHANGELOG 2026-02-03 09:05:40 +03:00
Thomas Hansen
e4248e2749 Add --tls option to dash command (#1073)
* Add --tls option to dash command

* Switch order so it works better when both --tls and --tls_server are set
2026-02-03 09:05:40 +03:00
Joe
c4876e7247 Feature: Add Headers Support to Tasks (#1070)
* feat(task): Add headers support to tasks

* fix: cleanup copy map code

* fix: Add tests
2026-02-03 09:05:40 +03:00
Artemii Kulikov
dd2c3de356 feat: done add-username-cli (#1083) 2026-02-03 09:05:40 +03:00
Thomas Hansen
ff887e1f89 Add --tls option to dash command (#1073)
* Add --tls option to dash command

* Switch order so it works better when both --tls and --tls_server are set
2025-11-10 11:08:53 +03:00
Joe
5de9b1faf0 Feature: Add Headers Support to Tasks (#1070)
* feat(task): Add headers support to tasks

* fix: cleanup copy map code

* fix: Add tests
2025-11-04 20:07:59 +03:00
Artemii Kulikov
8261a03f0d feat: done add-username-cli (#1083) 2025-11-04 20:02:14 +03:00
Mohammed Sohail
74c47eb8bb pre-release: v0.26.0
* deps upgrades
* min go version set to 1.24.0
2025-11-04 19:42:48 +03:00
Mohammed Sohail
e9037f003d ci: prepare github ci for go 1.24.x and 1.25.x, turn off noisy linter
* linter is producing uncessary noise
* benchstat is slow on PRs, we will turn it off untill it can be upgraded
2025-11-04 19:29:49 +03:00
Mohammed Sohail
604175e6ca ci: format code with golangci-lint 2025-11-04 19:06:18 +03:00
AmirReza Fahimi
1831a07efe fix: correct error message text in ResultWriter.Write (#1054)
Co-authored-by: amirreza.fahimidero <amirreza.fahimidero@snapp.cab>
Co-authored-by: Mohamed Sohail <sohailsameja@gmail.com>
2025-11-04 18:28:08 +03:00
Benjamin Grosse
d64f0b7ed0 wrap all fmt.Errorf errors (#1047)
Users need to be able to match with `errors.Is()` also on external
errors, for example `context.Canceled`.
2025-11-04 18:25:47 +03:00
aziz-the-dev
a889ef0b08 Implement UpdateTaskPayload method for inspector (#1042)
Co-authored-by: Aziz Aliyev <aziz.aliyev@idda.az>
2025-11-04 18:25:10 +03:00
Marin Atanasov Nikolov
093ba04266 servemux: NotFoundHandler returns ErrHandlerNotFound error (#1031)
This allows asynq middlewares to be used to inspect returned errors and decide
to silence errors about `handlers not found for task'.
2025-11-04 18:17:50 +03:00
Khash Sajadi
c327bc40a2 docs: Update server.go (#1010)
Typo in the docs
2025-04-01 09:06:12 +03:00
Broderick Westrope
ea0c6e93f0 chore: fix godoc comment (#1009) 2025-04-01 09:05:18 +03:00
Mohammed Sohail
489e21920b release: v0.25.1 v0.25.1 2024-12-11 09:19:37 +03:00
Mohamed Sohail
043dcfbf56 fix: call Stop on all other signals to correctly set the server state for the shutdown procedure to complete successfully (#982)
* fixes: #979
2024-12-11 09:05:00 +03:00
Robin Joseph
02907551b4 feat(dash): Add --insecure option (#980) 2024-12-09 09:09:12 +03:00
Mohamed Sohail
127fac2e90 fix: NewScheduler incorrectly creates underlying Client, closing broker properly (#977)
* fix: NewScheduler wrongly creates a client whose sharedConnection value is always true

* This is affecting the PeriodicManager as well as the Scheduler

* fix: closing the Client also closes the broker

* The error was also previously unhandled. For shared connections an error will be returned by the broker itself because the sharedConnection bool is also set on the client. This also means we can get rid of the sharedConnection flag on the Scheduler itself and let it work internally.
2024-12-06 08:40:04 +03:00
dependabot[bot]
106c07adaa build(deps): bump codecov/codecov-action from 4 to 5 (#970)
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4 to 5.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v4...v5)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-19 07:37:44 +03:00
dependabot[bot]
1c7195ff1a build(deps): bump google.golang.org/protobuf from 1.35.1 to 1.35.2 (#971)
Bumps google.golang.org/protobuf from 1.35.1 to 1.35.2.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-19 07:37:13 +03:00
Xijun Dai
12cbba4926 feat(periodic_task_manager): Add RedisUniversalClient support (#958)
Signed-off-by: Xijun Dai <daixijun1990@gmail.com>
2024-11-13 14:48:56 +03:00
Khash Sajadi
80479b528d Include registration error in the log (#657)
* Include registration error in the log

* remove chatty debug log

this will show in the logs every 5 seconds as debug (not even trace) which leads to a lot of noise
2024-11-13 14:09:59 +03:00
dependabot[bot]
e14c312fe3 build(deps): bump golang.org/x/sys from 0.26.0 to 0.27.0 (#963)
Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.26.0 to 0.27.0.
- [Commits](https://github.com/golang/sys/compare/v0.26.0...v0.27.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-12 08:38:50 +03:00
dependabot[bot]
ad1f587403 build(deps): bump golang.org/x/time from 0.7.0 to 0.8.0 (#964)
Bumps [golang.org/x/time](https://github.com/golang/time) from 0.7.0 to 0.8.0.
- [Commits](https://github.com/golang/time/compare/v0.7.0...v0.8.0)

---
updated-dependencies:
- dependency-name: golang.org/x/time
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-12 08:13:01 +03:00