pkg/asynq
2
0
Fork 0
mirror of https://github.com/hibiken/asynq.git synced 2026-06-17 05:18:22 +08:00
Code Issues Projects Releases Wiki Activity
954 Commits 52 Branches 54 Tags
68f03688e30e9bf9ea5ec2ff5f20dfeec3758759
Commit Graph

954 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Erik Nilsen
68f03688e3 docs: add WARNING line for silent duplicate skipping in BatchEnqueue 2026-04-10 09:56:35 -07:00
Erik Nilsen
06a06970d6 Document atomicity guarantees and fix NOSCRIPT bug in BatchEnqueue 
Add comprehensive doc comments to BatchEnqueueContext, the Broker
interface method, and the RDB implementation explaining that the batch
uses a Redis pipeline (not MULTI/EXEC), so partial success is possible
and individual Lua scripts are atomic but the batch is not.

Fix a bug where Script.Run inside a pipeline only sends EVALSHA without
the automatic EVAL fallback that non-pipeline calls get. On a fresh
Redis (or after SCRIPT FLUSH), this caused NOSCRIPT errors for every
pipeline-batched script invocation. The fix preloads the required Lua
scripts before building the pipeline.

Also roll back the in-memory queuesPublished cache when the pipeline
fails, preventing stale entries from suppressing future SADD calls.
 2026-04-09 11:35:11 -07:00
Erik Nilsen
7ae0b3fe22 Add Options() accessor on Task for external option merging 
Exposes the opts field so callers can read a task's options to merge
with additional per-task options at batch time.
 2026-02-25 09:42:30 -08:00
Erik Nilsen
71ebcfa129 Fix BatchEnqueueContext time comparison and add scheduled task support 
BatchEnqueueContext had a time comparison bug where `now` was captured
before the loop but `processAt` was set to time.Now() inside
composeOptions during each iteration, causing all immediate tasks to be
incorrectly classified as scheduled and rejected.

Fix: move `now` capture inside the loop, after composeOptions.

Additionally, extend BatchEnqueueContext to support scheduled tasks in
the same pipeline. Tasks with a future ProcessAt are now routed to
scheduleCmd (ZADD to scheduled set) instead of being rejected. Only
unique and group tasks remain unsupported.

Changes:
- Add BatchEnqueueItem type pairing TaskMessage with optional ProcessAt
- Update Broker interface, RDB, and testbroker to use BatchEnqueueItem
- Route immediate tasks to enqueueCmd, scheduled tasks to scheduleCmd
- Return correct TaskState (Pending vs Scheduled) in results
- Add tests for immediate, scheduled, and mixed batch scenarios
 2026-02-25 09:42:30 -08:00
Erik Nilsen
4e62d7e29d Add tests for BatchEnqueue: multiple tasks, empty batch, duplicate IDs 2026-02-23 16:30:35 -08:00
Erik Nilsen
f919a605d5 Add BatchEnqueue for pipelined multi-task enqueue 
Adds BatchEnqueue to the Broker interface and RDB implementation that
sends multiple enqueueCmd Lua script invocations in a single Redis
pipeline round-trip. Also adds BatchEnqueueContext to the Client as
the public API, returning per-task results for partial-success handling.

Ref: hibiken/asynq#1069
 2026-02-23 16:25:49 -08:00
Mohamed Sohail
d704b68a42 Prepare release (docs): v0.26.0 (#1084) 
* pre-release: v0.26.0

* deps upgrades
* min go version set to 1.24.0

* feat: done add-username-cli (#1083)

* Feature: Add Headers Support to Tasks (#1070)

* feat(task): Add headers support to tasks

* fix: cleanup copy map code

* fix: Add tests

* Add --tls option to dash command (#1073)

* Add --tls option to dash command

* Switch order so it works better when both --tls and --tls_server are set

* docs: update CHANGELOG

---------

Co-authored-by: Artemii Kulikov <91570054+vlle@users.noreply.github.com>
Co-authored-by: Joe <85931983+joejoe-am@users.noreply.github.com>
Co-authored-by: Thomas Hansen <th4019@gmail.com>
v0.26.0 		2026-02-03 09:36:26 +03:00
Mohammed Sohail
a8db5b5571 docs: update CHANGELOG 2026-02-03 09:05:40 +03:00
Thomas Hansen
e4248e2749 Add --tls option to dash command (#1073) 
* Add --tls option to dash command

* Switch order so it works better when both --tls and --tls_server are set
 2026-02-03 09:05:40 +03:00
Joe
c4876e7247 Feature: Add Headers Support to Tasks (#1070) 
* feat(task): Add headers support to tasks

* fix: cleanup copy map code

* fix: Add tests
 2026-02-03 09:05:40 +03:00
Artemii Kulikov
dd2c3de356 feat: done add-username-cli (#1083) 2026-02-03 09:05:40 +03:00
Thomas Hansen
ff887e1f89 Add --tls option to dash command (#1073) 
* Add --tls option to dash command

* Switch order so it works better when both --tls and --tls_server are set
 2025-11-10 11:08:53 +03:00
Joe
5de9b1faf0 Feature: Add Headers Support to Tasks (#1070) 
* feat(task): Add headers support to tasks

* fix: cleanup copy map code

* fix: Add tests
 2025-11-04 20:07:59 +03:00
Artemii Kulikov
8261a03f0d feat: done add-username-cli (#1083) 2025-11-04 20:02:14 +03:00
Mohammed Sohail
74c47eb8bb pre-release: v0.26.0 
* deps upgrades
* min go version set to 1.24.0
 2025-11-04 19:42:48 +03:00
Mohammed Sohail
e9037f003d ci: prepare github ci for go 1.24.x and 1.25.x, turn off noisy linter 
* linter is producing uncessary noise
* benchstat is slow on PRs, we will turn it off untill it can be upgraded
 2025-11-04 19:29:49 +03:00
Mohammed Sohail
604175e6ca ci: format code with golangci-lint 2025-11-04 19:06:18 +03:00
AmirReza Fahimi
1831a07efe fix: correct error message text in ResultWriter.Write (#1054) 
Co-authored-by: amirreza.fahimidero <amirreza.fahimidero@snapp.cab>
Co-authored-by: Mohamed Sohail <sohailsameja@gmail.com>
 2025-11-04 18:28:08 +03:00
Benjamin Grosse
d64f0b7ed0 wrap all fmt.Errorf errors (#1047) 
Users need to be able to match with `errors.Is()` also on external
errors, for example `context.Canceled`.
 2025-11-04 18:25:47 +03:00
aziz-the-dev
a889ef0b08 Implement UpdateTaskPayload method for inspector (#1042) 
Co-authored-by: Aziz Aliyev <aziz.aliyev@idda.az>
 2025-11-04 18:25:10 +03:00
Marin Atanasov Nikolov
093ba04266 servemux: NotFoundHandler returns ErrHandlerNotFound error (#1031) 
This allows asynq middlewares to be used to inspect returned errors and decide
to silence errors about `handlers not found for task'.
 2025-11-04 18:17:50 +03:00
Khash Sajadi
c327bc40a2 docs: Update server.go (#1010) 
Typo in the docs
 2025-04-01 09:06:12 +03:00
Broderick Westrope
ea0c6e93f0 chore: fix godoc comment (#1009) 2025-04-01 09:05:18 +03:00
Mohammed Sohail
489e21920b release: v0.25.1 v0.25.1 2024-12-11 09:19:37 +03:00
Mohamed Sohail
043dcfbf56 fix: call Stop on all other signals to correctly set the server state for the shutdown procedure to complete successfully (#982) 
* fixes: #979
 2024-12-11 09:05:00 +03:00
Robin Joseph
02907551b4 feat(dash): Add --insecure option (#980) 2024-12-09 09:09:12 +03:00
Mohamed Sohail
127fac2e90 fix: NewScheduler incorrectly creates underlying Client, closing broker properly (#977) 
* fix: NewScheduler wrongly creates a client whose sharedConnection value is always true

* This is affecting the PeriodicManager as well as the Scheduler

* fix: closing the Client also closes the broker

* The error was also previously unhandled. For shared connections an error will be returned by the broker itself because the sharedConnection bool is also set on the client. This also means we can get rid of the sharedConnection flag on the Scheduler itself and let it work internally.
 2024-12-06 08:40:04 +03:00
dependabot[bot]
106c07adaa build(deps): bump codecov/codecov-action from 4 to 5 (#970) 
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4 to 5.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v4...v5)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-19 07:37:44 +03:00
dependabot[bot]
1c7195ff1a build(deps): bump google.golang.org/protobuf from 1.35.1 to 1.35.2 (#971) 
Bumps google.golang.org/protobuf from 1.35.1 to 1.35.2.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-19 07:37:13 +03:00
Xijun Dai
12cbba4926 feat(periodic_task_manager): Add RedisUniversalClient support (#958) 
Signed-off-by: Xijun Dai <daixijun1990@gmail.com>
 2024-11-13 14:48:56 +03:00
Khash Sajadi
80479b528d Include registration error in the log (#657) 
* Include registration error in the log

* remove chatty debug log

this will show in the logs every 5 seconds as debug (not even trace) which leads to a lot of noise
 2024-11-13 14:09:59 +03:00
dependabot[bot]
e14c312fe3 build(deps): bump golang.org/x/sys from 0.26.0 to 0.27.0 (#963) 
Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.26.0 to 0.27.0.
- [Commits](https://github.com/golang/sys/compare/v0.26.0...v0.27.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-12 08:38:50 +03:00
dependabot[bot]
ad1f587403 build(deps): bump golang.org/x/time from 0.7.0 to 0.8.0 (#964) 
Bumps [golang.org/x/time](https://github.com/golang/time) from 0.7.0 to 0.8.0.
- [Commits](https://github.com/golang/time/compare/v0.7.0...v0.8.0)

---
updated-dependencies:
- dependency-name: golang.org/x/time
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-12 08:13:01 +03:00
dependabot[bot]
8b32b38fd5 build(deps): bump github.com/mattn/go-runewidth in /tools (#965) 
Bumps [github.com/mattn/go-runewidth](https://github.com/mattn/go-runewidth) from 0.0.13 to 0.0.16.
- [Commits](https://github.com/mattn/go-runewidth/compare/v0.0.13...v0.0.16)

---
updated-dependencies:
- dependency-name: github.com/mattn/go-runewidth
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-12 08:12:43 +03:00
dependabot[bot]
96a84fac0c build(deps): bump github.com/hibiken/asynq in /tools (#966) 
Bumps [github.com/hibiken/asynq](https://github.com/hibiken/asynq) from 0.24.1 to 0.25.0.
- [Release notes](https://github.com/hibiken/asynq/releases)
- [Changelog](https://github.com/hibiken/asynq/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hibiken/asynq/compare/v0.24.1...v0.25.0)

---
updated-dependencies:
- dependency-name: github.com/hibiken/asynq
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-12 08:11:52 +03:00
ghosx
d2c207fbb8 fix: queues map init with size (#673) 
Co-authored-by: yipinhe <yipinhe@tencent.com>
 2024-11-11 08:25:42 +03:00
Pior Bastida
1a7c61ac49 Use string concat instead of fmt.Sprintf (#962) 2024-11-11 08:20:16 +03:00
dependabot[bot]
87375b5534 Bump codecov/codecov-action from 1 to 4 (#930) 
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 1 to 4.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1...v4)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-08 08:23:18 +03:00
dependabot[bot]
ffd75ebb5f Bump actions/upload-artifact from 3 to 4 (#929) 
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 3 to 4.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-08 08:23:10 +03:00
dependabot[bot]
d64fd328cb Bump actions/download-artifact from 3 to 4 (#931) 
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 3 to 4.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-08 08:23:00 +03:00
Pior Bastida
4f00f52c1d Add the scheduler option HeartbeatInterval (#956) 
* Add the scheduler option HeartbeatInterval

* Fix possible premature expiration of scheduler entries
 2024-11-07 08:34:28 +03:00
dependabot[bot]
580d69e88f build(deps): bump github.com/redis/go-redis/v9 in /tools (#957) 
Bumps [github.com/redis/go-redis/v9](https://github.com/redis/go-redis) from 9.0.5 to 9.7.0.
- [Release notes](https://github.com/redis/go-redis/releases)
- [Changelog](https://github.com/redis/go-redis/blob/master/CHANGELOG.md)
- [Commits](https://github.com/redis/go-redis/compare/v9.0.5...v9.7.0)

---
updated-dependencies:
- dependency-name: github.com/redis/go-redis/v9
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-05 07:59:30 +03:00
dependabot[bot]
c97652d408 build(deps): bump github.com/google/go-cmp from 0.5.9 to 0.6.0 in /tools (#954) 
Bumps [github.com/google/go-cmp](https://github.com/google/go-cmp) from 0.5.9 to 0.6.0.
- [Release notes](https://github.com/google/go-cmp/releases)
- [Commits](https://github.com/google/go-cmp/compare/v0.5.9...v0.6.0)

---
updated-dependencies:
- dependency-name: github.com/google/go-cmp
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-02 09:58:40 +03:00
dependabot[bot]
4644d37ef4 Bump github.com/prometheus/client_golang from 1.11.1 to 1.20.5 in /x (#934) 
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.11.1 to 1.20.5.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.11.1...v1.20.5)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-02 09:58:08 +03:00
dependabot[bot]
45c0fc6ad9 build(deps): bump github.com/hibiken/asynq from 0.24.1 to 0.25.0 in /x (#955) 
Bumps [github.com/hibiken/asynq](https://github.com/hibiken/asynq) from 0.24.1 to 0.25.0.
- [Release notes](https://github.com/hibiken/asynq/releases)
- [Changelog](https://github.com/hibiken/asynq/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hibiken/asynq/compare/v0.24.1...v0.25.0)

---
updated-dependencies:
- dependency-name: github.com/hibiken/asynq
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-02 08:16:01 +03:00
Mohamed Sohail
fd3eb86d95 release: v0.25.0 
* prepare release (docs): v0.25.0

* docs: add PR 946 to changelog

* docs: update issue templates, add releatively stable update

* Ths project should be considered relatively stable because we haven't broken the API in over 2 years.

* docs: add Redis Cluster compatibility caveat
v0.25.0 		2024-11-01 11:13:57 +03:00
Pior Bastida
3dbda60333 Improve performance of enqueueing tasks (#946) 
* Improve performance of enqueueing tasks

Add an in-memory cache to keep track of all the queues. Use this cache
to avoid sending an SADD since after the first call, that extra network
call isn't necessary.

The cache will expire every 10 secs so for cases where the queue is
deleted from asynq:queues set, it can be added again next time a task is
enqueued to it.

* Use sync.Map to simplify the conditional SADD

* Cleanup queuePublished in RemoveQueue

---------

Co-authored-by: Yousif <753751+yousifh@users.noreply.github.com>
 2024-10-30 08:25:35 +03:00
dependabot[bot]
02c6dae7eb Bump golang.org/x/time from 0.3.0 to 0.7.0 (#948) 
Bumps [golang.org/x/time](https://github.com/golang/time) from 0.3.0 to 0.7.0.
- [Commits](https://github.com/golang/time/compare/v0.3.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/time
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-10-29 10:12:44 +03:00
dependabot[bot]
5cfcb71139 Bump github.com/spf13/cast from 1.5.1 to 1.7.0 (#938) 
Bumps [github.com/spf13/cast](https://github.com/spf13/cast) from 1.5.1 to 1.7.0.
- [Release notes](https://github.com/spf13/cast/releases)
- [Commits](https://github.com/spf13/cast/compare/v1.5.1...v1.7.0)

---
updated-dependencies:
- dependency-name: github.com/spf13/cast
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-10-29 09:59:38 +03:00
dependabot[bot]
c78e7b0ccd Bump golang.org/x/sys from 0.16.0 to 0.26.0 (#933) 
Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.16.0 to 0.26.0.
- [Commits](https://github.com/golang/sys/compare/v0.16.0...v0.26.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-10-29 09:59:20 +03:00