2
0
mirror of https://github.com/hibiken/asynq.git synced 2026-07-11 12:01:49 +08:00
Commit Graph

132 Commits

Author SHA1 Message Date
Erik Nilsen
06a06970d6 Document atomicity guarantees and fix NOSCRIPT bug in BatchEnqueue
Add comprehensive doc comments to BatchEnqueueContext, the Broker
interface method, and the RDB implementation explaining that the batch
uses a Redis pipeline (not MULTI/EXEC), so partial success is possible
and individual Lua scripts are atomic but the batch is not.

Fix a bug where Script.Run inside a pipeline only sends EVALSHA without
the automatic EVAL fallback that non-pipeline calls get. On a fresh
Redis (or after SCRIPT FLUSH), this caused NOSCRIPT errors for every
pipeline-batched script invocation. The fix preloads the required Lua
scripts before building the pipeline.

Also roll back the in-memory queuesPublished cache when the pipeline
fails, preventing stale entries from suppressing future SADD calls.
2026-04-09 11:35:11 -07:00
Erik Nilsen
71ebcfa129 Fix BatchEnqueueContext time comparison and add scheduled task support
BatchEnqueueContext had a time comparison bug where `now` was captured
before the loop but `processAt` was set to time.Now() inside
composeOptions during each iteration, causing all immediate tasks to be
incorrectly classified as scheduled and rejected.

Fix: move `now` capture inside the loop, after composeOptions.

Additionally, extend BatchEnqueueContext to support scheduled tasks in
the same pipeline. Tasks with a future ProcessAt are now routed to
scheduleCmd (ZADD to scheduled set) instead of being rejected. Only
unique and group tasks remain unsupported.

Changes:
- Add BatchEnqueueItem type pairing TaskMessage with optional ProcessAt
- Update Broker interface, RDB, and testbroker to use BatchEnqueueItem
- Route immediate tasks to enqueueCmd, scheduled tasks to scheduleCmd
- Return correct TaskState (Pending vs Scheduled) in results
- Add tests for immediate, scheduled, and mixed batch scenarios
2026-02-25 09:42:30 -08:00
Erik Nilsen
f919a605d5 Add BatchEnqueue for pipelined multi-task enqueue
Adds BatchEnqueue to the Broker interface and RDB implementation that
sends multiple enqueueCmd Lua script invocations in a single Redis
pipeline round-trip. Also adds BatchEnqueueContext to the Client as
the public API, returning per-task results for partial-success handling.

Ref: hibiken/asynq#1069
2026-02-23 16:25:49 -08:00
Joe
c4876e7247 Feature: Add Headers Support to Tasks (#1070)
* feat(task): Add headers support to tasks

* fix: cleanup copy map code

* fix: Add tests
2026-02-03 09:05:40 +03:00
Mohammed Sohail
74c47eb8bb pre-release: v0.26.0
* deps upgrades
* min go version set to 1.24.0
2025-11-04 19:42:48 +03:00
Mohammed Sohail
489e21920b release: v0.25.1 2024-12-11 09:19:37 +03:00
ghosx
d2c207fbb8 fix: queues map init with size (#673)
Co-authored-by: yipinhe <yipinhe@tencent.com>
2024-11-11 08:25:42 +03:00
Pior Bastida
1a7c61ac49 Use string concat instead of fmt.Sprintf (#962) 2024-11-11 08:20:16 +03:00
Mohamed Sohail
fd3eb86d95 release: v0.25.0
* prepare release (docs): v0.25.0

* docs: add PR 946 to changelog

* docs: update issue templates, add releatively stable update

* Ths project should be considered relatively stable because we haven't broken the API in over 2 years.

* docs: add Redis Cluster compatibility caveat
2024-11-01 11:13:57 +03:00
Ahmed Radwan
e70de721b8 remove deprecated protobuf ptypes (#942)
* remove deprecated protobuf ptypes

* tidy compiled proto and go mod

* bump protobuf
2024-10-29 09:21:27 +03:00
Pior Bastida
03f4799712 Run golangci-lint in CI (#927)
* Setup golangci-lint in CI and local-dev

* Fix linting error or locally disable linter
2024-10-26 08:48:12 +03:00
Tedja
d04888e748 feature: configurable janitor interval and deletion batch size (#715)
* feature: configurable janitor interval and deletion batch size

* warn user when they set a big number of janitor batch size

* Update CHANGELOG.md

---------

Co-authored-by: Agung Hariadi Tedja <agung.tedja@kumparan.com>
2024-05-06 14:11:52 +08:00
Ken Hibino
fde294be32 v0.24.1 2023-05-01 06:48:07 -07:00
Emanuel Bennici
0275df8df4 Update redis/go-redis to v9
Version v9 implements the support for Redis v7 and has some
other improvements.
2023-04-17 22:30:33 -07:00
Ken Hibino
783071c47f v0.24.0 2023-01-02 14:55:33 -08:00
Trịnh Đức Bảo Linh
30d409371b Fix comment typos 2022-05-16 21:14:15 -07:00
Ken Hibino
245d4fe663 v0.23.0 2022-04-11 16:57:33 -07:00
Ken Hibino
8b16ede8bc Declare ReclaimStaleAggregationSets 2022-04-11 16:55:43 -07:00
Ken Hibino
888b5590fb Make GroupMaxSize and GroupMaxDelay config optional 2022-04-11 16:55:43 -07:00
Ken Hibino
99c00bffeb Implement RDB.AggregationCheck 2022-04-11 16:55:43 -07:00
Ken Hibino
d841dc2f8d Add initial implementation of aggregator 2022-04-11 16:55:43 -07:00
Ken Hibino
401f7fb4fe Add GroupKey field to TaskMessage 2022-04-11 16:55:43 -07:00
Ken Hibino
61854ea1dc Update RDB.ForwardIfReady to forward to group if groupKey is specified 2022-04-11 16:55:43 -07:00
Ken Hibino
f17c157b0f Update Client to add task to group if Group option is specified 2022-04-11 16:55:43 -07:00
Ken Hibino
e3d2939a4c Add helper functions to generate group key 2022-04-11 16:55:43 -07:00
Ken Hibino
c04fd41653 v0.22.1 2022-02-20 06:22:55 -08:00
Ken Hibino
a19909f5f4 v0.22.0 2022-02-19 06:20:05 -08:00
Ken Hibino
d4006894ad Remove base.DeadlinesKey 2022-02-19 06:15:44 -08:00
Ken Hibino
d7169cd445 Update heartbeat to extend lease of active workers 2022-02-19 06:15:44 -08:00
Ken Hibino
dfae8638e1 Update RDB methods to work with lease 2022-02-19 06:15:44 -08:00
Ken Hibino
b9943de2ab Add Lease type to base package 2022-02-19 06:15:44 -08:00
Ken Hibino
87dc392c7f Add RDB.ExtendLease method 2022-02-19 06:15:44 -08:00
Ken Hibino
dabcb120d5 Update recoverer to use ListLeaseExpired 2022-02-19 06:15:44 -08:00
Ken Hibino
d865d89900 Update RDB.Dequeue to insert task ID to lease set 2022-02-19 06:15:44 -08:00
Ken Hibino
852af7abd1 Add base.LeaseKey helper function 2022-02-19 06:15:44 -08:00
Ken Hibino
dff2e3a336 v0.21.0 2022-01-22 06:15:29 -08:00
Ken Hibino
25832e5e95 Fix bug related to concurrently executing server state changes 2022-01-12 09:10:56 -08:00
Ken Hibino
5f20edcbd1 v0.20.0 2021-12-19 07:00:21 -08:00
Ken Hibino
1ddb2f7bce Use math.MaxInt64 instead of custom const 2021-12-19 06:58:12 -08:00
Ken Hibino
82d18e3d91 Record total tasks processed/failed 2021-12-16 16:53:02 -08:00
Ken Hibino
970cb7a606 v0.19.1 2021-12-12 06:16:13 -08:00
Ken Hibino
9f2c321e98 Add EnqueueContext method to Client 2021-11-15 16:34:26 -08:00
Ken Hibino
413afc2ab6 v0.19.0 2021-11-06 15:20:09 -07:00
Ken Hibino
f4ddac4dcc Introduce Task Results
* Added Retention Option to specify retention TTL for tasks
* Added ResultWriter as a client interface to write result data for the associated task
2021-11-06 15:18:42 -07:00
Ken Hibino
2261c7c9a0 Change TaskMessage.ID type from uuid.UUID to string 2021-11-06 15:18:42 -07:00
Ken Hibino
d612a8a9e4 v0.18.6 2021-10-03 05:55:49 -07:00
Jason White
b3ef9e91a9 Upgrade go-redis/redis to version 8 2021-09-02 05:56:02 -07:00
Ken Hibino
05534c6f24 v0.18.5 2021-09-01 06:02:49 -07:00
Ken Hibino
f0db219f6a Add IsFailure to Config
With this IsFailure config, users can provide a predicate function to 
determine whether the error returned from Handler counts as a failure.
2021-09-01 06:00:54 -07:00
Ken Hibino
421dc584ff v0.18.4 2021-08-17 17:12:33 -07:00