Redis queue

Queueredis-queue

Redis list or stream used as a job queue by Sidekiq, Bull, RQ or Celery; loses popped jobs on worker crash.

Provider: default
Kind: Queue
Category: Messaging
Official docs: https://redis.io/docs/latest/develop/data-types/lists/

Properties · 5 set by this component

property	type	values	required	kind default	this component
backoff	enum	none \| fixed \| exponential \| exponential-jitter	no	—	inherited
deadLetterQueue	boolean	—	no	false	▸ false
delivery	enum	at-most-once \| at-least-once \| exactly-once	yes	—	▸ at-most-once
implementation · component property	enum	list \| stream	no	list	inherited
instances	integer	—	no	—	inherited
managed	boolean	—	no	false	inherited
maxAttempts	integer	—	no	—	inherited
ordering	enum	none \| per-key \| global	no	none	▸ global
persistence	boolean	—	no	true	▸ false
replay	boolean	—	no	false	▸ false
retentionDays	integer	—	no	—	inherited
role	enum	work \| dlq \| retry \| offline-mailbox \| frontier	no	work	inherited

Rules that apply · 3

Every rule whose selector covers this component's kind or its category. Whether a rule fires depends on the design: its properties, its edges and its groups.

Queue without dead-letter queueerror

queue-missing-dlq

A queue that receives asynchronous traffic and has no dead-letter queue either drops poison messages or retries them forever. Neither outcome leaves anything for a human to investigate.

— all of these hold:
— the node's `deadLetterQueue` is false
— the node has at least one asynchronous incoming edge

Retry queue without dead-letter queueerror

retry-queue-without-dlq

A retry queue is where poison messages end up. With no dead-letter queue they are retried forever or dropped in silence, and nobody ever looks at the failed payment.

— all of these hold:
— the node's `role` is "retry"
— the node's `deadLetterQueue` is false

Retry queue without backoffwarning

retry-without-backoff

Retrying a transient failure at full speed turns one slow dependency into a retry storm. A retry queue backs off, preferably exponentially and with jitter.

— all of these hold:
— the node's `role` is "retry"
— any of these hold:
— the node's `backoff` has no value
— the node's `backoff` is "none"

Use it in a design

nodes:
  - { id: "redis-queue", component: "redis-queue" }

Other Queue products · 23

Source

1# yaml-language-server: $schema=../../schema/component.schema.json
2# Redis used as a queue (LPUSH/BRPOP list, or a Stream via Sidekiq, Bull, RQ, Celery).
3# A popped list entry is gone if the worker dies, so delivery is at-most-once and nothing is persisted by default.
4schemaVersion: "1"
5type: "component"
6id: "redis-queue"
7kind: "queue"
8provider: "default"
9docs: "https://redis.io/docs/latest/develop/data-types/lists/"
10defaults:
11  delivery: "at-most-once"
12  ordering: "global"
13  deadLetterQueue: false
14  replay: false
15  persistence: false
16properties:
17  implementation:
18    type: "enum"
19    values: ["list", "stream"]
20    default: "list"

The same page as markdown, for an agent: /catalogue/components/redis-queue.md