Retry queue without backoff

warningretry-without-backoff

Retrying a transient failure at full speed turns one slow dependency into a retry storm. A retry queue backs off, preferably exponentially and with jitter.

Severity: warning
Selects: every node of kind queue
Components in scope: 24

What it checks

It looks at every node of kind queue, and fires when:

— all of these hold:
— the node's `role` is "retry"
— any of these hold:
— the node's `backoff` has no value
— the node's `backoff` is "none"

Message

Retry queue '{{node.id}}' declares no backoff. Retries without backoff amplify the outage they are retrying through.

{{node.id}} style paths are filled in from the node, the edge or the design the finding is about.

Applies to · 24 components

Queue queue

Tripped by · 4

Reference designs whose broken twin names this rule in its header. The gate reads the same header, and fails when a rule no design trips.

distributed-emailreference meta-threadsreference payment-systemreference robinhood-tradingreference

Source

1# yaml-language-server: $schema=../schema/rule.schema.json
2# Retrying a transient failure at full speed turns one slow dependency into a retry
3# storm. A retry queue must back off, preferably exponentially with jitter.
4# Tripped by: designs/payment-system.broken.yaml
5schemaVersion: "1"
6type: "rule"
7id: "retry-without-backoff"
8severity: "warning"
9select:
10  node:
11    kind: "queue"
12when:
13  all:
14    - prop: "role"
15      equals: "retry"
16    - any:
17        - prop: "backoff"
18          exists: false
19        - prop: "backoff"
20          equals: "none"
21message: "Retry queue '{{node.id}}' declares no backoff. Retries without backoff amplify the outage they are retrying through."

The same page as markdown, for an agent: /catalogue/rules/retry-without-backoff.md