Reconnect stampede without guard

errorreconnect-stampede-without-guard

When a gateway holding millions of sockets restarts, every client reconnects at once and the gateway asks the coordination service about each of them. Without a stampede guard that lookup storm takes the coordination service down too.

Severity: error
Selects: every node of kind realtime-gateway
Components in scope: 20

What it checks

It looks at every node of kind realtime-gateway, and fires when:

— all of these hold:
— the node has at least one synchronous outgoing edge to a coordination-service
— the node's `stampedeGuard` is false

Message

Realtime gateway '{{node.id}}' calls a coordination service synchronously on connect but has no reconnect stampede guard.

{{node.id}} style paths are filled in from the node, the edge or the design the finding is about.

Applies to · 20 components

Real-time gateway realtime-gateway

Tripped by · 5

Reference designs whose broken twin names this rule in its header. The gate reads the same header, and fails when a rule no design trips.

discord-messagesreference facebook-messengerreference nearby-friendsreference slack-messagingreference uber-dispatchreference

Source

1# yaml-language-server: $schema=../schema/rule.schema.json
2# When a gateway holding millions of sockets restarts, every client reconnects at once
3# and the gateway hits the coordination service (discovery, ring lookup) for each of them.
4# Without a stampede guard (jittered reconnect, connection admission) that lookup storm
5# takes the coordination service down with it.
6# Tripped by: designs/slack-messaging.broken.yaml
7schemaVersion: "1"
8type: "rule"
9id: "reconnect-stampede-without-guard"
10severity: "error"
11select:
12  node:
13    kind: "realtime-gateway"
14when:
15  all:
16    - hasEdge:
17        direction: "out"
18        mode: "sync"
19        peerKind: "coordination-service"
20    - prop: "stampedeGuard"
21      equals: false
22message: "Realtime gateway '{{node.id}}' calls a coordination service synchronously on connect but has no reconnect stampede guard."

The same page as markdown, for an agent: /catalogue/rules/reconnect-stampede-without-guard.md