Push fan-out without a celebrity threshold

warningpush-fanout-without-celebrity-path

Pure push fan-out over a social graph is brutal at the tail: one post from an account with millions of followers becomes millions of cache writes. A follower threshold above which readers merge at read time bounds that write amplification.

Severity: warning
Selects: every node of kind worker
Components in scope: 83

What it checks

It looks at every node of kind worker, and fires when:

— all of these hold:
— the node's `fanoutStrategy` is "push"
— the node's `fanoutThreshold` has no value
— the design contains a node of kind graph-database

Message

Worker '{{node.id}}' pushes fan-out over a social graph with no fanoutThreshold. Declare a follower threshold above which timelines are merged at read time.

{{node.id}} style paths are filled in from the node, the edge or the design the finding is about.

Applies to · 83 components

Worker worker

Tripped by · 2

Reference designs whose broken twin names this rule in its header. The gate reads the same header, and fails when a rule no design trips.

meta-threadsreference twitter-timelinereference

Source

1# yaml-language-server: $schema=../schema/rule.schema.json
2# Pure push fan-out over a social graph is brutal at the tail: one post by an account with
3# millions of followers becomes millions of cache writes. A threshold above which the worker
4# stops pushing and readers merge at read time bounds that write amplification.
5# Tripped by: designs/twitter-timeline.broken.yaml
6schemaVersion: "1"
7type: "rule"
8id: "push-fanout-without-celebrity-path"
9severity: "warning"
10select:
11  node:
12    kind: "worker"
13when:
14  all:
15    - prop: "fanoutStrategy"
16      equals: "push"
17    - prop: "fanoutThreshold"
18      exists: false
19    - hasNode:
20        kind: "graph-database"
21message: "Worker '{{node.id}}' pushes fan-out over a social graph with no fanoutThreshold. Declare a follower threshold above which timelines are merged at read time."

The same page as markdown, for an agent: /catalogue/rules/push-fanout-without-celebrity-path.md