Several AI vendors and no gateway

infoai-no-gateway

With three or more AI vendors and no gateway, every caller holds every key, writes its own retries and fallbacks, and spend is spread across vendor dashboards. An AI gateway such as LiteLLM, Portkey or Cloudflare AI Gateway gives one place for keys, fallbacks, rate limits and cost.

Severity: info
Selects: every design
Components in scope: 2,366

What it checks

It looks at every design, and fires when:

— all of these hold:
— the design contains more than 2 nodes of kind ai-service
— this does not hold:
— the design contains a node of kind api-gateway

Message

Design '{{design.id}}' calls three or more AI services and has no API gateway to route, retry and meter them.

{{node.id}} style paths are filled in from the node, the edge or the design the finding is about.

Applies to · 2,366 components

This rule is about the whole design, not one node.

This rule names no kind: it reads the design as a whole.

Tripped by · 1

Reference designs whose broken twin names this rule in its header. The gate reads the same header, and fails when a rule no design trips.

llm-gatewayreference

Source

1# yaml-language-server: $schema=../schema/rule.schema.json
2# Three or more AI vendors in one design and no gateway anywhere: every caller holds every key,
3# retries and fallbacks are written per vendor, and spend is not seen in one place. Counted once
4# for the design. A gateway such as litellm, portkey or cloudflare-ai-gateway closes it.
5# Tripped by: designs/reference/llm-gateway/design.broken.yaml
6schemaVersion: "1"
7type: "rule"
8id: "ai-no-gateway"
9severity: "info"
10select:
11  design: true
12when:
13  all:
14    - hasNode:
15        kind: "ai-service"
16        count:
17          gt: 2
18    - not:
19        hasNode:
20          kind: "api-gateway"
21message: "Design '{{design.id}}' calls three or more AI services and has no API gateway to route, retry and meter them."

The same page as markdown, for an agent: /catalogue/rules/ai-no-gateway.md