AI Crawl Control

cloudflare Bot managementcloudflare-ai-crawl-control

Reports which AI crawlers reach a zone and enforces allow or block policy per crawler.

Provider: cloudflare
Kind: Bot management
Category: Security
Official docs: https://developers.cloudflare.com/ai-crawl-control/features/manage-ai-crawlers/
Also known as: AI Audit

Properties · 3 set by this component

property	type	values	required	kind default	this component
challengeAction	enum	block \| managed-challenge \| js-challenge \| captcha \| log	no	—	▸ log
detectionMethod · component property	enum	user-agent \| bot-score	no	user-agent	inherited
jsDetections	boolean	—	no	—	inherited
machineLearning	boolean	—	no	—	inherited
managed	boolean	—	no	false	▸ true
payPerCrawl · component property	boolean	—	no	false	inherited
robotsTxtTracking · component property	boolean	—	no	true	inherited
scoreHeader	boolean	—	no	—	inherited
scoreThreshold	integer	—	no	—	inherited
sessionTokenLifetimeMinutes	integer	—	no	—	inherited
verifiedBotsAllowed	boolean	—	no	true	▸ true

Rules that apply · 0

Every rule whose selector covers this component's kind or its category. Whether a rule fires depends on the design: its properties, its edges and its groups.

No rule reads this kind yet. That is a gap, not a guarantee.

Use it in a design

nodes:
  - { id: "ai-crawl-control", component: "cloudflare-ai-crawl-control" }

Other Bot management products · 3

Cloudflare Bot Managementcloudflare-bot-managementcloudflarebot-management Cloudflare Turnstilecloudflare-turnstilecloudflarebot-management Generic bot managementbot-managementbot-management

Source

1# yaml-language-server: $schema=../../schema/component.schema.json
2# AI Crawl Control: reports which AI crawlers reach a zone and enforces per-crawler policy.
3# It "works automatically on all Cloudflare plans" with zero configuration and only observes
4# until a crawler is blocked, at which point a WAF custom rule is created ("Manage AI
5# crawlers"). Detection is user-agent based without Bot Management ("Get started").
6schemaVersion: "1"
7type: "component"
8id: "cloudflare-ai-crawl-control"
9kind: "bot-management"
10provider: "cloudflare"
11docs: "https://developers.cloudflare.com/ai-crawl-control/features/manage-ai-crawlers/"
12aliases: ["AI Audit"]
13defaults:
14  managed: true
15  challengeAction: "log"
16  verifiedBotsAllowed: true
17properties:
18  detectionMethod:
19    type: "enum"
20    values: ["user-agent", "bot-score"]
21    default: "user-agent"
22  robotsTxtTracking:
23    type: "boolean"
24    default: true
25  payPerCrawl:
26    type: "boolean"
27    default: false

The same page as markdown, for an agent: /catalogue/components/cloudflare-ai-crawl-control.md