---
id: "cloudflare-logpush"
type: "component"
kind: "stream-processor"
provider: "cloudflare"
category: "messaging"
docs: "https://developers.cloudflare.com/logs/logpush/logpush-job/api-configuration/"
url: "https://localhost:3000/catalogue/components/cloudflare-logpush"
---

# Cloudflare Logpush

Job that continuously batches a Cloudflare log dataset to storage, a SIEM or a log provider.

| | |
|---|---|
| Provider | [cloudflare](https://localhost:3000/catalogue?provider=cloudflare) |
| Kind | [Stream processor](https://localhost:3000/catalogue/kinds/stream-processor) |
| Category | Messaging |
| Official docs | [https://developers.cloudflare.com/logs/logpush/logpush-job/api-configuration/](https://developers.cloudflare.com/logs/logpush/logpush-job/api-configuration/) |

## Properties

| property | type | values | required | kind default | this component |
|---|---|---|---|---|---|
| `checkpointIntervalSeconds` | integer | — | no | — | inherited |
| `exactlyOnce` | boolean | — | no | false | inherited |
| `instances` | integer | — | no | — | inherited |
| `lateDataPolicy` | enum | `drop`, `shift`, `sideline` | no | — | inherited |
| `managed` | boolean | — | no | false | **true** |
| `maxUploadRecords` | integer | — | no | 100000 | inherited |
| `outputFormat` | enum | `ndjson`, `csv` | no | "ndjson" | inherited |
| `sampled` | boolean | — | no | false | inherited |
| `stateBackend` | enum | `memory`, `rocksdb`, `external` | no | "memory" | inherited |
| `timestampFormat` | enum | `unixnano`, `unix`, `rfc3339` | no | "unixnano" | inherited |
| `watermarkSeconds` | integer | — | no | — | inherited |
| `windowing` | enum | `none`, `event-time`, `processing-time` | no | "none" | inherited |

Bold values are the ones this component pins.

## Rules that apply · 1

### Event-time windows without watermark `windowed-aggregation-without-watermark` (warning)

- Page: https://localhost:3000/catalogue/rules/windowed-aggregation-without-watermark.md
- Why: Event-time windows need a watermark to know when a window is complete. Without one the processor either waits forever or closes windows early and silently drops late events.
- Applies to: every node of kind stream-processor
- Message: Stream processor '{{node.id}}' windows by event time but declares no watermark. Late data handling is undefined.

It fires when:

- all of these hold:
  - the node's `windowing` is "event-time"
  - the node's `watermarkSeconds` has no value

## Other products of the same kind · 33

- [Realtime Compute for Apache Flink](https://localhost:3000/catalogue/components/alibaba-flink.md) `alibaba-flink` — alibabacloud
- [Amazon EventBridge Pipes](https://localhost:3000/catalogue/components/aws-eventbridge-pipes.md) `aws-eventbridge-pipes` — aws
- [Amazon Data Firehose](https://localhost:3000/catalogue/components/aws-firehose.md) `aws-firehose` — aws
- [Amazon Managed Service for Apache Flink](https://localhost:3000/catalogue/components/aws-managed-flink.md) `aws-managed-flink` — aws
- [Amazon OpenSearch Ingestion](https://localhost:3000/catalogue/components/aws-opensearch-ingestion.md) `aws-opensearch-ingestion` — aws
- [Azure Stream Analytics](https://localhost:3000/catalogue/components/azure-stream-analytics.md) `azure-stream-analytics` — azure
- [Cloudflare Pipelines](https://localhost:3000/catalogue/components/cloudflare-pipelines.md) `cloudflare-pipelines` — cloudflare
- [Arroyo](https://localhost:3000/catalogue/components/arroyo.md) `arroyo` — default
- [Apache Beam](https://localhost:3000/catalogue/components/beam.md) `beam` — default
- [Bytewax](https://localhost:3000/catalogue/components/bytewax.md) `bytewax` — default
- [Drasi](https://localhost:3000/catalogue/components/drasi.md) `drasi` — default
- [Apache Flink](https://localhost:3000/catalogue/components/flink.md) `flink` — default
- [Kafka Streams](https://localhost:3000/catalogue/components/kafka-streams.md) `kafka-streams` — default
- [ksqlDB](https://localhost:3000/catalogue/components/ksqldb.md) `ksqldb` — default
- [Logstash](https://localhost:3000/catalogue/components/logstash.md) `logstash` — default
- [Materialize](https://localhost:3000/catalogue/components/materialize.md) `materialize` — default
- [Apache NiFi](https://localhost:3000/catalogue/components/nifi.md) `nifi` — default
- [Norikra](https://localhost:3000/catalogue/components/norikra.md) `norikra` — default
- [Redpanda Connect](https://localhost:3000/catalogue/components/redpanda-connect.md) `redpanda-connect` — default
- [RisingWave](https://localhost:3000/catalogue/components/risingwave.md) `risingwave` — default
- [Apache Samza](https://localhost:3000/catalogue/components/samza.md) `samza` — default
- [Snowplow](https://localhost:3000/catalogue/components/snowplow.md) `snowplow` — default
- [Spark Structured Streaming](https://localhost:3000/catalogue/components/spark-streaming.md) `spark-streaming` — default
- [Apache Storm](https://localhost:3000/catalogue/components/storm.md) `storm` — default
- [Generic stream processor](https://localhost:3000/catalogue/components/stream-processor.md) `stream-processor` — default
- [Tremor](https://localhost:3000/catalogue/components/tremor.md) `tremor` — default
- [Google Cloud Dataflow](https://localhost:3000/catalogue/components/google-dataflow.md) `google-dataflow` — gcp
- [Manufacturing Data Engine](https://localhost:3000/catalogue/components/google-manufacturing-data-engine.md) `google-manufacturing-data-engine` — gcp
- [Activity Tracker Event Routing](https://localhost:3000/catalogue/components/ibm-activity-tracker-event-routing.md) `ibm-activity-tracker-event-routing` — ibm
- [IBM Cloud Logs Routing](https://localhost:3000/catalogue/components/ibm-logs-routing.md) `ibm-logs-routing` — ibm
- [IBM Cloud Metrics Routing](https://localhost:3000/catalogue/components/ibm-metrics-routing.md) `ibm-metrics-routing` — ibm
- [OCI Connector Hub](https://localhost:3000/catalogue/components/oci-connector-hub.md) `oci-connector-hub` — oci
- [OCI GoldenGate Stream Analytics](https://localhost:3000/catalogue/components/oci-goldengate-stream-analytics.md) `oci-goldengate-stream-analytics` — oci

## Use it in a design

```yaml
nodes:
  - { id: "cloudflare-logpush", component: "cloudflare-logpush" }
```

## Source (`component` YAML, verbatim)

```yaml
# yaml-language-server: $schema=../../schema/component.schema.json
# Logpush job: continuously batches a Cloudflare dataset out to storage, a SIEM or a log provider.
# "Log output options" gives ndjson as the default record format; the API configuration page gives
# an unsampled default (sample 1), unixnano timestamps and 100,000 records per upload.
schemaVersion: "1"
type: "component"
id: "cloudflare-logpush"
kind: "stream-processor"
provider: "cloudflare"
docs: "https://developers.cloudflare.com/logs/logpush/logpush-job/api-configuration/"
defaults:
  managed: true
properties:
  outputFormat:
    type: "enum"
    values: ["ndjson", "csv"]
    default: "ndjson"
  timestampFormat:
    type: "enum"
    values: ["unixnano", "unix", "rfc3339"]
    default: "unixnano"
  sampled:
    type: "boolean"
    default: false
  maxUploadRecords:
    type: "integer"
    default: 100000
```

Human page: https://localhost:3000/catalogue/components/cloudflare-logpush