---
id: "stream-processor"
type: "kind"
category: "messaging"
url: "https://localhost:3000/catalogue/kinds/stream-processor"
---

# Stream processor

Continuous computation over streams with windows and state.

| | |
|---|---|
| Category | Messaging |
| Properties | 8 |
| Components | 34 |
| Rules | 1 |

## Properties

| property | type | values | required | default |
|---|---|---|---|---|
| `checkpointIntervalSeconds` | integer | — | no | — |
| `exactlyOnce` | boolean | — | no | false |
| `instances` | integer | — | no | — |
| `lateDataPolicy` | enum | `drop`, `shift`, `sideline` | no | — |
| `managed` | boolean | — | no | false |
| `stateBackend` | enum | `memory`, `rocksdb`, `external` | no | "memory" |
| `watermarkSeconds` | integer | — | no | — |
| `windowing` | enum | `none`, `event-time`, `processing-time` | no | "none" |

## Rules for this kind · 1

### Event-time windows without watermark `windowed-aggregation-without-watermark` (warning)

- Page: https://localhost:3000/catalogue/rules/windowed-aggregation-without-watermark.md
- Why: Event-time windows need a watermark to know when a window is complete. Without one the processor either waits forever or closes windows early and silently drops late events.
- Applies to: every node of kind stream-processor
- Message: Stream processor '{{node.id}}' windows by event time but declares no watermark. Late data handling is undefined.

It fires when:

- all of these hold:
  - the node's `windowing` is "event-time"
  - the node's `watermarkSeconds` has no value

## Components · 34

### vendor-neutral

- [Generic stream processor](https://localhost:3000/catalogue/components/stream-processor.md) `stream-processor` — the kind itself

### alibabacloud

- [Realtime Compute for Apache Flink](https://localhost:3000/catalogue/components/alibaba-flink.md) `alibaba-flink` — 2 defaults

### aws

- [Amazon EventBridge Pipes](https://localhost:3000/catalogue/components/aws-eventbridge-pipes.md) `aws-eventbridge-pipes` — 3 defaults
- [Amazon Data Firehose](https://localhost:3000/catalogue/components/aws-firehose.md) `aws-firehose` — 4 defaults
- [Amazon Managed Service for Apache Flink](https://localhost:3000/catalogue/components/aws-managed-flink.md) `aws-managed-flink` — 4 defaults
- [Amazon OpenSearch Ingestion](https://localhost:3000/catalogue/components/aws-opensearch-ingestion.md) `aws-opensearch-ingestion` — 4 defaults

### azure

- [Azure Stream Analytics](https://localhost:3000/catalogue/components/azure-stream-analytics.md) `azure-stream-analytics` — 5 defaults

### cloudflare

- [Cloudflare Logpush](https://localhost:3000/catalogue/components/cloudflare-logpush.md) `cloudflare-logpush` — 1 defaults
- [Cloudflare Pipelines](https://localhost:3000/catalogue/components/cloudflare-pipelines.md) `cloudflare-pipelines` — 3 defaults

### default

- [Arroyo](https://localhost:3000/catalogue/components/arroyo.md) `arroyo` — 4 defaults
- [Apache Beam](https://localhost:3000/catalogue/components/beam.md) `beam` — 4 defaults
- [Bytewax](https://localhost:3000/catalogue/components/bytewax.md) `bytewax` — 4 defaults
- [Drasi](https://localhost:3000/catalogue/components/drasi.md) `drasi` — 4 defaults
- [Apache Flink](https://localhost:3000/catalogue/components/flink.md) `flink` — 4 defaults
- [Kafka Streams](https://localhost:3000/catalogue/components/kafka-streams.md) `kafka-streams` — 4 defaults
- [ksqlDB](https://localhost:3000/catalogue/components/ksqldb.md) `ksqldb` — 5 defaults
- [Logstash](https://localhost:3000/catalogue/components/logstash.md) `logstash` — 4 defaults
- [Materialize](https://localhost:3000/catalogue/components/materialize.md) `materialize` — 3 defaults
- [Apache NiFi](https://localhost:3000/catalogue/components/nifi.md) `nifi` — 4 defaults
- [Norikra](https://localhost:3000/catalogue/components/norikra.md) `norikra` — 3 defaults
- [Redpanda Connect](https://localhost:3000/catalogue/components/redpanda-connect.md) `redpanda-connect` — 4 defaults
- [RisingWave](https://localhost:3000/catalogue/components/risingwave.md) `risingwave` — 4 defaults
- [Apache Samza](https://localhost:3000/catalogue/components/samza.md) `samza` — 4 defaults
- [Snowplow](https://localhost:3000/catalogue/components/snowplow.md) `snowplow` — 2 defaults
- [Spark Structured Streaming](https://localhost:3000/catalogue/components/spark-streaming.md) `spark-streaming` — 3 defaults
- [Apache Storm](https://localhost:3000/catalogue/components/storm.md) `storm` — 4 defaults
- [Tremor](https://localhost:3000/catalogue/components/tremor.md) `tremor` — 4 defaults

### gcp

- [Google Cloud Dataflow](https://localhost:3000/catalogue/components/google-dataflow.md) `google-dataflow` — 5 defaults
- [Manufacturing Data Engine](https://localhost:3000/catalogue/components/google-manufacturing-data-engine.md) `google-manufacturing-data-engine` — 2 defaults

### ibm

- [Activity Tracker Event Routing](https://localhost:3000/catalogue/components/ibm-activity-tracker-event-routing.md) `ibm-activity-tracker-event-routing` — 1 defaults
- [IBM Cloud Logs Routing](https://localhost:3000/catalogue/components/ibm-logs-routing.md) `ibm-logs-routing` — 1 defaults
- [IBM Cloud Metrics Routing](https://localhost:3000/catalogue/components/ibm-metrics-routing.md) `ibm-metrics-routing` — 1 defaults

### oci

- [OCI Connector Hub](https://localhost:3000/catalogue/components/oci-connector-hub.md) `oci-connector-hub` — 3 defaults
- [OCI GoldenGate Stream Analytics](https://localhost:3000/catalogue/components/oci-goldengate-stream-analytics.md) `oci-goldengate-stream-analytics` — 1 defaults

## Source (`kind` YAML, verbatim)

```yaml
# yaml-language-server: $schema=../schema/kind.schema.json
# Continuous computation over one or more streams (Flink, Samza, Kafka Streams, Dataflow).
# watermarkSeconds is optional with no default so windowed-aggregation-without-watermark can fire.
schemaVersion: "1"
type: "kind"
id: "stream-processor"
category: "messaging"
properties:
  windowing:
    type: "enum"
    values: ["none", "event-time", "processing-time"]
    default: "none"
  watermarkSeconds:
    type: "integer"
  checkpointIntervalSeconds:
    type: "integer"
  exactlyOnce:
    type: "boolean"
    default: false
  stateBackend:
    type: "enum"
    values: ["memory", "rocksdb", "external"]
    default: "memory"
  lateDataPolicy:
    type: "enum"
    values: ["drop", "shift", "sideline"]
  instances:
    type: "integer"
  managed:
    type: "boolean"
    default: false
```

Human page: https://localhost:3000/catalogue/kinds/stream-processor