Luigi

Python batch pipeline framework with a central in-memory task scheduler.

Provider: default
Kind: Workflow engine
Category: Compute
Official docs: https://luigi.readthedocs.io/en/stable/index.html

Properties · 4 set by this component

property	type	values	required	kind default	this component
compensation	boolean	—	no	false	▸ false
durable	boolean	—	no	true	▸ false
humanReview	boolean	—	no	false	inherited
managed	boolean	—	no	false	▸ false
pattern	enum	saga \| tcc \| two-phase-commit \| dag	no	dag	▸ dag
retryCount · component property	integer	—	no	0	inherited
stepsIdempotent	boolean	—	no	false	inherited

Rules that apply · 1

Every rule whose selector covers this component's kind or its category. Whether a rule fires depends on the design: its properties, its edges and its groups.

Stateful node without zone redundancyerror

stateful-node-without-zone-redundancy

A stateful compute node pinned to exactly one availability zone disappears with that zone, and its state goes with it. Nodes inside a cell are exempt, because there the cell is deliberately single-zone and redundancy comes from the other cells.

— all of these hold:
— the node's `stateless` is false
— the node sits inside exactly 1 availability-zone group
— this does not hold:
— the node sits inside at least one cell group

Use it in a design

nodes:
  - { id: "luigi", component: "luigi" }

Other Workflow engine products · 44

CloudFlowalibaba-cloudflowalibabacloudworkflow-engine CloudOps Orchestration Servicealibaba-oosalibabacloudworkflow-engine Amazon Augmented AI (A2I)aws-augmented-aiawsworkflow-engine Amazon Managed Workflows for Apache Airflowaws-mwaaawsworkflow-engine AWS Systems Manager Incident Manager and Change Manageraws-ssm-incident-and-change-managerawsworkflow-engine AWS Step Functionsaws-step-functionsawsworkflow-engine Amazon Simple Workflow Serviceaws-swfawsworkflow-engine Azure Durable Functionsazure-durable-functionsazureworkflow-engine Azure Logic Appsazure-logic-appsazureworkflow-engine Cloudflare Workflowscloudflare-workflowscloudflareworkflow-engine Apache Airflowairflowworkflow-engine Argo Workflowsargo-workflowsworkflow-engine Astronomer Astroastronomerworkflow-engine Cadencecadenceworkflow-engine Camunda 8camundaworkflow-engine Conductorconductorworkflow-engine Dagsterdagsterworkflow-engine Digdagdigdagworkflow-engine Flowableflowableworkflow-engine Inngestinngestworkflow-engine jBPMjbpmworkflow-engine Kestrakestraworkflow-engine n8nn8nworkflow-engine Node-REDnode-redworkflow-engine Prefectprefectworkflow-engine Prefect Cloudprefect-cloudworkflow-engine Restaterestateworkflow-engine Temporaltemporalworkflow-engine Temporal Cloudtemporal-cloudworkflow-engine Trigger.devtrigger-devworkflow-engine Windmillwindmillworkflow-engine Workatoworkatoworkflow-engine Generic workflow engineworkflow-engineworkflow-engine Zapierzapierworkflow-engine Application Integrationgoogle-application-integrationgcpworkflow-engine Cloud Composer (Managed Service for Apache Airflow)google-cloud-composergcpworkflow-engine Workflowsgoogle-cloud-workflowsgcpworkflow-engine Orchestration Pipelinesgoogle-orchestration-pipelinesgcpworkflow-engine IBM App Connectibm-app-connectibmworkflow-engine Oracle Integration 3oci-integrationociworkflow-engine

and 4 more → Workflow engine page

Source

1# yaml-language-server: $schema=../../schema/component.schema.json
2# Luigi: Python batch pipelines resolved as a task dependency graph by a central scheduler that keeps
3# state in memory. Tasks are not retried automatically; retry-count is zero unless configured ("Luigi").
4schemaVersion: "1"
5type: "component"
6id: "luigi"
7kind: "workflow-engine"
8provider: "default"
9docs: "https://luigi.readthedocs.io/en/stable/index.html"
10defaults:
11  durable: false
12  pattern: "dag"
13  compensation: false
14  managed: false
15properties:
16  retryCount:
17    type: "integer"
18    default: 0

The same page as markdown, for an agent: /catalogue/components/luigi.md