Job

k8sWorkerk8s-job

Runs Pods to completion and retries failures up to a backoff limit.

Provider
k8s
Kind
Worker
Category
Compute
Properties · 3 set by this component
propertytypevaluesrequiredkind defaultthis component
acceleratorenumnone | gpu | asicnononeinherited
backoffLimit · component propertyinteger—no6inherited
completionMode · component propertyenumnon-indexed | indexednonon-indexedinherited
completions · component propertyinteger—no1inherited
concurrencyinteger—no—▸ 1
fanoutStrategy · advancedenumnone | push | pull | hybridnononeinherited
fanoutThreshold · advancedinteger—no—inherited
idempotentboolean—nofalseinherited
instancesinteger—no—inherited
managedboolean—nofalse▸ false
rateControl · advancedenumnone | fixed | aimdnononeinherited
restartPolicy · component propertyenumnever | on-failureno—inherited
schedulestring—no—inherited
suspend · component propertyboolean—nofalseinherited
triggerenumqueue | stream | schedule | batch | manualyes—▸ batch
Rules that apply · 3

Every rule whose selector covers this component's kind or its category. Whether a rule fires depends on the design: its properties, its edges and its groups.

push-fanout-without-celebrity-path

Pure push fan-out over a social graph is brutal at the tail: one post from an account with millions of followers becomes millions of cache writes. A follower threshold above which readers merge at read time bounds that write amplification.

  • — all of these hold:
  • — the node's `fanoutStrategy` is "push"
  • — the node's `fanoutThreshold` has no value
  • — the design contains a node of kind graph-database
shared-worker-pool-across-queues

One worker pool consuming several queues lets a slow or poisoned queue starve the others. That is precisely what the design's per-queue isolation requirement forbids.

  • — all of these hold:
  • — the node has more than 1 incoming edge from a queue
  • — the design requirement `isolation.perQueue` is true
stateful-node-without-zone-redundancy

A stateful compute node pinned to exactly one availability zone disappears with that zone, and its state goes with it. Nodes inside a cell are exempt, because there the cell is deliberately single-zone and redundancy comes from the other cells.

  • — all of these hold:
  • — the node's `stateless` is false
  • — the node sits inside exactly 1 availability-zone group
  • — this does not hold:
  • — the node sits inside at least one cell group
Use it in a design
nodes:
  - { id: "job", component: "k8s-job" }
Other Worker products · 82
SchedulerXalibaba-schedulerxalibabacloudworkerAmazon AppFlowaws-appflowawsworkerAWS B2B Data Interchangeaws-b2b-data-interchangeawsworkerAWS Batchaws-batchawsworkerAWS Glue DataBrewaws-databrewawsworkerAWS DataSyncaws-datasyncawsworkerAWS Deadline Cloudaws-deadline-cloudawsworkerAWS Entity Resolutionaws-entity-resolutionawsworkerAmazon EventBridge Scheduleraws-eventbridge-schedulerawsworkerAWS Fault Injection Serviceaws-fisawsworkerAWS Glue crawleraws-glue-crawlerawsworkerAWS Glue for Ray jobaws-glue-ray-jobawsworkerAWS Application Migration Serviceaws-mgnawsworkerAmazon MSK Connectaws-msk-connectawsworkerAmazon SageMaker AI Training Jobaws-sagemaker-training-jobawsworkerAWS Systems Manageraws-systems-managerawsworkerAWS Thinkbox Deadline 10aws-thinkbox-deadlineawsworkerAWS Glue jobglue-jobawsworkerAzure Automationazure-automationazureworkerAzure Batchazure-batchazureworkerAzure Chaos Studioazure-chaos-studioazureworkerAzure Data Factoryazure-data-factoryazureworkerAzure Data Shareazure-data-shareazureworkerAzure Database Migration Serviceazure-database-migration-serviceazureworkerMicrosoft Entra Connect Syncazure-entra-connect-syncazureworkerMicrosoft Graph Data Connectazure-graph-data-connectazureworkerAzure Storage Moverazure-storage-moverazureworkerAzure Update Managerazure-update-managerazureworkerBrowser Runcloudflare-browser-runcloudflareworkerCloudflare Snippetscloudflare-snippetscloudflareworkerCloudflare Zarazcloudflare-zarazcloudflareworkerAirbyteairbyteworkerAirflow DAGairflow-dagworkerChaos Meshchaos-meshworkerChunkrchunkrworkerContainer workercontainer-workerworkerCron jobcron-jobworkerDaytonadaytonaworkerE2Be2bworkerEmbulkembulkworker
and 42 more → Worker page
Source
# yaml-language-server: $schema=../../schema/component.schema.json
# Kubernetes Job: runs Pods to completion and tracks how many succeeded. Per "Jobs" and the
# JobSpec reference backoffLimit defaults to 6 retries before the Job is marked failed,
# completionMode defaults to NonIndexed, parallelism and completions default to 1, suspend
# defaults to false, and the Pod template restartPolicy must be Never or OnFailure.
schemaVersion: "1"
type: "component"
id: "k8s-job"
kind: "worker"
provider: "k8s"
docs: "https://kubernetes.io/docs/concepts/workloads/controllers/job/"
defaults:
  trigger: "batch"
  managed: false
  concurrency: 1
properties:
  backoffLimit:
    type: "integer"
    default: 6
  completions:
    type: "integer"
    default: 1
  completionMode:
    type: "enum"
    values: ["non-indexed", "indexed"]
    default: "non-indexed"
  restartPolicy:
    type: "enum"
    values: ["never", "on-failure"]
  suspend:
    type: "boolean"
    default: false

The same page as markdown, for an agent: /catalogue/components/k8s-job.md