E-MapReduce

alibabacloud Big data clusteralibaba-e-mapreduce

Semi-managed big data platform running Hadoop, Spark, Flink, Hive and StarRocks on ECS instances, ACK or serverless engines.

Provider: alibabacloud
Kind: Big data cluster
Category: Compute
Official docs: https://www.alibabacloud.com/help/en/emr/
Also known as: EMR

Properties · 1 set by this component

property	type	values	required	kind default	this component
autoscaling	boolean	—	no	false	inherited
deploymentMode · component property	enum	emr-on-ecs \| emr-on-ack \| emr-serverless-spark \| emr-serverless-starrocks	no	emr-on-ecs	inherited
engine	enum	spark \| hadoop \| presto \| flink \| hpc \| ray \| dask \| kubernetes-batch	no	—	inherited
managed	boolean	—	no	false	▸ false
runtimeVersion	string	—	no	—	inherited
spotInstances	boolean	—	no	false	inherited
storageBackend · component property	enum	hdfs \| oss	no	—	inherited
workerNodes	integer	—	no	—	inherited
zoneRedundant	boolean	—	no	—	inherited

Rules that apply · 1

Every rule whose selector covers this component's kind or its category. Whether a rule fires depends on the design: its properties, its edges and its groups.

Stateful node without zone redundancyerror

stateful-node-without-zone-redundancy

A stateful compute node pinned to exactly one availability zone disappears with that zone, and its state goes with it. Nodes inside a cell are exempt, because there the cell is deliberately single-zone and redundancy comes from the other cells.

— all of these hold:
— the node's `stateless` is false
— the node sits inside exactly 1 availability-zone group
— this does not hold:
— the node sits inside at least one cell group

Use it in a design

nodes:
  - { id: "e-mapreduce", component: "alibaba-e-mapreduce" }

Other Big data cluster products · 25

Elastic High Performance Computingalibaba-e-hpcalibabacloudbig-data-cluster AWS Clean Roomsaws-clean-roomsawsbig-data-cluster Amazon EMRaws-emrawsbig-data-cluster Amazon EMR Serverlessaws-emr-serverlessawsbig-data-cluster AWS Parallel Computing Serviceaws-pcsawsbig-data-cluster Amazon SageMaker HyperPodaws-sagemaker-hyperpodawsbig-data-cluster Azure CycleCloudazure-cyclecloudazurebig-data-cluster Azure Databricksazure-databricksazurebig-data-cluster Azure HDInsightazure-hdinsightazurebig-data-cluster Armadaarmadabig-data-cluster Generic big data clusterbig-data-clusterbig-data-cluster Daskdaskbig-data-cluster Databricksdatabricksbig-data-cluster Apache Hadoop YARNhadoop-yarnbig-data-cluster Rayraybig-data-cluster Slurmslurmbig-data-cluster Apache Spark clusterspark-clusterbig-data-cluster Volcanovolcanobig-data-cluster Cluster Directorgoogle-cluster-directorgcpbig-data-cluster Dataprocgoogle-dataprocgcpbig-data-cluster Managed Service for Apache Spark (serverless)google-dataproc-serverlessgcpbig-data-cluster IBM Analytics Engineibm-analytics-engineibmbig-data-cluster OCI AI Data Platformoci-ai-data-platformocibig-data-cluster OCI Big Data Serviceoci-big-data-serviceocibig-data-cluster OCI Data Flowoci-data-flowocibig-data-cluster

Source

1# yaml-language-server: $schema=../../schema/component.schema.json
2# E-MapReduce (EMR): a big data platform that runs Apache Hadoop, Spark, Flink, Hive and StarRocks on
3# Alibaba Cloud ECS instances with data in OSS ("What is EMR on ECS"). That page states EMR on ECS is
4# semi-managed - cluster resources belong to your own account and you are responsible for the routine O&M of
5# the open-source services - so managed is false, and that cluster resources scale up or down based on load
6# or on a schedule. "Introduction to E-MapReduce" lists the EMR on ACK and EMR Serverless deployments.
7schemaVersion: "1"
8type: "component"
9id: "alibaba-e-mapreduce"
10kind: "big-data-cluster"
11provider: "alibabacloud"
12docs: "https://www.alibabacloud.com/help/en/emr/"
13aliases: ["EMR"]
14defaults:
15  managed: false
16properties:
17  deploymentMode:
18    type: "enum"
19    values: ["emr-on-ecs", "emr-on-ack", "emr-serverless-spark", "emr-serverless-starrocks"]
20    default: "emr-on-ecs"
21  storageBackend:
22    type: "enum"
23    values: ["hdfs", "oss"]

The same page as markdown, for an agent: /catalogue/components/alibaba-e-mapreduce.md