OCI Data Science Model Deployment

oci ML inferenceoci-data-science-model-deployment

HTTP predict endpoint serving a catalogued model behind a load balancer.

Provider: oci
Kind: ML inference
Category: Compute
Official docs: https://docs.oracle.com/en-us/iaas/data-science/using/model_dep_create.htm

Properties · 4 set by this component

property	type	values	required	kind default	this component
accelerator	enum	none \| gpu \| asic	no	none	▸ none
acceleratorModel · advanced	enum	nvidia-h100 \| nvidia-h200 \| nvidia-b200 \| nvidia-l4 \| nvidia-a100 \| amd-mi300x \| aws-trainium \| aws-inferentia \| google-tpu \| intel-gaudi \| groq-lpu \| cerebras-wse	no	—	inherited
autoscaling · component property	boolean	—	no	false	inherited
embeddingSyncMinutes · advanced	integer	—	no	—	inherited
endpointType · component property	enum	public \| private	no	public	inherited
inputs	enum	live \| historical \| both	no	live	▸ live
instances	integer	—	no	—	inherited
loadBalancerBandwidthMbps · component property	integer	—	no	10	inherited
logging · component property	boolean	—	no	false	inherited
managed	boolean	—	no	false	▸ true
maxModelSizeGb · component property	integer	—	no	400	inherited
networking · component property	enum	default-networking \| custom-vcn	no	default-networking	inherited
snapshotToObjectStore · advanced	boolean	—	no	false	inherited
training	enum	batch \| online	no	batch	▸ batch

Rules that apply · 1

Every rule whose selector covers this component's kind or its category. Whether a rule fires depends on the design: its properties, its edges and its groups.

Stateful node without zone redundancyerror

stateful-node-without-zone-redundancy

A stateful compute node pinned to exactly one availability zone disappears with that zone, and its state goes with it. Nodes inside a cell are exempt, because there the cell is deliberately single-zone and redundancy comes from the other cells.

— all of these hold:
— the node's `stateless` is false
— the node sits inside exactly 1 availability-zone group
— this does not hold:
— the node sits inside at least one cell group

Use it in a design

nodes:
  - { id: "data-science-model-deployment", component: "oci-data-science-model-deployment" }

Other ML inference products · 37

SageMaker endpointsagemaker-endpointawsml-inference Azure ML endpointazure-ml-endpointazureml-inference Workers AIcloudflare-workers-aicloudflareml-inference Anyscale Endpointsanyscale-endpointsml-inference Bananabanana-devml-inference Basetenbasetenml-inference Beambeam-cloudml-inference BentoMLbentomlml-inference Hugging Face Inference Endpointshuggingface-inferenceml-inference Infinityinfinity-embeddingsml-inference Janjanml-inference KServekserveml-inference llama.cpp serverllama-cpp-serverml-inference llm-dllm-dml-inference LM Studiolm-studioml-inference LMDeploylmdeployml-inference LocalAIlocalaiml-inference Generic ML inferenceml-inferenceml-inference MLC LLMmlc-llmml-inference Modalmodalml-inference Mystic AImystic-aiml-inference NVIDIA NIM (self-hosted)nvidia-nimml-inference OctoAIoctoaiml-inference Ollamaollamaml-inference Ray Serveray-serveml-inference Seldon Coreseldon-coreml-inference SGLangsglangml-inference TensorFlow Servingtensorflow-servingml-inference TensorRT-LLMtensorrt-llmml-inference Text Embeddings Inferencetext-embeddings-inferenceml-inference Text Generation Inferencetext-generation-inferenceml-inference TorchServetorchserveml-inference NVIDIA Tritontritonml-inference vLLMvllmml-inference Xinferencexinferenceml-inference Vertex AI endpointvertex-ai-endpointgcpml-inference OCI Generative AI Dedicated AI Clusteroci-generative-ai-clusterociml-inference

Source

1# yaml-language-server: $schema=../../schema/component.schema.json
2# Oracle OCI Data Science model deployment: an HTTP predict endpoint serving a model from the
3# model catalog behind a load balancer. "Creating a Model Deployment": the load balancer
4# bandwidth defaults to 10 Mbps, the endpoint type defaults to a public endpoint on
5# service-managed default networking, autoscaling and logging are off unless enabled, and the
6# model artifact may be up to 400 GB. No instance count is defaulted; it must be entered.
7schemaVersion: "1"
8type: "component"
9id: "oci-data-science-model-deployment"
10kind: "ml-inference"
11provider: "oci"
12docs: "https://docs.oracle.com/en-us/iaas/data-science/using/model_dep_create.htm"
13defaults:
14  training: "batch"
15  inputs: "live"
16  accelerator: "none"
17  managed: true
18properties:
19  endpointType:
20    type: "enum"
21    values: ["public", "private"]
22    default: "public"
23  networking:
24    type: "enum"
25    values: ["default-networking", "custom-vcn"]
26    default: "default-networking"
27  loadBalancerBandwidthMbps:
28    type: "integer"
29    default: 10
30  autoscaling:
31    type: "boolean"
32    default: false
33  logging:
34    type: "boolean"
35    default: false
36  maxModelSizeGb:
37    type: "integer"
38    default: 400

The same page as markdown, for an agent: /catalogue/components/oci-data-science-model-deployment.md