DeepEval

Open-source LLM evaluation framework run as unit tests in CI.

Provider: default
Kind: CI/CD pipeline
Category: Operations
Official docs: https://deepeval.com/docs/evaluation-unit-testing-in-ci-cd

Properties · 2 set by this component

property	type	values	required	kind default	this component
approvalRequired	boolean	—	no	—	inherited
artifactSigning	boolean	—	no	—	inherited
concurrency	integer	—	no	—	inherited
confidentReporting · component property	boolean	—	no	false	inherited
managed	boolean	—	no	false	▸ false
secretsFromVault	boolean	—	no	—	inherited
selfHostedRunners	boolean	—	no	—	inherited
testRunner · component property	enum	deepeval-test-run \| pytest	no	deepeval-test-run	inherited
trigger	enum	manual \| commit \| pull-request \| schedule	no	—	▸ pull-request

Rules that apply · 0

Every rule whose selector covers this component's kind or its category. Whether a rule fires depends on the design: its properties, its edges and its groups.

No rule reads this kind yet. That is a gap, not a guarantee.

Use it in a design

nodes:
  - { id: "deepeval", component: "deepeval" }

Other CI/CD pipeline products · 56

Alibaba Cloud DevOps Flowalibaba-devops-flowalibabacloudci-pipeline AWS CodeBuildaws-codebuildawsci-pipeline Amazon CodeCatalystaws-codecatalystawsci-pipeline AWS CodeDeployaws-codedeployawsci-pipeline AWS CodePipelineaws-codepipelineawsci-pipeline EC2 Image Builderaws-ec2-image-builderawsci-pipeline AWS Protonaws-protonawsci-pipeline AWS Telco Network Builder (TNB)aws-tnbawsci-pipeline Azure Deployment Environmentsazure-deployment-environmentsazureci-pipeline Managed DevOps Poolsazure-managed-devops-poolsazureci-pipeline Azure Operator Service Managerazure-operator-service-managerazureci-pipeline Azure Pipelinesazure-pipelinesazureci-pipeline Azure VM Image Builderazure-vm-image-builderazureci-pipeline Argo CDargo-cdci-pipeline Atlantisatlantisci-pipeline Automation pipelineautomation-pipelineci-pipeline Ansible AWXawxci-pipeline Atlassian Bamboobambooci-pipeline Buildkitebuildkiteci-pipeline Chef Infrachefci-pipeline Generic CI/CD pipelineci-pipelineci-pipeline CircleCIcirclecici-pipeline Concourseconcourseci-pipeline Crossplanecrossplaneci-pipeline Dronedroneci-pipeline Fluxfluxci-pipeline GitHub Actionsgithub-actionsci-pipeline GitLab CI/CDgitlab-cici-pipeline HCP Terraformhcp-terraformci-pipeline Jenkinsjenkinsci-pipeline PipeCDpipecdci-pipeline promptfoopromptfooci-pipeline Puppet Enterprisepuppetci-pipeline Salt Projectsaltci-pipeline Spinnakerspinnakerci-pipeline JetBrains TeamCityteamcityci-pipeline Tektontektonci-pipeline Travis CItravis-cici-pipeline Woodpecker CIwoodpeckerci-pipeline Zuulzuul-cici-pipeline

and 16 more → CI/CD pipeline page

Source

1# yaml-language-server: $schema=../../schema/component.schema.json
2# DeepEval: Apache-2.0 LLM evaluation framework whose tests run in CI through `deepeval test run`. Per "Unit
3# Testing in CI/CD", the GitHub Actions example triggers on push and pull_request to main, plain pytest is
4# discouraged, and a CONFIDENT_API_KEY optionally sends test reports to Confident AI.
5schemaVersion: "1"
6type: "component"
7id: "deepeval"
8kind: "ci-pipeline"
9provider: "default"
10docs: "https://deepeval.com/docs/evaluation-unit-testing-in-ci-cd"
11defaults:
12  managed: false
13  trigger: "pull-request"
14properties:
15  testRunner:
16    type: "enum"
17    values: ["deepeval-test-run", "pytest"]
18    default: "deepeval-test-run"
19  confidentReporting:
20    type: "boolean"
21    default: false

The same page as markdown, for an agent: /catalogue/components/deepeval.md