Skip to main content

​
Display Helm Chart Values

helm show values dify/dify

​
1. Adjust Base Resources

Tune replicas and resource limits based on workload, cluster capacity, database connection limits, and queue backlog.
CategoryComponentReplicasRequest CPURequest MemLimit CPULimit MemNotes
Core ApplicationAPI211 GB12 GBScale based on API QPS and latency
Worker444 GB48 GBDefault catch-all worker; for 3.9.x, prefer the additionalWorkers split strategy
Worker Beat112 GB24 GBKeep one replica
Web10.51 GB12 GBScale based on frontend traffic
Sandbox122 GB24 GBTune based on code execution workload
EnterpriseEnterprise122 GB22 GBScale as needed
Enterprise_Audit112 GB24 GBScale as needed
Enterprise_Frontend112 GB12 GBScale as needed
PluginPlugin Daemon112 GB24 GBTune for plugin call and install volume
Plugin Controller10.51 GB12 GBScale as needed
Plugin Connector112 GB12 GBScale as needed
Plugin Manager112 GB24 GBScale as needed
InfrastructureSSRF Proxy10.50.5 GB11 GBScale as needed
Gateway112 GB24 GBTune for plugin traffic
Unstructured-----Tune based on document parsing workload
MinIO-----Tune based on storage throughput and capacity
Base configuration example: 
api:
  replicas: 2
  serverWorkerAmount: 1

worker:
  replicas: 4
  celeryWorkerAmount: 1

workerBeat:
  resources:
    requests:
      cpu: 1
      memory: 2Gi
    limits:
      cpu: 2
      memory: 4Gi
workerBeat should always run as a single replica. Do not scale it horizontally.

​
2. Split Workers by Queue

In 3.9.x, the Helm chart supports additionalWorkers. It splits Celery queues that were previously handled by the default worker into dedicated Deployments. Each worker can have its own replicas, CPU, memory, and scheduling policy. The default worker is a catch-all worker and consumes all queues. After enabling additionalWorkers, use one of the following patterns:
  • Conservative mode: keep worker.enabled: true and only enable dedicated workers for hot queues. The default worker continues to consume queues that have not been split out.
  • Full split mode: set worker.enabled: false and make sure every queue used by your deployment is consumed by an enabled additionalWorkers entry. Otherwise, background tasks in uncovered queues will accumulate.
Common worker splits:
WorkerQueuesWhen to use
dataset-workerdataset,priority_dataset,pipeline,priority_pipelineHeavy knowledge base import, document parsing, or indexing workloads
workflow-workerworkflow,workflow_storage,workflow_based_app_executionHigh workflow execution or workflow storage load
general-workermail,ops_trace,app_deletion,conversation,api_token,plugin,retention,enterprise_telemetryEmail, conversation, plugin, retention, audit, and telemetry background tasks
trigger-workerschedule_poller,schedule_executor,triggered_workflow_dispatcher,trigger_refresh_executorScheduled tasks and trigger-related workloads
If you disable the default worker, confirm that dataset, workflow, general background, and trigger queues are all covered by enabled workers. Disabling the default worker after enabling only part of additionalWorkers can leave some queues without consumers.
Conservative mode example: 
worker:
  enabled: true
  replicas: 1

additionalWorkers:
  - name: workflow-worker
    enabled: true
    replicas: 2
    celeryQueues: "workflow,workflow_storage,workflow_based_app_execution"
    celeryWorkerAmount: 2
Full split mode example: 
worker:
  enabled: false

additionalWorkers:
  - name: dataset-worker
    enabled: true
    replicas: 1
    celeryQueues: "dataset,priority_dataset,pipeline,priority_pipeline"
    celeryWorkerAmount: 4

  - name: workflow-worker
    enabled: true
    replicas: 2
    celeryQueues: "workflow,workflow_storage,workflow_based_app_execution"
    celeryWorkerAmount: 2

  - name: general-worker
    enabled: true
    replicas: 1
    celeryQueues: "mail,ops_trace,app_deletion,conversation,api_token,plugin,retention,enterprise_telemetry"
    celeryWorkerAmount: 1

  - name: trigger-worker
    enabled: true
    replicas: 1
    celeryQueues: "schedule_poller,schedule_executor,triggered_workflow_dispatcher,trigger_refresh_executor"
    celeryWorkerAmount: 1
Recommendations:
  • If concurrency is high or a queue backlog is visible, increase the corresponding worker’s replicas first.
  • If a single task is memory-heavy, memory usage stays close to the limit, or pods are OOMKilled, increase the corresponding worker’s memory limit.
  • Do not blindly increase celeryWorkerAmount. It increases per-pod concurrency and may increase database connections, Redis connections, and memory usage.
  • Keep workerBeat at one replica. It schedules tasks but does not replace worker queue consumers.

​
3. Improve External Postgres Performance

Estimate required max_connections from API and worker concurrency: 
Maximum database connections =
  (SQLALCHEMY_POOL_SIZE + SQLALCHEMY_MAX_OVERFLOW) × API serverWorkerAmount × API replicas
+ (SQLALCHEMY_POOL_SIZE + SQLALCHEMY_MAX_OVERFLOW) × default worker celeryWorkerAmount × default worker replicas
+ Σ[(SQLALCHEMY_POOL_SIZE + SQLALCHEMY_MAX_OVERFLOW) × additionalWorker celeryWorkerAmount × additionalWorker replicas]
Example:
  • API: replicas=2, serverWorkerAmount=1
  • Default worker: replicas=4, celeryWorkerAmount=1
  • SQLALCHEMY_POOL_SIZE=100
  • SQLALCHEMY_MAX_OVERFLOW=150
(100 + 150) × 1 × 2 + (100 + 150) × 1 × 4 = 500 + 1000 = 1500
Reserve 20-30% headroom for traffic spikes. In this example, set max_connections to 1800-2000 or higher, and confirm that your database instance can support it.

​
4. Performance Monitoring

SymptomPossible CauseSolution
Slow API responsesInsufficient API replicas or serverWorkerAmount; database slow queriesIncrease API replicas and investigate database slow queries
Workflow queue backlogInsufficient workflow-worker capacityIncrease workflow-worker.replicas; raise resource limits if needed
Slow knowledge base importInsufficient dataset-worker resources; slow document parsing or vector DB writesEnable or scale dataset-worker; check Unstructured and vector database performance
Delayed email, plugin, or retention jobsgeneral-worker queue backlogEnable or scale general-worker
Delayed scheduled tasks or triggerstrigger-worker queue backlogEnable or scale trigger-worker; keep workerBeat at one replica
Database connections exhaustedAPI/worker concurrency and pool settings are too highIncrease max_connections or reduce pool/concurrency settings
OOMKilled podsTask memory usage is high or memory limit is too lowIncrease memory limits for the affected component and identify memory-heavy tasks
CPU throttlingCPU limit is too lowIncrease CPU limits or replicas for the affected component