Overview
When enabled, Dify will trust both system-wide CA certificates and your custom CA certificate for all outbound HTTPS connections. This is particularly useful when:- Your organization uses an internal PKI (Public Key Infrastructure).
- You are deploying in air-gapped environments with self-signed certificates.
- External services (Redis, databases, S3, vector databases, etc.) use certificates signed by a private CA.
- You need to establish trust with internal HTTPS endpoints.
How It Works
The custom CA implementation uses the following approach:- Initialization: An init container runs before each Dify pod starts.
- CA Bundle Creation: The init container combines system CA certificates (from the base image) with your custom CA certificate (from a Kubernetes Secret).
- Environment Setup: Environment variables are configured to point Python, Node.js, curl, and AWS CLI to the combined CA bundle.
- Runtime Trust: All Dify components use the combined CA bundle for SSL/TLS verification.
Prerequisites
- Kubernetes cluster with Dify Helm chart installed (or ready to install).
- Your custom CA certificate in PEM format.
kubectlaccess to your cluster.- Basic understanding of Kubernetes Secrets.
Configuration Steps
1. Prepare Your CA Certificate
Your CA certificate must be in PEM format (Base64-encoded, enclosed inBEGIN CERTIFICATE and END CERTIFICATE markers).
Verify your certificate format:
2. Create a Kubernetes Secret
Create a Kubernetes Secret containing your CA certificate.Note:Verify the secret was created:
- Secret name:
dify-custom-ca(you can choose any name).- Key name:
ca.crt(default, can be customized invalues.yaml).- Namespace: Must be in the same namespace as your Dify deployment.
3. Configure values.yaml
Edit your Helmvalues.yaml file to enable custom CA.
4. Deploy or Update Dify
For new installations:Note: Changes to the custom CA configuration require a pod restart to take effect. The CA bundle is generated once during pod initialization.
Verification
After deployment, verify that the custom CA is properly configured.1. Check Init Containers
Check if the init containers ran successfully.2. Verify Environment Variables
Exec into a running pod and check the environment variables.3. Verify CA Bundle Content
Inside the pod, verify the bundle contains your certificate.Environment Variables Reference
The custom CA feature sets the following environment variables to ensure broad compatibility. All variables point to the same combined CA bundle:/etc/dify/custom-ca/ca-bundle.crt (or your custom mountPath).
| Variable | Purpose | Used By |
|---|---|---|
SSL_CERT_FILE | OpenSSL/LibreSSL | General SSL libraries, Python requests |
REQUESTS_CA_BUNDLE | Python requests library | Python HTTP clients |
CURL_CA_BUNDLE | curl command | Shell scripts, curl-based tools |
NODE_EXTRA_CA_CERTS | Node.js | Node.js HTTPS client |
AWS_CA_BUNDLE | AWS SDK | S3, AWS service connections |
Affected Components
When custom CA is enabled, the following Dify components automatically use the custom CA bundle:- API Server (
api-deployment) - Worker (
worker-deployment) - Worker Beat (
worker-beat-deployment) - Additional Workers (
additional-worker-deployment) - Enterprise Service (
enterprise-deployment) - Enterprise Audit (
enterprise-audit-deployment) - Enterprise Collector (
enterprise-collector-deployment) - Sandbox (
sandbox-deployment) - Gateway (
caddy-deploy) - SSRF Proxy (
ssrf-proxy-deployment) - Unstructured (
unstructured-deployment) - Plugin Daemon (
plugin-daemon) - Plugin Manager (
plugin-manager) - Plugin Controller (
plugin-controller)
Troubleshooting
Pods stuck in Init:0/1
Cause: Init containerinit-custom-ca is failing.
Solution:
initImagecannot be pulled (air-gapped environment).- Secret not found or wrong namespace.
- Wrong key name in secret.
SSL verification errors after enabling custom CA
Cause: Your custom CA certificate might not be properly formatted or is incomplete. Solution:- Verify certificate is valid PEM format:
- Check if it’s a full chain (if your CA requires intermediate certificates). Concatenate multiple certificates if needed:
- Update the secret and restart pods:
Changes to custom CA secret not taking effect
Cause: The CA bundle is created during pod initialization. Updating the secret doesn’t automatically update running pods. Solution: You must restart pods after updating the secret.Init container cannot pull alpine:3.20 image
Cause: Air-gapped environment or restricted registry access. Solution: Use your private registry invalues.yaml.
- Alpine-based:
alpine:3.20,alpine:3.19 - Debian-based:
ubuntu:22.04,debian:12 - Any image containing
/etc/ssl/certs/ca-certificates.crt(Debian/Ubuntu) or/etc/ssl/cert.pem(Alpine).
Advanced Configuration
Using a Different Secret Key Name
If your secret uses a different key name:Custom Mount Path
If you need to change where the CA bundle is mounted:
Note: Changing mountPath updates all environment variables automatically.
Multiple CA Certificates
If you need to trust multiple custom CAs, concatenate them into a single file:Air-Gapped Environments
For completely air-gapped deployments:- Mirror the init image to your private registry:
- Update values.yaml:
FAQ
Can I disable custom CA after enabling it?
Yes, setglobal.customCA.enabled: false and upgrade the Helm release. Pods will restart without the custom CA configuration.
Does this affect TLS between Dify components?
No, this only affects outbound HTTPS connections from Dify to external services (databases, S3, APIs, etc.). Internal component communication is not affected.What if my database also requires a custom CA?
The custom CA feature covers database connections too. Ensure your database server’s certificate is signed by the CA you’ve configured.Can I use intermediate certificates?
Yes, concatenate the full chain (intermediate + root CA) into a single file before creating the secret.How do I update the CA certificate without downtime?
Kubernetes Secrets can be updated, but pods must be restarted. For zero-downtime updates:- Update the secret.
- Use
kubectl rollout restartwith rolling update strategy (default). - Pods will restart one by one, maintaining service availability.