Deepgram - Cerebrium

Cerebrium’s partnership with Deepgram enables simple deployment of speech-to-text (STT) services with simplified configuration and independent scaling.

Using Deepgram services requires an Enterprise Deepgram account and API key for self-hosted models. Contact Deepgram support to access this feature.Obtain links to the Deepgram model files referenced below (file extension .dg) from the Deepgram Account Representative.Consult the Deepgram representative on how to achieve parity with the Deepgram API.

Deepgram Partner Service is available from CLI version 1.39.0 and greater

The Deepgram Partner Service is in beta. It’s available to all users and ready for production workloads, but expect occasional rough edges while the integration matures. Reach out to support if you hit any issues.

Setup

Create a Cerebrium app with the CLI:

cerebrium init deepgram

Create a self-hosted API key from the Deepgram dashboard. Navigate to the Secrets tab in the Cerebrium dashboard and add the API key with the name DEEPGRAM_API_KEY. This secret automatically becomes available as an environment variable in the deployment.
Download model files from Deepgram’s self-hosted section in the Deepgram dashboard using the guide available, here. Select the ‘license proxy’ deployment type. Upload downloaded model files using the links provided by your Account Representative (with .dg extension) to persistent-storage in the /deepgram-models folder. This folder automatically attaches to the engine container. Use this command to upload the files:

cerebrium cp <model>.dg deepgram-models/<model>.dg

#example
cerebrium cp nova-3-general.en.streaming.123456.dg deepgram-models/nova-3-general.en.streaming.123456.dg

The deepgram-models directory remains at the root level of persistent storage and is shared across all Deepgram apps in the project. Configuration files (api.toml and engine.toml), however, must be placed under the app name directory (see steps 4 and 5 below).

Create a file named engine.toml with the following content and upload to your persistent storage under the app name directory (e.g., {appName}/engine.toml). These are the default settings. Adjust as needed. For example, if your app is named deepgram:
cerebrium cp engine.toml deepgram/engine.toml

### Keep in mind that all paths are in-container paths and do not need to exist
### on the host machine.

### Limit the number of active requests handled by a single Engine container.
### Engine will reject additional requests from API beyond this limit, and the
### API container will continue with the retry logic configured in `api.toml`.
###
### The default is no limit.
# max_active_requests =


### Configure license validation by passing in a DEEPGRAM_API_KEY environment variable
### See https://developers.deepgram.com/docs/deploy-deepgram-services#credentials
[license]
server_url = ["https://license.deepgram.com"]


### Configure the server to listen for requests from the API.
[server]
### The IP address to listen on. Since this is likely running in a Docker
### container, you will probably want to listen on all interfaces.
host = "0.0.0.0"
### The port to listen on
port = 8055


### To support metrics we need to expose an Engine endpoint.
### See https://developers.deepgram.com/docs/metrics-guide#deepgram-engine
[metrics_server]
host = "0.0.0.0"
port = 9991


[model_manager]
### The number of models to have concurrently loaded in system memory.
### If managing a deployment with dozens of models this setting will
### help prevent instances where models consume too much memory and
### offload the models to disk as needed on a least-recently-used basis.
###
### The default is no limit.
# max_concurrently_loaded_models = 20

### Inference models. You can place these in one or multiple directories.
search_paths = ["/models"]


### Enable ancillary features
[features]
### Allow multichannel requests by setting this to true, set to false to disable
multichannel = true # or false
### Enables language detection *if* a valid language detection model is available
language_detection = true # or false
### Enables streaming entity formatting *if* a valid NER model is available
streaming_ner = false # or true

### Size of audio chunks to process in seconds.
[chunking.batch]
# min_duration =
# max_duration =
[chunking.streaming]
# min_duration =
# max_duration =

### How often to return interim results, in seconds. Default is 1.0s.
###
### This value may be lowered to increase the frequency of interim results.
### However, this may cause a signficant decrease in number of concurrent
### streams supported by a single GPU. Please contact your Deepgram Account
### representative for more details.
step = 0.2


### Engine will automatically enable half precision operations if your GPU supports
### them. You can explicitly enable or disable this behavior with the state parameter
### which supports enabled, disabled, and auto (the default).
[half_precision]
    state = "auto"
# state = "disabled" # or "enabled" or "auto"

Create a file named api.toml with the following content and upload to your persistent storage under the app name directory (e.g., {appName}/api.toml). These are the default settings. Adjust as needed. For example, if your app is named deepgram:
cerebrium cp api.toml deepgram/api.toml

### Keep in mind that all paths are in-container paths, and do not need to exist
### on the host machine.


### Configure license validation by passing in a DEEPGRAM_API_KEY environment variable
### See https://developers.deepgram.com/docs/deploy-deepgram-services#credentials
[license]
server_url = ["https://license.deepgram.com"]


### Configure how the API will listen for your requests
[server]
### The base URL (prefix) for requests to the API.
base_url = "/v1"
### The IP address to listen on. Since this is likely running in a Docker
### container, you will probably want to listen on all interfaces.
host = "0.0.0.0"
### The port to listen on
port = 8082

### How long to wait for a connection to a callback URL.
callback_conn_timeout = "1s"
### How long to wait for a response to a callback URL.
callback_timeout = "10s"

### How long to wait for a connection to a fetch URL.
fetch_conn_timeout = "1s"
### How long to wait for a response to a fetch URL.
fetch_timeout = "60s"


### By default, the API listens over HTTP. By passing both a certificate
### and a key file, the API will instead listen over HTTPS.
###
### This performs TLS termination only, and does not provide any
### additional authentication.
[server.https]
# cert_file = "/path/to/cert.pem"
# key_file = "/path/to/key.pem"


### Specify custom DNS resolution options.
[resolver]
### Specify custom domain name server(s).
### Format is "{IP} {PORT} {PROTOCOL (tcp or udp)}"
# nameservers = ["127.0.0.1 53 udp"]

### If specifying a custom DNS nameserver, set the DNS TTL value.
# max_ttl = 10


### Limit the number of active requests handled by a single API container.
### If additional requests beyond the limit are sent, API will return
### a 429 HTTP status code. Default is no limit.
[concurrency_limit]
# active_requests =


### Enable ancillary features
[features]
### Enables topic detection *if* a valid topic detection model is available
topic_detection = true # or false

### Enables summarization *if* a valid summarization model is available
summarization = true # or false

### Enables pre-recorded entity detection *if* a valid entity detection model is available
entity_detection = false # or true

### Enables pre-recorded entity-based redaction *if* a valid entity detection model is available
entity_redaction = false # or true

### Enables pre-recorded entity formatting *if* a valid NER model is available
format_entity_tags = false # or true

### If API is receiving requests faster than Engine can process them, a request
### queue will form. By default, this queue is stored in memory. Under high load,
### the queue may grow too large and cause Out-Of-Memory errors. To avoid this,
### set a disk_buffer_path to buffer the overflow on the request queue to disk.
###
### WARN: This is only to temporarily buffer requests during high load.
### If there is not enough Engine capacity to process the queued requests over time,
### the queue (and response time) will grow indefinitely.
# disk_buffer_path = "/path/to/disk/buffer/directory"

### Enables streaming TTS *if* a valid Aura TTS model is available
speak_streaming = true # or false

### Configure the backend pool of speech engines (generically referred to as
### "drivers" here). The API will load-balance among drivers in the standard
### pool; if one standard driver fails, the next one will be tried.
###
### Each driver URL will have its hostname resolved to an IP address. If a domain
### name resolves to multiple IP addresses, the API will load-balance across each
### IP address.
###
### This behavior is provided for convenience, and in a production environment
### other tools can be used, such as HAProxy.
###
### Below is a new Speech Engine ("driver") in the "standard" pool.
[[driver_pool.standard]]
### Host to connect to. If you are using a different method of orchestrating,
### then adjust the IP address accordingly.
###
### WARN: This must be HTTPS.
###
### Docker Compose and Podman Compose create a dedicated network that allows inter-container communication by app name.
### See [Networking in Compose](https://docs.docker.com/compose/networking/) for details.
url = "https://0.0.0.0:8055/v2"
### Factor to increase the timeout by for each additional retry (for
### exponential backoff).
timeout_backoff = 1.2

### Before attempting a retry, sleep for this long (in seconds)
retry_sleep = "2s"
### Factor to increase the retry sleep by for each additional retry (for
### exponential backoff).
retry_backoff = 1.6

### Maximum response to deserialize from Driver (in bytes)
max_response_size = 1073741824 # 1GB

Update cerebrium.toml with the following configuration for hardware, scaling, region, and other settings:

[cerebrium.deployment]
name = "deepgram"
# Enable below in production environments
disable_auth = true

[cerebrium.runtime.deepgram]

[cerebrium.hardware]
cpu = 4
memory = 32
compute = "AMPERE_A10"
gpu_count = 1

[cerebrium.scaling]
min_replicas = 0
max_replicas = 2
cooldown = 120
replica_concurrency = 150

Run cerebrium deploy. After deployment, an endpoint for the Deepgram services appears in the terminal output. The URL is also available on the app’s overview page in the dashboard.
Download an example audio file for use with the deepgram service:

wget https://dpgr.am/bueller.wav

Call the Deepgram endpoint with appropriate parameters:

curl -X POST --data-binary @bueller.wav "https://api.cerebrium.ai/v4/p-xxxxxxxx/deepgram/v1/listen?model=nova-3&smart_format=true"

Parameters accepted by the Deepgram service can be found in the speech-to-text API reference.

If ‘disable_auth’ in cerebrium.toml is set to false, include the inference token in the Authorization header to authenticate with the Cerebrium service. The Deepgram API key is pulled automatically from secrets.

API Key Configuration

To use Deepgram services:

Sign up at deepgram.com
Create an API key in the Deepgram dashboard
Add the API key to Cerebrium:

Navigate to Secrets tab in the Cerebrium dashboard
Add the Deepgram API key as an app-specific or project-wide secret named DEEPGRAM_API_KEY
This secret automatically becomes available as an environment variable in the deployment

Scaling and Concurrency

Deepgram services support independent scaling configurations:

min_replicas: Minimum number of instances to maintain (0 for scale-to-zero)
max_replicas: Maximum number of instances that can be created during high load
replica_concurrency: Number of concurrent requests each instance can handle
cooldown: Time window (in seconds) that must pass at reduced concurrency before scaling down

Adjust these parameters based on traffic patterns and latency requirements.

Usage Examples

Cerebrium runs both Deepgram STT models and applications on the same network alongside LiveKit workers, reducing latency by approximately 400ms—a significant advantage for voice agent applications. For a complete implementation reference, see the LiveKit Outbound Agent example.

​Setup

​API Key Configuration

​Scaling and Concurrency

​Usage Examples

Setup

API Key Configuration

Scaling and Concurrency

Usage Examples