LiteLLM

Deploys LiteLLM Proxy — a unified OpenAI-compatible gateway to 100+ LLM providers — via Docker Compose, backed by PostgreSQL for virtual keys, spend tracking, budgets and Admin UI model management.

Requirements

• Docker
• community.docker Ansible collection (>=5.2.0)

Variables

Variable	Default	Description
litellm_instance	litellm	Unique instance name. Used as Compose project name and container names.
litellm_data_path	/data/litellm	Host path for persistent data (PostgreSQL data).
litellm_docker_path	/docker/litellm	Host path for the Compose file and rendered config.yaml.
litellm_image	ghcr.io/berriai/litellm	LiteLLM proxy image. Override for a private mirror/registry.
litellm_version	v1.98.0	LiteLLM proxy image tag. Pin to a specific version in production.
litellm_pg_version	16	PostgreSQL image tag. Pin to a specific version in production.
litellm_domain	~	Public domain. When set, enables Traefik routing labels and attaches to the traefik_web network.
litellm_traefik_certresolver	default	Traefik certificate resolver to use for litellm_domain.
litellm_port	~	When set, exposes LiteLLM on this host port (maps to container port 4000). Omit when using litellm_domain.
litellm_pg_user	litellm	PostgreSQL user, password owner and database name (single value used for all three).
litellm_pg_pass	~	PostgreSQL password. Required. Generate with: openssl rand -hex 32
litellm_master_key	~	Admin/gateway API key. Required, must start with sk-. Generate with: echo "sk-$(openssl rand -hex 32)"
litellm_salt_key	~	Encrypts provider credentials stored in the DB. Required. Set once, never change. Generate with: openssl rand -hex 32
litellm_loglevel	WARNING	Log verbosity. One of: DEBUG, INFO, WARNING, ERROR.
litellm_num_workers	4	Uvicorn worker processes. See Notes for how this interacts with connection pooling.
litellm_model_list	~	Required. Raw YAML block inserted as model_list: in config.yaml. See Usage.
litellm_general_settings	pool/batching tuning	Raw YAML block inserted as general_settings: in config.yaml.
litellm_settings	retries/timeouts/keepalive	Raw YAML block inserted as litellm_settings: in config.yaml.
litellm_router_settings	~	Raw YAML block inserted as router_settings: in config.yaml. Unset by default.
litellm_extra_env	{}	Dict of extra environment variables for the proxy container (e.g. provider API keys).
litellm_uninstall	false	Stops and removes containers and the compose directory.
litellm_purge	false	Also removes the data directory (including the database). Requires litellm_uninstall: true. Irreversible.

Usage

Minimal setup:

litellm_pg_pass: "{{ vault_litellm_pg_pass }}"
litellm_master_key: "{{ vault_litellm_master_key }}"
litellm_salt_key: "{{ vault_litellm_salt_key }}"
litellm_domain: "llm.example.com"
litellm_model_list: |
  - model_name: gpt-4o
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY
  - model_name: claude-sonnet
    litellm_params:
      model: anthropic/claude-sonnet-5
      api_key: os.environ/ANTHROPIC_API_KEY
litellm_extra_env:
  OPENAI_API_KEY: "{{ vault_openai_api_key }}"
  ANTHROPIC_API_KEY: "{{ vault_anthropic_api_key }}"

Without a reverse proxy (direct host port instead of litellm_domain):

litellm_port: 4000

Overriding retries/timeouts/fallbacks (merged with the defaults, since this fully replaces the block):

litellm_settings: |
  request_timeout: 600
  num_retries: 2
  set_verbose: false
  json_logs: true
  sse_keepalive_ping_interval_seconds: 15
  fallbacks:
    - gpt-4o: [claude-sonnet]

Uninstall

To stop and remove all containers and the compose directory:

litellm_uninstall: true

Add litellm_purge: true to also delete the database.

Notes

• No Redis. Redis is only needed to coordinate rate-limit state, router state and caching across multiple replicas. This role runs a single proxy instance, so it's deliberately left out to keep the stack minimal. If you later scale to multiple replicas, add a Redis service and wire REDIS_HOST/REDIS_PORT/REDIS_PASSWORD into litellm_router_settings and litellm_settings yourself.
• Postgres is required, not optional. Without it, virtual keys, budgets, spend tracking and Admin UI model management don't work. It's included by default rather than made toggleable, since "all the features" was the goal. Postgres is only reachable on the internal Docker network — no host port is published.
• Worker count vs. connection pool. Each Uvicorn worker (litellm_num_workers) holds its own Postgres connection pool (database_connection_pool_limit in litellm_general_settings, default 10). Total possible connections is roughly litellm_num_workers * database_connection_pool_limit; keep this comfortably under Postgres' max_connections (100 by default). The defaults (4 workers × 10) land at 40, well within range — raise litellm_num_workers on bigger hosts, but lower the pool limit if you also raise worker count a lot.
• Streaming timeouts. sse_keepalive_ping_interval_seconds (default 15, in litellm_settings) makes the proxy send SSE ping frames during silent gaps in a streaming response, so a reverse proxy's idle-connection timeout (Traefik defaults to 180s) doesn't truncate a slow model's output mid-stream.
• config.yaml + Admin UI together. STORE_MODEL_IN_DB=True is always set, so litellm_model_list seeds the baseline models on every deploy (infrastructure-as-code), while additional models can still be added at runtime through the Admin UI/API — those are stored in Postgres and layered on top.
• litellm_model_list changes restart the container. config.yaml is a bind-mounted file the compose service definition itself doesn't reference, so a content-only change doesn't trigger a recreate on its own — a handler restarts the container whenever the rendered file changes, so a new/changed model is live immediately after the play finishes.
• litellm_salt_key is permanent. It encrypts provider credentials in the database. Changing it after the first deploy makes existing stored credentials unreadable.
• DB outage behaviour. By default, a Postgres outage fails closed (auth/spend checks fail, matching LiteLLM's default). If you want the proxy to keep serving on the master key while the DB is down — recommended only when LiteLLM is not reachable from the public internet — add allow_requests_on_db_unavailable: true to litellm_general_settings.
• Containers are not proxied by Traefik unless litellm_domain is set — it must already have a traefik_web network and a matching certresolver (see the traefik role).