homelab-blueprint/docs/troubleshooting.md
Brian Pooe 1815beadf4 refactor: restructure docs, sync stacks with Caddyfile, harden compose templates
- Add compose stacks for apps already proxied by Caddy: bento-pdf,
  it-tools, paperless-ngx, and beszel-hub (with Makefile targets)
- Remove the unused Vault stack and the drawio Caddyfile block
- Fix Caddyfile gramps upstream to port 80 to match the compose file
- Bring immich and gramps-web up to the shared template pattern
  (no-new-privileges, log rotation, memory limits, healthchecks)
- Single-quote vaultwarden SSO_AUDIENCE_TRUSTED so regex values like
  ^\d{18}$ render as valid YAML
- Move root docs into kebab-case topic folders, rebuild the docs index
  to cover every doc, and fix all broken links
2026-07-03 17:45:16 +02:00

23 KiB
Raw Blame History

Docker Stack Troubleshooting Guide

Quick diagnostics

Run these first:

docker ps -a
docker stats --no-stream
docker network ls
docker-compose -f docker-compose.arr-stack.yml config

Common issues

1. Variables not substituted

Symptoms:

  • Generated file still contains {{VAR}}
  • Container fails due to missing env values

Fix:

./substitute_env.sh docker-compose-files/arr-stack/template.yaml docker-compose.arr-stack.yml .env

The script prints any unresolved placeholders. Add missing keys to .env (or provide defaults in templates).

2. Recyclarr errors (base_url must start with http)

Use service names, not localhost:

sonarr:
  sonarr-main:
    base_url: http://sonarr:8989

radarr:
  radarr-main:
    base_url: http://radarr:7878

Generate your file from this repo template:

./substitute_env.sh docker-compose-files/arr-stack/config/recyclarr_template.yml /opt/arr-stack/appdata/recyclarr/recyclarr.yml .env

2b. Recyclarr errors (Unable to find include template ...)

Symptoms:

  • Unable to find include template with name 'radarr-quality-definition-movie'
  • Similar errors for other include: - template: entries

Cause:

  • You are using an older pre-v8 recyclarr.yml that still has include: - template: entries.
  • Recyclarr v8 removed official include templates from the default provider.

Fix (v8 path used by this repository):

# 1) Regenerate recyclarr config from the v8 template
./substitute_env.sh docker-compose-files/arr-stack/config/recyclarr_template.yml /opt/arr-stack/appdata/recyclarr/recyclarr.yml .env

# 2) Regenerate compose and recreate recyclarr (v8.6.0 pin)
./substitute_env.sh docker-compose-files/arr-stack/template.yaml docker-compose.arr-stack.yml .env
docker-compose -f docker-compose.arr-stack.yml pull recyclarr
docker-compose -f docker-compose.arr-stack.yml up -d recyclarr

3. qBittorrent/SAB not using VPN IP

In this template, only download clients are expected behind VPN tunnel.

  • Behind VPN: gluetun, qbittorrent
  • Not behind VPN by default: radarr, sonarr, prowlarr, bazarr, emby, seerr

Checks:

docker logs gluetun | grep -i connected
docker exec qbittorrent curl -s ifconfig.me

4. Dependency/healthcheck startup failures

Symptoms:

  • dependency failed to start
  • Services wait forever on service_healthy

Checks:

docker inspect gluetun | grep -A 20 Health
docker inspect prowlarr | grep -A 20 Health
docker logs <container_name>

Fixes:

  • Increase start_period for slow-start services.
  • Test healthcheck command inside the container.
  • Validate generated compose YAML syntax with docker-compose ... config.

5. Network communication failures

Checks:

docker network inspect vpn_network
docker exec radarr wget -qO- http://prowlarr:9696/ping

Fixes:

  • Ensure relevant services are on vpn-network in generated compose.
  • Recreate stack networks:
docker-compose -f docker-compose.arr-stack.yml down
docker network prune
docker-compose -f docker-compose.arr-stack.yml up -d

5b. qBittorrent stuck in "Created" / network service:gluetun not found

Symptoms:

  • qbittorrent does not start after updates and remains in Created
  • Recreate shows: Error response from daemon: network service:gluetun not found

Cause:

  • network_mode: service:gluetun can fail during single-service recreate/update flows on some Docker setups.

Fix:

# Ensure template uses container mode for qbittorrent:
# network_mode: "container:gluetun"

# Re-render compose and recreate the pair
./substitute_env.sh docker-compose-files/arr-stack/template.yaml docker-compose.arr-stack.yml .env
docker compose -f docker-compose.arr-stack.yml up -d --force-recreate gluetun qbittorrent

Verify:

docker compose -f docker-compose.arr-stack.yml ps gluetun qbittorrent
docker compose -f docker-compose.arr-stack.yml logs --tail 100 qbittorrent

5c. qBittorrent fails with joining network namespace of container: No such container

Symptoms:

  • docker compose up -d qbittorrent exits with: Error response from daemon: joining network namespace of container: No such container: <hash>
  • Happens after gluetun was recreated (e.g., you ran docker compose up -d and gluetun got a new container ID because its config changed).

Cause:

  • qbittorrent stores the gluetun container ID at start time. When gluetun is recreated its container ID changes, but the old qbittorrent container still holds a reference to the obsolete ID. docker compose up -d leaves an already- running qbittorrent alone, so the stale reference is never cleared.

Fix — force-remove the stale container and recreate it:

docker rm -f qbittorrent
docker compose -f docker-compose.arr-stack.yml up -d qbittorrent

Prevention: any time you explicitly recreate gluetun, recreate qbittorrent in the same command so it picks up the new container ID:

docker compose -f docker-compose.arr-stack.yml up -d --force-recreate gluetun qbittorrent

5d. qBittorrent HTTPS 502 after stack updates

Symptoms:

  • qBittorrent opens via HTTPS reverse proxy URL and returns 502 after stack updates/recreates.

Common causes:

  • gluetun and qbittorrent are out of sync after recreate/update and need a paired restart.
  • qBittorrent WebUI can be left in HTTPS mode while your reverse proxy still targets HTTP upstream.

Fix:

# 1) Recreate the VPN + torrent pair together
docker-compose -f docker-compose.arr-stack.yml up -d --force-recreate gluetun qbittorrent

# 2) If 502 persists, force qBittorrent WebUI back to HTTP
CONF="$(find /opt/arr-stack/appdata/qbittorrent -type f -name 'qBittorrent.conf' | head -n1)"
echo "Using config: $CONF"
sudo sed -i 's#^WebUI\\HTTPS\\Enabled=.*#WebUI\\HTTPS\\Enabled=false#' "$CONF"
sudo sed -i 's#^WebUI\\HTTPS\\CertificatePath=.*#WebUI\\HTTPS\\CertificatePath=#' "$CONF"
sudo sed -i 's#^WebUI\\HTTPS\\KeyPath=.*#WebUI\\HTTPS\\KeyPath=#' "$CONF"
docker-compose -f docker-compose.arr-stack.yml restart qbittorrent

Verify:

docker-compose -f docker-compose.arr-stack.yml ps gluetun qbittorrent
docker-compose -f docker-compose.arr-stack.yml logs --tail 120 qbittorrent

5e. Radarr/Sonarr reports a missing qBittorrent download path

Symptoms:

  • Health reports that qBittorrent places downloads in a path that does not exist inside Radarr or Sonarr.
  • qBittorrent logs contain Permission denied for paths such as /data/.torrents/....
  • The same torrent repeatedly rechecks or appears to restart because qBittorrent first tries the stale path, fails the move, then resumes against the corrected incomplete path.

Checks:

grep -E 'DefaultSavePath|TempPath' /opt/arr-stack/appdata/qbittorrent/config/qBittorrent.conf
docker exec radarr ls -ld /data/torrents /data/torrents/movies /data/torrents/tv
docker exec sonarr ls -ld /data/torrents /data/torrents/movies /data/torrents/tv

Required qBittorrent settings:

Default save path: /data/torrents
Incomplete path:   /data/torrents/incomplete
movies category:  /data/torrents/movies
tv category:      /data/torrents/tv

Do not use /data/.torrents, and do not add remote path mappings when all containers use the documented /data layout. After correcting an existing torrent's location, force recheck and resume it, then rerun the Radarr/Sonarr health check.

5f. No indexers available with RSS sync enabled

Cause:

  • Prowlarr indexers use an app profile such as Automatic Search that has RSS disabled. Prowlarr synchronizes that disabled flag to Radarr and Sonarr.

Fix:

  1. In Prowlarr, edit each affected indexer.
  2. Set App Profile to Standard, or another profile with RSS, automatic search, and interactive search enabled.
  3. Open Settings > Apps and run Sync App Indexers for Radarr and Sonarr.
  4. Confirm the synchronized indexers show RSS enabled in both applications.

5g. Library Import leaves unmatched movie or series folders

Radarr expects one movie per folder. Split trilogy, franchise, and collection packs into individual Movie Title (Year) folders before importing them. Sample clips inside a movie folder can be mistaken for the feature; move or remove samples before rescanning.

Sonarr permits only one root path per series. If duplicate series folders exist, compare their episode coverage and quality, merge any unique episodes into the preferred folder, update the series path, and verify the episode-file count before removing the obsolete folder.

Do not delete unmatched folders solely because Library Import skipped them. First verify whether they contain unique media, higher-quality replacements, or files that were matched to the wrong title. Use exact TMDB/TVDB matches for ambiguous titles.

5h. Download appears to restart mid-progress (qBittorrent OOM-killed)

Symptoms:

  • A torrent regresses to a lower progress percentage mid-download (commonly at 6080 % on large files).
  • qbittorrent.log shows a new qBittorrent v... started. line with no preceding error, roughly 1015 minutes after the previous start.
  • docker inspect qbittorrent --format '{{.State.OOMKilled}}' returns true.
  • docker stats --no-stream qbittorrent shows memory near or at its limit.

Cause:

  • qBittorrent buffers piece data in memory during download and hash-verification. For large torrents (20 GB+) this can exceed the 1 GiB container limit, causing the cgroup OOM killer to terminate the process. s6-overlay restarts qBittorrent inside the same container (Docker restart count stays at 0), which triggers a full piece re-verify from disk, visually resetting progress.

Fix — raise the limit live without restarting the container or losing progress:

docker update --memory 3g --memory-swap 6g qbittorrent

Then persist the change in the compose file and re-render from the template (the template already sets 3 GiB):

./substitute_env.sh docker-compose-files/arr-stack/template.yaml docker-compose.arr-stack.yml .env

Do not lower memory below 2G for a stack that downloads files larger than 10 GB.

5j. Download appears to restart mid-progress (Decluttarr auth ban loop)

Symptoms:

  • A torrent that was at, say, 60 % progress appears to restart from a lower percentage after qBittorrent logs show a process restart.
  • qbittorrent.log contains a long run of:
    (W) WebAPI login failure. Reason: IP has been banned, IP: ::ffff:172.18.0.12
    
    followed by qBittorrent v... started.
  • docker logs decluttarr shows repeated 401/403 errors against http://gluetun:8080/api/v2/auth/login.

Cause:

  • Decluttarr has blank (or wrong) QBITTORRENT_USERNAME / QBITTORRENT_PASSWORD in .env. It polls qBittorrent every 30 seconds; after several failures qBittorrent bans 172.18.0.0/16 (the Docker bridge), which forces a qBittorrent process restart. After restart, qBittorrent re-verifies pieces on disk, which can drop the reported progress before it climbs back.

Fix — whitelist the Docker bridge in qBittorrent so internal containers bypass authentication:

# Run from inside the qbittorrent container (LocalHostAuth=false means no creds needed from localhost)
docker exec qbittorrent curl -s -X POST 'http://localhost:8080/api/v2/app/setPreferences' \
  -d 'json={"bypass_auth_subnet_whitelist":"10.0.10.0/24\n172.18.0.0/16","bypass_auth_subnet_whitelist_enabled":true}'

# Verify
docker exec qbittorrent curl -s 'http://localhost:8080/api/v2/app/preferences' \
  | python3 -c "import json,sys; p=json.load(sys.stdin); print(p.get('bypass_auth_subnet_whitelist'))"

qBittorrent saves this to its persisted config on the volume so it survives container restarts. Alternatively, fill in QBITTORRENT_USERNAME and QBITTORRENT_PASSWORD in .env, re-render the compose, and recreate the stack.

Verify fix:

docker restart decluttarr
docker logs decluttarr --tail 20
# Should show: INFO | OK | qBittorrent (http://gluetun:8080)

5k. Sonarr/Radarr warning: "Download client qBittorrent is set to remove completed downloads"

Symptoms:

  • System status in Sonarr or Radarr shows: Download client qBittorrent is set to remove completed downloads. This can result in downloads being removed from your client before Sonarr can import them.

Cause:

  • Session\ShareLimitAction=Remove in qBittorrent.conf. When seeding limits are reached (ratio or time), qBittorrent removes the torrent entry before Sonarr/Radarr can import it.
  • The qBittorrent API setPreferences call does not persist max_ratio_act to disk in the hotio image. The value must be changed directly in the config file.

Fix — edit the config file and recreate the container:

# 1. Edit the config
sed -i 's/Session\\ShareLimitAction=Remove/Session\\ShareLimitAction=Pause/' \
  /opt/arr-stack/appdata/qbittorrent/config/qBittorrent.conf

# Verify
grep 'ShareLimitAction' /opt/arr-stack/appdata/qbittorrent/config/qBittorrent.conf
# Expected: Session\ShareLimitAction=Pause

# 2. Recreate qBittorrent (must use rm + up, not restart, due to gluetun network_mode)
cd /opt/homelab-blueprint
docker rm -f qbittorrent
docker compose -f docker-compose.arr-stack.yml up -d qbittorrent

With Pause set, qBittorrent pauses the torrent when limits are hit instead of removing it. Sonarr and Radarr then import the file and signal qBittorrent to remove it themselves.

Verify the warning clears in Sonarr/Radarr System → Status (may need a minute for the health check to re-run).

Note: The default seeding limits in this stack are GlobalMaxRatio=0.1 and GlobalMaxSeedingMinutes=20. These are intentionally aggressive for public trackers. Raise them in qBittorrent Settings → BitTorrent if you use private trackers, which typically require a minimum ratio of 1.0.

5l. Sonarr "Downloaded - Waiting to Import" — Episode has a TBA title and recently aired

Symptoms:

  • A completed download sits in the Sonarr queue under Activity → Queue with status Downloaded - Waiting to Import.
  • The status message reads: Episode has a TBA title and recently aired.
  • The episode title in TVDB is still TBA (common for hot new episodes within 2448 hours of airing).

Cause:

  • Sonarr's Episode Title Required setting (Settings → Media Management) is set to Always. When TVDB hasn't published the real title yet, Sonarr holds the download rather than importing it with "TBA" in the filename.

Fix — change the setting to allow single-episode imports even with a TBA title:

curl -s -X PUT 'http://localhost:8989/api/v3/config/mediamanagement' \
  -H 'X-Api-Key: <your-sonarr-api-key>' \
  -H 'Content-Type: application/json' \
  -d '{"episodeTitleRequired": "bulkSeasonReleases", ... <rest of current config>}'

Or via the UI: Settings → Media Management → Episode Title Required → Bulk Season Releases.

Value Behaviour
Always Block import if title is TBA — safest naming, but you wait
Bulk Season Releases (recommended) Only block for full-season dumps; allow single-episode imports even if TBA
Never Always import regardless of title

After changing the setting, Sonarr will automatically process any held queue items within its next scan cycle.

If you already have a TBA file imported and want to rename it:

Once TVDB publishes the real title, go to Series → Refresh & Scan in Sonarr and it will rename the file automatically. To rename immediately using a title from an external source (e.g., Overseerr):

# On the Proxmox host, via pct exec into the LXC:
OLD='/data/media/tv/Series Name/Season 03/Series Name (Year) - S03E01 - TBA [...].mkv'
NEW='/data/media/tv/Series Name/Season 03/Series Name (Year) - S03E01 - Real Title [...].mkv'
pct exec 101 -- mv "$OLD" "$NEW"

# Then trigger a Sonarr rescan to update the database path
curl -s -X POST 'http://10.0.10.20:8989/api/v3/command' \
  -H 'X-Api-Key: <your-sonarr-api-key>' \
  -H 'Content-Type: application/json' \
  -d '{"name": "RescanSeries", "seriesId": <id>}'

5m. Prowlarr indexer recommendations and usenet retention gaps

When usenet articles expire (typical retention: 1214 months), SABnzbd logs Article ... unavailable on all servers, discarding and the download shows an ETA of hundreds of days. Public torrent indexers serve as a fallback for older content.

Active indexer list (as of 2026-06-22):

Indexer Type Priority Notes
NZBgeek Usenet 1 Primary usenet source
Nzb.life Usenet 2 Secondary usenet source
altHUB Usenet 3 Tertiary usenet source
1337x Torrent 20 Requires FlareSolverr (Cloudflare-protected)
EZTV Torrent 10 TV-specific; requires FlareSolverr (primary URL eztvx.to is Cloudflare-protected)
Knaben Torrent 40 Aggregator; good coverage of older content
The Pirate Bay Torrent 30 Broad fallback; magnet-based

FlareSolverr must be running for 1337x and EZTV — both use eztvx.to/Cloudflare-protected primary URLs. FlareSolverr is included in the arr-stack compose and is configured in Prowlarr under Settings → Indexers → FlareSolverr Proxy (tag ID 1). Assign this tag to any Cardigann indexer whose primary URL returns a Cloudflare challenge.

To add a missing indexer via the Prowlarr API:

# 1. Get the schema for the indexer you want to add
curl -s 'http://10.0.10.20:9696/api/v1/indexer/schema' \
  -H 'X-Api-Key: <prowlarr-api-key>' | jq '[.[] | {id: .id, name: .name}]'

# 2. Pull the specific schema
curl -s 'http://10.0.10.20:9696/api/v1/indexer/schema' \
  -H 'X-Api-Key: <prowlarr-api-key>' \
  | jq '.[] | select(.name == "EZTV")'

# 3. POST it (set enable: true and appProfileId: 1)
curl -s -X POST 'http://10.0.10.20:9696/api/v1/indexer' \
  -H 'X-Api-Key: <prowlarr-api-key>' \
  -H 'Content-Type: application/json' \
  -d '<schema JSON with enable: true>'

# 4. Sync to Sonarr and Radarr
curl -s -X POST 'http://10.0.10.20:9696/api/v1/applications/sync' \
  -H 'X-Api-Key: <prowlarr-api-key>'

If EZTV shows "Indexer Disabled / Initial Failure":

EZTV sometimes rate-limits or returns errors on first test. The 1-minute disabled-until window is automatic. To re-enable:

  1. In Prowlarr, go to Indexers and click the EZTV pencil icon.
  2. Click Test — if it passes, save and the indexer re-enables.
  3. If the test still fails, check that eztv.re is reachable from inside gluetun (VPN IP may be blocked by EZTV). In that case, try the alternate URL eztvx.to in the indexer base URL field.

Private trackers (upgrade path):

If you gain access to BTN (BroadcastTheNet) or TorrentLeech, add them via Prowlarr — they have far better TV coverage and ratio requirements over EZTV/1337x for current-season content.

6. Data/config persistence problems

Checks:

docker inspect radarr | grep -A 20 Mounts
ls -la /opt/arr-stack/appdata/radarr

Fixes:

mkdir -p /opt/arr-stack/appdata/radarr
chown -R <PUID>:<PGID> /opt/arr-stack/appdata/radarr

For stacks that use named volumes, inspect them with docker volume ls rather than changing ownership on an old host appdata directory.

7. Caddy reverse proxy issues

If Caddy refuses to start, validate syntax first:

./substitute_env.sh docker-compose-files/caddy/Caddyfile_template docker-compose-files/caddy/Caddyfile .env
./substitute_env.sh docker-compose-files/caddy/template.yaml docker-compose.caddy.yml .env
chmod 644 docker-compose-files/caddy/Caddyfile
docker compose -f docker-compose.caddy.yml run --rm caddy \
  caddy validate --config /etc/caddy/Caddyfile

If validation reports open /etc/caddy/Caddyfile: permission denied, the template renderer created the generated file with mode 600, and the container's mapped user cannot read it. Apply the chmod 644 command above. The generated Caddyfile does not contain the Cloudflare token. Do not apply that mode to docker-compose.caddy.yml, which does contain the rendered token.

If logs report mkdir /data/caddy: permission denied or mkdir /config/caddy: permission denied, the generated Compose file still uses host bind mounts that are not writable through Docker's user-namespace mapping. Re-render it from the current Caddy template, which uses named volumes, then recreate the container:

./substitute_env.sh docker-compose-files/caddy/template.yaml docker-compose.caddy.yml .env
docker compose -f docker-compose.caddy.yml down
docker compose -f docker-compose.caddy.yml up -d --build
docker compose -f docker-compose.caddy.yml logs --tail=100 caddy

The same risk applies to any writable bind mount when Docker uses rootless or user-namespace mapping. The Caddy, Beszel Agent, PostgreSQL/pgAdmin, Technitium, Gramps Web, Home Assistant, and Mosquitto templates use named volumes for application state to avoid this issue.

Arr Stack, Gramps media, Vault config/TLS, and other explicit external storage mounts intentionally remain host-managed. Verify their ownership and access on the target Docker installation; do not replace those mounts with named volumes without planning configuration edits, backups, and existing-data migration.

Before changing an existing bind-mount deployment to named volumes, stop the stack and copy its current data. Starting against a new empty named volume does not import the old bind-mounted data automatically.

If certificate issuance fails with errors like:

  • could not determine zone for domain ... / expected 1 zone, got 0
  • dial tcp 1.0.0.1:53: i/o timeout

these are two distinct failure modes with the same underlying cause: Caddy's DNS-01 resolver cannot reach 1.1.1.1/1.0.0.1 on port 53.

Distinguish the two errors:

Error Meaning
expected 1 zone, got 0 Caddy reached a resolver, but it was Technitium (split-DNS). NAT exception misconfigured or alias IP wrong.
dial tcp 1.0.0.1:53: i/o timeout Caddy's packets are being dropped. A firewall block rule on OPT2 is above the pass rule, or the NAT exception is not set up at all.

Check DNS reachability from the Caddy host:

# Must return Cloudflare SOA, not Technitium SOA (dns01.home.example.com)
dig +short SOA home.example.com @1.1.1.1

# Distinguish firewall drop from redirect:
dig @1.1.1.1 +tcp +time=5 google.com   # timeout = firewall drop
dig @1.1.1.1 +notcp +time=5 google.com # timeout = same; success = TCP-specific block

Keep public resolvers in the Caddy TLS block — do not switch to Technitium as a workaround, as it causes expected 1 zone, got 0 because Technitium is authoritative for the internal zone:

tls you@example.com {
  dns cloudflare {env.CLOUDFLARE_DNS_TOKEN}
  resolvers 1.1.1.1 1.0.0.1
}

If your network forces DNS redirection, exclude Caddy from that redirect and allow it to query public resolvers on TCP and UDP port 53.

If a proxied app fails over HTTPS with self-signed backend certs, ensure the target block uses:

transport http {
  tls_insecure_skip_verify
}

Update flow (safe)

  1. Generate compose files from templates.
  2. Validate with docker-compose -f <file> config.
  3. Pull images.
  4. Recreate services.
  5. Review logs after startup.
docker-compose -f docker-compose.arr-stack.yml pull
docker-compose -f docker-compose.arr-stack.yml up -d

Last Updated: 2026-06-22