130 lines
3.8 KiB
Markdown
130 lines
3.8 KiB
Markdown
# Troubleshooting Guide
|
|
|
|
## Permission Denied / Readonly Database / Not Writable
|
|
|
|
Common causes:
|
|
- App placed on NFS (tank) when it needs local SSD
|
|
- NFS ownership mismatch between host user and container user
|
|
|
|
Checks:
|
|
1. `docker logs <container> --tail 30`
|
|
2. `ls -la <host-appdata-path>` — check ownership and permissions
|
|
3. `docker run --rm --entrypoint sh <image> -lc 'id'` — check container UID/GID
|
|
|
|
Fix options:
|
|
- Move appdata to `/mnt/docker-ssd/docker/appdata/<service>` if write-heavy
|
|
- `sudo chmod -R 777 <appdata-path>` for apps with fixed internal user (e.g. shlink)
|
|
- `sudo chown -R <uid>:<gid> <appdata-path>` if UID/GID is known
|
|
|
|
## SQLite Fails on ZFS with nfsv4 ACLs (SQLITE_CANTOPEN error 14)
|
|
|
|
Symptom: `unable to open database file: out of memory (14)`
|
|
|
|
Cause: SQLite cannot acquire POSIX file locks on ZFS datasets configured with `acltype=nfsv4`.
|
|
|
|
Diagnosis:
|
|
```bash
|
|
sudo zfs get acltype,aclmode,xattr <dataset>
|
|
```
|
|
|
|
Fix options:
|
|
1. Change acltype: `sudo zfs set acltype=posix <dataset>`
|
|
2. Move the SQLite database to local SSD
|
|
3. Use an app version that supports Postgres instead of SQLite
|
|
|
|
Note: scriberr CUDA image is shelved for this reason.
|
|
|
|
## Container Can't Resolve Shared Database Hostname
|
|
|
|
Symptom: `lookup shared-postgres on 127.0.0.11:53: no such host`
|
|
|
|
Fix:
|
|
```bash
|
|
docker compose --env-file .env down <service>
|
|
docker compose --env-file .env up -d <service>
|
|
```
|
|
|
|
Verify network attachment:
|
|
```bash
|
|
docker inspect <container> --format '{{json .NetworkSettings.Networks}}' | tr ',' '\n' | grep -o '"[^"]*"'
|
|
```
|
|
|
|
## Postgres Auth Fails Over TCP but Works via Socket
|
|
|
|
Diagnosis:
|
|
```bash
|
|
sudo docker exec shared-postgres psql -h 127.0.0.1 -U <user> -d <db> -W -c "\conninfo"
|
|
```
|
|
|
|
Fix:
|
|
- Verify pg_hba.conf has `host all all all scram-sha-256`
|
|
- Check DATABASE_URL password matches: `ALTER USER <user> WITH PASSWORD '<pass>';`
|
|
- Avoid special characters in passwords in DATABASE_URLs
|
|
|
|
## Healthcheck Failing Despite Container Being Healthy
|
|
|
|
Diagnosis:
|
|
```bash
|
|
sudo docker exec <container> sh -c "which curl; which wget" 2>&1
|
|
sudo docker inspect <container> --format '{{json .State.Health.Log}}' | python3 -m json.tool | tail -20
|
|
```
|
|
|
|
After fixing the compose file, must do `down && up` (not just restart).
|
|
|
|
## App Ignores Environment Variables for DB Type
|
|
|
|
Some apps (e.g. donetick) require a YAML config file for certain settings even when env vars are set.
|
|
|
|
For donetick, create `/mnt/tank/docker/appdata/donetick/config/selfhosted.yaml`:
|
|
```yaml
|
|
database:
|
|
type: postgres
|
|
host: shared-postgres
|
|
port: 5432
|
|
user: donetick
|
|
password: <password>
|
|
name: donetick
|
|
migration: true
|
|
jwt:
|
|
secret: <32-byte-hex>
|
|
session_time: 168h
|
|
max_refresh: 168h
|
|
server:
|
|
port: 2021
|
|
serve_frontend: true
|
|
```
|
|
|
|
## JWT Invalid / Login Fails After Restart
|
|
|
|
Cause: JWT secret changed. Fix: open app in incognito or clear cookies/localStorage.
|
|
|
|
## Port Already In Use
|
|
|
|
Common offenders:
|
|
- `5353/udp` — held by system avahi/mDNS. Remove from Plex ports.
|
|
- `3306`, `5432`, `6379` — only one DB stack can bind these.
|
|
|
|
## Auto-Upgrade Crashes on TrueNAS (exit code 128)
|
|
|
|
Symptom: `error while creating mount source path '/compose/scripts': mkdir /compose: read-only file system`
|
|
|
|
Cause: Auto-upgrade container runs docker compose from inside a container where the project dir is mounted as `/compose`. Relative bind mount paths like `./scripts` resolve to `/compose/scripts` on the host. TrueNAS root filesystem is read-only so Docker can't auto-create the directory.
|
|
|
|
Fix: Pre-create the directory manually on the host before starting the stack:
|
|
```bash
|
|
mkdir -p /mnt/docker-ssd/docker/compose/<stack>/scripts
|
|
```
|
|
|
|
## Databases Stack Network Name Wrong
|
|
|
|
Must have `name: ix-databases` at top of databases compose. Verify:
|
|
```bash
|
|
docker network ls | grep databases
|
|
# Should show: ix-databases_shared-databases
|
|
```
|
|
|
|
## Checking Container Runtime UID
|
|
```bash
|
|
docker run --rm --entrypoint sh <image> -lc 'id'
|
|
```
|