homelab-blueprint/docs/nvidia/proxmox-nvidia-lxc.md

NVIDIA GPU Passthrough: Proxmox VE 9 -> LXC -> Docker

This guide documents the procedure for installing NVIDIA drivers (version 595.80+) on Proxmox VE 9 (Kernel 7.0 / 6.12+) and passing the GPU through to an Emby container running inside Docker within a privileged LXC.

Prerequisites

• Proxmox VE 9.2+ with Kernel 7.0+.
• NVIDIA RTX A400 (or similar Ampere/Lovelace GPU).
• Kernel Headers: Ensure they match your running kernel exactly:

  apt update && apt install proxmox-headers-$(uname -r)
  

• Privileged LXC: The container must have unprivileged: 0 in its configuration.

1. Proxmox Host Preparation

Blacklist Conflicting Drivers

Kernel 7.0 introduces the Rust-based nova driver, which must be blacklisted alongside nouveau.

cat <<EOF > /etc/modprobe.d/blacklist-nvidia.conf
blacklist nouveau
blacklist nvidiafb
blacklist nova
blacklist nova_core
EOF

update-initramfs -u -k all
reboot

Install Build Dependencies

apt update
apt install build-essential dkms proxmox-headers-$(uname -r) pkg-config libglvnd-dev

2. Host Driver Installation (Version 595.80)

Download and Extract

wget https://us.download.nvidia.com/XFree86/Linux-x86_64/595.80/NVIDIA-Linux-x86_64-595.80.run
chmod +x NVIDIA-Linux-x86_64-595.80.run
./NVIDIA-Linux-x86_64-595.80.run --extract-only
cd NVIDIA-Linux-x86_64-595.80

Apply Surgical Patches for Kernel 7.0

# Fix strlcpy -> strscpy (Removed in Kernel 7.0)
find . -type f -name "*.c" -o -name "*.h" | xargs sed -i 's/strlcpy/strscpy/g'

# Fix __assign_str (Changed in Linux 6.10+)
find . -name "nvswitch_event.h" | xargs sed -i -E 's/__assign_str\(([^,)]+),\s*[^)]+\);/__assign_str(\1);/g'

# Fix VMA Locking (Removed VMA_LOCK_OFFSET in 7.0)
sed -i '/VMA_LOCK_OFFSET/d' kernel-open/nvidia/nv-mmap.c

Install

We use the Open Kernel Modules (preferred for Ampere/RTX A400).

./nvidia-installer --dkms --no-questions --ui=none --no-x-check --kernel-module-type=open

Verify: nvidia-smi should show the GPU.

---

3. LXC Configuration (Host Side)

Edit the LXC config file /etc/pve/lxc/<ID>.conf and append the following:

# NVIDIA GPU Passthrough
lxc.cgroup2.devices.allow: c 195:* rwm
lxc.cgroup2.devices.allow: c 236:* rwm
lxc.mount.entry: /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
lxc.mount.entry: /dev/nvidiactl dev/nvidiactl none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm-tools dev/nvidia-uvm-tools none bind,optional,create=file

Ensure the LXC is Privileged and Nesting is enabled.

---

4. LXC Internal Setup

The kernel module is already loaded on the Proxmox host. The LXC only needs the userspace libraries so that nvidia-container-cli can inject them into Docker containers. Do these four steps in order — the Container Toolkit must be configured after the libraries are installed, not before.

4a. Copy the installer into the LXC

The .run file is already on the Proxmox host from Section 2. Push it into the LXC without entering it:

# Run on the Proxmox host (replace 101 with your VMID)
pct push 101 /root/NVIDIA-Linux-x86_64-595.80.run /root/NVIDIA-Linux-x86_64-595.80.run

4b. Install Userspace Libraries

Enter the LXC and run the installer with --no-kernel-module. The kernel module is on the host; only the libraries (libcuda, libnvidia-ml, etc.) go into the LXC.

pct enter 101
chmod +x /root/NVIDIA-Linux-x86_64-595.80.run
/root/NVIDIA-Linux-x86_64-595.80.run --no-kernel-module --no-questions --ui=none

Refresh the linker cache so the libraries are discoverable immediately:

ldconfig
ldconfig -p | grep libnvidia-ml   # must return a path

4c. Install NVIDIA Container Toolkit

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
  gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
  tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

apt update && apt install -y nvidia-container-toolkit
nvidia-ctk runtime configure --runtime=docker

4d. Switch runtime mode to legacy

nvidia-ctk runtime configure sets mode = "auto" by default. In auto mode the toolkit tries to generate CDI specs by calling NVML — this fails inside a privileged LXC with ERROR_LIBRARY_NOT_FOUND even after the userspace libraries are installed, because CDI spec generation happens in a different code path from regular container injection. The legacy mode uses nvidia-container-cli with ldconfig discovery and works reliably in a privileged LXC.

sed -i 's/^mode = "auto"/mode = "legacy"/' /etc/nvidia-container-runtime/config.toml
grep '^mode' /etc/nvidia-container-runtime/config.toml   # should print: mode = "legacy"

Restart Docker:

systemctl restart docker

---

5. Docker Integration (Emby Example)

In your docker-compose.yaml, update the service to use the nvidia runtime:

services:
  emby:
    image: emby/embyserver:latest
    runtime: nvidia
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=compute,video,utility

Verification

Run inside the LXC:

docker run --rm --gpus all nvidia/cuda:12.3.0-base-ubuntu22.04 nvidia-smi