homelab-blueprint/docs/proxmox/storage-layout.md

Proxmox NVMe Storage Layout

This note documents the intended storage layout for the proxmox node after a fresh Proxmox installation, before any VMs or LXCs have been created.

Goal

Use the smaller NVMe for the Proxmox operating system and use the larger NVMe for VM and LXC disks.

Target state:

Device	Role	Proxmox use
Smaller NVMe, about 250 GB	Proxmox OS, boot, root filesystem	local directory storage for ISOs/templates/snippets
Larger NVMe, about 2 TB	VM/LXC workload storage	vm-lxc-disks LVM-thin storage

This keeps the host OS independent from the workload disk. It also makes the Proxmox UI easier to reason about:

• local: ISO images, LXC templates, and snippets on the OS disk.
• vm-lxc-disks: VM disks and LXC root disks on the large NVMe.
• local-lvm: the default small LVM-thin pool from the OS install. Remove it on this fresh host and give its space back to the OS filesystem, because it is on the same small OS disk anyway.

Do not try to collapse the OS disk and the 2 TB workload disk into one storage entry. The useful cleanup is to merge the small default local-lvm space back into the OS filesystem, restrict local to file-style content, then use vm-lxc-disks as the storage selected when creating new VMs and LXCs.

What the Current UI Means

The current visible storage entries are probably:

UI item	Type	Meaning
local (proxmox)	Directory	Files stored under the Proxmox OS filesystem, normally /var/lib/vz.
local-lvm (proxmox)	LVM-thin	VM/LXC disk pool created during the Proxmox install. Its size matching the smaller NVMe means it is probably on the OS disk.
localnetwork (proxmox)	SDN zone, not storage	A Software Defined Network zone using vmbr0. It is not consuming disk space.

localnetwork appears in the left tree because Proxmox shows SDN objects in the same server view. It is not related to local or local-lvm.

The disk view showing the 2 TB NVMe as Usage: LVM means the disk has LVM metadata on it. On a fresh install, that metadata can be replaced if the disk is intended to be dedicated to Proxmox workloads. Proxmox also needs a storage definition in /etc/pve/storage.cfg.

First: Confirm the Fresh Layout

Run these commands on the Proxmox host as root.

lsblk -o NAME,SIZE,TYPE,FSTYPE,MOUNTPOINTS
pvesm status
cat /etc/pve/storage.cfg
pvs -o pv_name,vg_name,pv_size,pv_free
vgs
lvs -a -o vg_name,lv_name,lv_size,pool_lv,data_percent,metadata_percent,devices

Expected findings:

• the 250 GB NVMe contains EFI/boot/root and the current local-lvm;
• the 2 TB NVMe appears as an LVM physical volume or otherwise unused disk.

Because this is immediately after Proxmox installation, these should show no workloads:

qm list
pct list

If either command lists a VM or LXC, stop and treat the host as no longer fresh. Check where those disks live before wiping or removing any storage.

Naming

Use the Proxmox default local storage ID for OS-disk file storage and a purpose-based name for the large workload disk:

Storage ID	Content	Backing disk
local	ISO images, LXC templates, snippets	250 GB OS NVMe
vm-lxc-disks	VM disks, LXC root disks	2 TB NVMe

These names make the creation workflow obvious:

• upload boot ISOs and LXC templates to local;
• create VM disks and LXC root disks on vm-lxc-disks.

Preferred Approach

Use a two-storage layout:

Storage	Backing disk	Purpose
local	250 GB OS NVMe	Proxmox OS filesystem plus ISOs, templates, snippets, and small host-local files
vm-lxc-disks	2 TB NVMe	VM disks and LXC root disks

Why this is the best fit:

• the small OS disk has one visible Proxmox storage entry instead of both local and local-lvm;
• LVM-thin is the native Proxmox default for fast local VM/LXC disks.
• VM disks and LXC root filesystems can live there directly.
• Snapshots work well for normal Proxmox VM/LXC operations.
• The OS disk stays small and recoverable.
• The large disk can be clearly reserved for workloads.

Keep local on the OS disk for ISO images, templates, snippets, and temporary files. Put active VM/LXC disks on vm-lxc-disks.

For the current post-reinstall state where local already exists and the old apps-storage VG still exists on the large NVMe, follow the remaining sections in this order:

1. Restrict local to File Storage.
2. Remove Leftover Large-Disk LVM.
3. Skip Create vm-lxc-disks on a Clean 2 TB Disk, because the leftover cleanup section already recreates vm-lxc-disks.
4. Clean Up the UI.

Merge local-lvm Into the OS Filesystem

On a fresh Proxmox install, local-lvm is usually the data thin pool inside the pve volume group on the OS disk. Since this host has no VMs or LXCs yet, remove that unused thin pool and extend the root filesystem. That makes the small NVMe available to the OS directory storage.

Confirm the root filesystem type:

findmnt -no FSTYPE /

Remove the Proxmox storage definition:

pvesm remove local-lvm

Remove the unused default thin pool and extend the root logical volume:

lvremove -y /dev/pve/data
lvextend -l +100%FREE /dev/pve/root

Resize the filesystem. Use the command that matches the findmnt result:

resize2fs /dev/pve/root

For XFS instead:

xfs_growfs /

Verify that / now has the reclaimed space and local-lvm is gone:

df -h /
pvesm status
lvs

Expected result:

• / is larger than before because the root filesystem has been expanded;
• local-lvm no longer appears in pvesm status;
• the pve/data logical volume no longer appears in lvs.

Restrict local to File Storage

On Proxmox 9, the local storage ID can reappear as a default directory storage even after removing it from /etc/pve/storage.cfg. Keep it and make its purpose clear by restricting its allowed content to ISO images, LXC templates, and snippets.

pvesm set local --content iso,vztmpl,snippets

Verify it:

pvesm status
cat /etc/pve/storage.cfg

Expected result:

• local appears as a directory storage;
• local points to /var/lib/vz;
• local content is limited to iso,vztmpl,snippets.

If an attempted rename left both local and iso-templates, remove the extra iso-templates storage and keep local:

pvesm remove iso-templates
pvesm set local --content iso,vztmpl,snippets
pvesm status

If /etc/pve/storage.cfg contains only iso-templates but pvesm status and pvesh get /storage still synthesize local, restore the intended local definition and remove iso-templates:

pvesm remove iso-templates
pvesm set local --content iso,vztmpl,snippets
pvesm status
cat /etc/pve/storage.cfg

Remove Leftover Large-Disk LVM

If the host was reinstalled but the 2 TB NVMe still has LVM objects from a previous install, pvesm status may show only local while lvs still shows a separate old volume group. For example:

apps-storage  apps-storage twi-aotz-- <1.79t
vm-100-disk-0 apps-storage Vwi---tz-- 4.00m apps-storage
vm-100-disk-1 apps-storage Vwi---tz-- 64.00g apps-storage

That means the OS disk is already merged, but the large NVMe still contains old LVM metadata and old VM disks. Do not try to merge this space into /; it is a separate physical disk and should become vm-lxc-disks.

First confirm there are no current workloads:

qm list
pct list

Then identify the physical device behind the old VG:

pvs -o pv_name,vg_name,pv_size,pv_free
lsblk -o NAME,SIZE,TYPE,FSTYPE,MOUNTPOINTS

Set PV to the exact physical volume shown for the old apps-storage VG. Set DISK to the parent 2 TB NVMe disk that should be wiped and reused. For the current two-NVMe layout, DISK is expected to be /dev/nvme0n1, but confirm it from pvs and lsblk.

PV=/dev/nvme0n1
DISK=/dev/nvme0n1

Remove the old VM volumes and old VG:

lvremove -y /dev/apps-storage/vm-100-disk-0
lvremove -y /dev/apps-storage/vm-100-disk-1
lvremove -y /dev/apps-storage/apps-storage
vgremove -y apps-storage
pvremove -y "$PV"
wipefs -a "$DISK"
sgdisk --zap-all "$DISK"

Recreate the large NVMe under the intended name:

pvcreate "$DISK"
vgcreate vm-lxc-vg "$DISK"
lvcreate -l 95%VG -T vm-lxc-vg/vmdata
pvesm add lvmthin vm-lxc-disks --vgname vm-lxc-vg --thinpool vmdata --content images,rootdir

Verify the result:

pvesm status
lvs -a -o vg_name,lv_name,lv_size,pool_lv,data_percent,metadata_percent,devices
pvs -o pv_name,vg_name,pv_size,pv_free

Expected result:

• local remains on the expanded OS disk;
• apps-storage no longer appears in lvs, vgs, or pvs;
• vm-lxc-disks appears in pvesm status;
• new VM and LXC disks can be created on vm-lxc-disks.

Create vm-lxc-disks on a Clean 2 TB Disk

Use this section only if the 2 TB disk has no old VG to remove. If you already followed Remove Leftover Large-Disk LVM, skip this section because that section already recreates vm-lxc-disks.

The commands below wipe /dev/nvme0n1; confirm that this is the 2 TB disk from lsblk before running them.

DISK=/dev/nvme0n1

wipefs -a "$DISK"
sgdisk --zap-all "$DISK"
pvcreate "$DISK"
vgcreate vm-lxc-vg "$DISK"
lvcreate -l 95%VG -T vm-lxc-vg/vmdata
pvesm add lvmthin vm-lxc-disks --vgname vm-lxc-vg --thinpool vmdata --content images,rootdir

Verify the storage:

pvesm status
lvs -a -o vg_name,lv_name,lv_size,pool_lv,data_percent,metadata_percent,devices

pvesm status should now show vm-lxc-disks with roughly the usable capacity of the 2 TB NVMe.

Clean Up the UI

Because this is a fresh install and there are no workloads yet:

1. Confirm local exists for ISOs, templates, and snippets.
2. Confirm local-lvm is gone.
3. Confirm vm-lxc-disks exists.
4. Use vm-lxc-disks as the storage for every new VM and LXC disk.

Create New Workloads

When creating a new VM:

1. Use local for ISO images.
2. On the VM disk step, choose vm-lxc-disks.
3. After creation, the VM hardware should show disk storage similar to vm-lxc-disks:vm-100-disk-0.

When creating a new LXC:

1. Use local for container templates.
2. On the root disk step, choose vm-lxc-disks.
3. After creation, the LXC resources should show root disk storage similar to vm-lxc-disks:subvol-101-disk-0.

What Not To Do

• Do not wipe /dev/nvme0n1 until lsblk confirms it is the 2 TB disk.
• Do not run the local-lvm merge after creating VMs or LXCs unless you first confirm nothing is using it.
• Do not create new VM/LXC disks on local.
• Do not put ISO images or LXC templates on vm-lxc-disks; LVM-thin is for VM and LXC volumes, not ordinary files.
• Do not worry about localnetwork as a storage item. It is an SDN network zone, not disk storage.

Final Desired Result

The left tree will still show more than one item, but the purpose should be clear:

local           -> OS disk file storage for ISOs, templates, snippets
vm-lxc-disks    -> 2 TB NVMe storage for VM disks and LXC root disks
localnetwork    -> Proxmox SDN network zone, not storage

local-lvm should no longer be shown. New VMs and LXCs should use vm-lxc-disks for their disks. Upload ISOs and templates to local.