381 lines
12 KiB
Markdown
381 lines
12 KiB
Markdown
# Proxmox Storage Audit Report
|
|
|
|
Generated: 2026-02-08
|
|
|
|
---
|
|
|
|
## Executive Summary
|
|
|
|
The Proxmox cluster consists of 3 nodes with a mixture of local and shared NFS storage. Total capacity is **~17 TB**, with significant redundancy across nodes. Current utilization varies widely by node.
|
|
|
|
- **proxmox-00**: High local storage utilization (84.47% root), extensive container deployment
|
|
- **proxmox-01**: Docker-focused, high disk utilization on dlx-docker (81.06%)
|
|
- **proxmox-02**: Lowest utilization, 2 VMs and 1 active container
|
|
|
|
---
|
|
|
|
## Physical Hardware
|
|
|
|
### proxmox-00 (192.168.200.10)
|
|
```
|
|
NAME SIZE TYPE
|
|
loop0 16G loop
|
|
loop1 4G loop
|
|
loop2 100G loop
|
|
loop3 100G loop
|
|
loop4 16G loop
|
|
loop5 100G loop
|
|
loop6 32G loop
|
|
loop7 100G loop
|
|
loop8 100G loop
|
|
sda 1.8T disk → /mnt/pve/dlx-sda (1.8TB dir)
|
|
sdb 1.8T disk → NFS mount (nfs-sdd)
|
|
sdc 1.8T disk → NFS mount (nfs-sdc)
|
|
sdd 1.8T disk → NFS mount (nfs-sde)
|
|
sde 1.8T disk → /mnt/dlx-nfs-sde (1.8TB NFS)
|
|
sdf 931.5G disk → dlx-sdf4 (785GB LVM)
|
|
sdg 0B disk → (unused/not configured)
|
|
sr0 1024M rom → (CD-ROM)
|
|
```
|
|
|
|
### proxmox-01 (192.168.200.11)
|
|
```
|
|
NAME SIZE TYPE
|
|
loop0 400G loop
|
|
loop1 400G loop
|
|
loop2 100G loop
|
|
sda 953.9G disk → /mnt/pve/dlx-docker (718GB dir, 81% full)
|
|
sdb 680.6G disk → (appears unused, no mount)
|
|
```
|
|
|
|
### proxmox-02 (192.168.200.12)
|
|
```
|
|
NAME SIZE TYPE
|
|
loop0 32G loop
|
|
sda 3.6T disk → NFS mount (nfs-sdb-02)
|
|
sdb 3.6T disk → /mnt/dlx-nfs-sdb-02 (3.6TB NFS)
|
|
nvme0n1 931.5G disk → /mnt/pve/dlx-data (670GB dir, 10% full)
|
|
```
|
|
|
|
---
|
|
|
|
## Storage Backend Configuration
|
|
|
|
### Shared NFS Storage (Accessible from all nodes)
|
|
|
|
| Storage | Type | Total | Used | Available | % Used | Content | Shared |
|
|
|---------|------|-------|------|-----------|--------|---------|--------|
|
|
| **dlx-nfs-sdb-02** | NFS | 3.9 TB | 2.9 GB | 3.7 TB | **0.07%** | images, rootdir, backup | ✓ |
|
|
| **dlx-nfs-sdc-00** | NFS | 1.9 TB | 139 GB | 1.7 TB | **7.47%** | images, rootdir | ✓ |
|
|
| **dlx-nfs-sdd-00** | NFS | 1.9 TB | 12 GB | 1.8 TB | **0.63%** | iso, vztmpl, rootdir, snippets, backup, images, import | ✓ |
|
|
| **dlx-nfs-sde-00** | NFS | 1.9 TB | 54 GB | 1.7 TB | **2.83%** | iso, vztmpl, rootdir, snippets, backup, images, import | ✓ |
|
|
| **TOTAL NFS** | - | **~9.7 TB** | **~209 GB** | **~8.7 TB** | **~2.2%** | - | ✓ |
|
|
|
|
---
|
|
|
|
### Local Storage by Node
|
|
|
|
#### proxmox-00 Storage
|
|
| Storage | Type | Status | Total | Used | Available | % Used | Notes |
|
|
|---------|------|--------|-------|------|-----------|--------|-------|
|
|
| **dlx-sda** | dir | ✓ active | 1.9 TB | 61 GB | 1.8 TB | **3.3%** | Local dir storage |
|
|
| **dlx-sdb** | zfspool | ✓ active | 1.9 TB | 4.2 GB | 1.9 TB | **0.2%** | ZFS pool |
|
|
| **dlx-sdf4** | lvm | ✓ active | 785 GB | 157 GB | 610 GB | **20.5%** | LVM thin pool |
|
|
| **local** | dir | ✓ active | 62 GB | 52 GB | 6.3 GB | **84.5%** | **⚠️ CRITICAL: 90% full on root FS** |
|
|
| **local-lvm** | lvmthin | ✓ active | 116 GB | 0 GB | 116 GB | **0%** | Thin provisioning pool |
|
|
|
|
#### proxmox-01 Storage
|
|
| Storage | Type | Status | Total | Used | Available | % Used | Notes |
|
|
|---------|------|--------|-------|------|-----------|--------|-------|
|
|
| **dlx-docker** | dir | ✓ active | 718 GB | 568 GB | 97 GB | **81.1%** | **⚠️ HIGH: Docker container storage** |
|
|
| **local** | dir | ✓ active | 62 GB | 42 GB | 15 GB | **69.5%** | Template storage |
|
|
| **local-lvm** | lvmthin | ✓ active | 116 GB | 0 GB | 116 GB | **0%** | Thin provisioning pool |
|
|
|
|
#### proxmox-02 Storage
|
|
| Storage | Type | Status | Total | Used | Available | % Used | Notes |
|
|
|---------|------|--------|-------|------|-----------|--------|-------|
|
|
| **dlx-data** | dir | ✓ active | 702 GB | 63 GB | 602 GB | **9.1%** | NVME-backed (fast) |
|
|
| **local** | dir | ✓ active | 92 GB | 43 GB | 44 GB | **47.2%** | Template/OS storage |
|
|
| **local-lvm** | lvmthin | ✓ active | 160 GB | 0 GB | 160 GB | **0%** | Thin provisioning pool |
|
|
|
|
### Disabled Storage (not currently in use)
|
|
|
|
| Storage | Type | Node | Reason |
|
|
|---------|------|------|--------|
|
|
| **dlx-docker** | dir | proxmox-00, proxmox-02 | Disabled on these nodes |
|
|
| **dlx-data** | dir | proxmox-00, proxmox-01 | Disabled on these nodes |
|
|
| **dlx-sda** | dir | proxmox-01 | Disabled |
|
|
| **dlx-sdb** | zfspool | proxmox-01, proxmox-02 | Disabled on these nodes |
|
|
| **dlx-sdf4** | lvm | proxmox-01, proxmox-02 | Disabled on these nodes |
|
|
|
|
---
|
|
|
|
## Container & VM Allocation
|
|
|
|
### proxmox-00: Infrastructure Hub (16 LXC Containers, 0 VMs)
|
|
**All Running**:
|
|
1. **dlx-postgres** (103) - PostgreSQL database
|
|
- Allocated: 100 GB | Used: 2.8 GB | Mem: 16 GB
|
|
|
|
2. **dlx-gitea** (102) - Git hosting
|
|
- Allocated: 100 GB | Used: 5.7 GB | Mem: 8 GB
|
|
|
|
3. **dlx-hiveops** (112) - Application
|
|
- Allocated: 100 GB | Used: 3.7 GB | Mem: 4 GB
|
|
|
|
4. **dlx-kafka** (113) - Message broker
|
|
- Allocated: 31 GB | Used: 2.2 GB | Mem: 4 GB
|
|
|
|
5. **dlx-redis-01** (115) - Cache
|
|
- Allocated: 100 GB | Used: 81 GB | Mem: 8 GB
|
|
|
|
6. **dlx-ansible** (106) - Ansible control
|
|
- Allocated: 16 GB | Used: 3.7 GB | Mem: 4 GB
|
|
|
|
7. **dlx-pihole** (100) - DNS/Ad-block
|
|
- Allocated: 16 GB | Used: 2.6 GB | Mem: 4 GB
|
|
|
|
8. **dlx-npm** (101) - Nginx Proxy Manager
|
|
- Allocated: 4 GB | Used: 2.4 GB | Mem: 4 GB
|
|
|
|
9. **dlx-mongo-01** (111) - MongoDB
|
|
- Allocated: 100 GB | Used: 7.6 GB | Mem: 8 GB
|
|
|
|
10. **dlx-smartjournal** (114) - Journal Application
|
|
- Allocated: 157 GB | Used: 54 GB | Mem: 33 GB
|
|
|
|
**Stopped** (5):
|
|
- dlx-wireguard (105) - 32 GB allocated
|
|
- dlx-mysql-02 (108) - 200 GB allocated
|
|
- dlx-mattermost (107) - 32 GB allocated
|
|
- dlx-mysql-03 (109) - 200 GB allocated
|
|
- dlx-nocodb (116) - 100 GB allocated
|
|
|
|
**Total Allocation**: 1.8 TB | **Running Utilization**: ~172 GB
|
|
|
|
---
|
|
|
|
### proxmox-01: Docker & Services (5 LXC Containers, 0 VMs)
|
|
**All Running**:
|
|
1. **dlx-docker** (200) - Docker host
|
|
- Allocated: 421 GB | Used: 36 GB | Mem: 16 GB
|
|
|
|
2. **dlx-sonar** (202) - SonarQube analysis
|
|
- Allocated: 422 GB | Used: 354 GB | Mem: 16 GB ⚠️ **HEAVY DISK USER**
|
|
|
|
3. **dlx-odoo** (201) - ERP system
|
|
- Allocated: 100 GB | Used: 3.7 GB | Mem: 16 GB
|
|
|
|
**Stopped** (10):
|
|
- dlx-swarm-01/02/03 (210, 211, 212) - 65 GB each
|
|
- dlx-snipeit (203) - 50 GB
|
|
- dlx-fleet (206) - 60 GB
|
|
- dlx-coolify (207) - 50 GB
|
|
- dlx-kube-01/02/03 (215-217) - 50 GB each
|
|
- dlx-www (204) - 32 GB
|
|
- dlx-svn (205) - 100 GB
|
|
|
|
**Total Allocation**: 1.7 TB | **Running Utilization**: ~393 GB
|
|
|
|
---
|
|
|
|
### proxmox-02: Development & Testing (2 VMs, 1 LXC Container)
|
|
**Running**:
|
|
1. **dlx-www** (303, LXC) - Web services
|
|
- Allocated: 31 GB | Used: 3.2 GB | Mem: 2 GB
|
|
|
|
**Stopped** (2 VMs):
|
|
1. **dlx-atm-01** (305) - ATM application VM
|
|
- Allocated: 8 GB (max disk 0)
|
|
|
|
2. **dlx-development** (306) - Dev environment VM
|
|
- Allocated: 160 GB | Mem: 16 GB
|
|
|
|
**Total Allocation**: 199 GB | **Running Utilization**: ~3.2 GB
|
|
|
|
---
|
|
|
|
## Storage Mapping & Usage Patterns
|
|
|
|
### Shared NFS Mounts
|
|
|
|
```
|
|
All Nodes can access:
|
|
├── dlx-nfs-sdb-02 → Backup/images (3.9 TB) - 0.07% used
|
|
├── dlx-nfs-sdc-00 → Images/rootdir (1.9 TB) - 7.47% used
|
|
├── dlx-nfs-sdd-00 → Templates/ISO/backup (1.9 TB) - 0.63% used
|
|
└── dlx-nfs-sde-00 → Templates/ISO/images (1.9 TB) - 2.83% used
|
|
```
|
|
|
|
### Node-Specific Storage
|
|
|
|
```
|
|
proxmox-00 (Control Hub):
|
|
├── local (62 GB) ⚠️ CRITICAL: 84.5% FULL
|
|
├── dlx-sda (1.9 TB) - 3.3% used
|
|
├── dlx-sdb ZFS (1.9 TB) - 0.2% used
|
|
├── dlx-sdf4 LVM (785 GB) - 20.5% used
|
|
└── local-lvm (116 GB) - 0% used
|
|
|
|
proxmox-01 (Docker/Services):
|
|
├── local (62 GB) - 69.5% used
|
|
├── dlx-docker (718 GB) ⚠️ HIGH: 81.1% USED
|
|
└── local-lvm (116 GB) - 0% used
|
|
|
|
proxmox-02 (Development):
|
|
├── local (92 GB) - 47.2% used
|
|
├── dlx-data (702 GB) - 9.1% used (NVME, fast)
|
|
└── local-lvm (160 GB) - 0% used
|
|
```
|
|
|
|
---
|
|
|
|
## Capacity & Utilization Summary
|
|
|
|
| Metric | Value | Status |
|
|
|--------|-------|--------|
|
|
| **Total Capacity** | ~17 TB | ✓ Adequate |
|
|
| **Total Used** | ~1.3 TB | ✓ 7.6% |
|
|
| **Total Available** | ~15.7 TB | ✓ Healthy |
|
|
| **Shared NFS** | 9.7 TB (2.2% used) | ✓ Excellent |
|
|
| **Local Storage** | 7.3 TB (18.3% used) | ⚠️ Mixed |
|
|
|
|
---
|
|
|
|
## Critical Issues & Recommendations
|
|
|
|
### 🔴 CRITICAL: proxmox-00 Root Filesystem
|
|
|
|
**Issue**: `/` (root) is 84.5% full (52.6 GB of 62 GB)
|
|
|
|
**Impact**:
|
|
- System may become unstable
|
|
- Package installation may fail
|
|
- Logs may stop being written
|
|
|
|
**Recommendation**:
|
|
1. Clean up old logs: `journalctl --vacuum=time:30d`
|
|
2. Check for old snapshots/backups
|
|
3. Consider moving `/var` to separate storage
|
|
4. Monitor closely for growth
|
|
|
|
---
|
|
|
|
### 🟠 HIGH PRIORITY: proxmox-01 dlx-docker
|
|
|
|
**Issue**: dlx-docker storage at 81.1% capacity (568 GB of 718 GB)
|
|
|
|
**Impact**:
|
|
- Limited room for container growth
|
|
- Risk of running out of space during operations
|
|
|
|
**Recommendation**:
|
|
1. Audit running containers: `docker ps -a --format "{{.Names}}: {{json .SizeRw}}"`
|
|
2. Remove unused images/layers
|
|
3. Consider expanding partition or migrating data
|
|
4. Set up monitoring for capacity
|
|
|
|
---
|
|
|
|
### 🟠 HIGH PRIORITY: proxmox-01 dlx-sonar
|
|
|
|
**Issue**: SonarQube using 354 GB (82% of allocated 422 GB)
|
|
|
|
**Impact**:
|
|
- Large analysis database
|
|
- May need separate storage strategy
|
|
|
|
**Recommendation**:
|
|
1. Review SonarQube retention policies
|
|
2. Archive old analysis data
|
|
3. Consider separate backup strategy
|
|
|
|
---
|
|
|
|
### ⚠️ Medium Priority: Storage Inconsistency
|
|
|
|
**Issue**: Disabled storage backends across nodes
|
|
|
|
| Backend | disabled on | Notes |
|
|
|---------|-------------|-------|
|
|
| dlx-docker | proxmox-00, 02 | Only enabled on 01 |
|
|
| dlx-data | proxmox-00, 01 | Only enabled on 02 |
|
|
| dlx-sda | proxmox-01 | Enabled on 00 only |
|
|
| dlx-sdb (ZFS) | proxmox-01, 02 | Only enabled on 00 |
|
|
| dlx-sdf4 (LVM) | proxmox-01, 02 | Only enabled on 00 |
|
|
|
|
**Recommendation**:
|
|
1. Document why each backend is disabled per node
|
|
2. Standardize storage configuration across cluster
|
|
3. Consider cluster-wide storage policy
|
|
|
|
---
|
|
|
|
### ⚠️ Medium Priority: Container Lifecycle
|
|
|
|
**Issue**: 15 containers are stopped but still allocating space (1.2 TB total)
|
|
|
|
**Recommendation**:
|
|
1. Audit stopped containers (dlx-swarm-*, dlx-kube-*, etc.)
|
|
2. Delete unused containers to reclaim space
|
|
3. Document intended purpose of stopped containers
|
|
|
|
---
|
|
|
|
## Recommendations Summary
|
|
|
|
### Immediate (Next week)
|
|
1. ✅ Compress logs on proxmox-00 root filesystem
|
|
2. ✅ Audit dlx-docker usage and remove unused images
|
|
3. ✅ Monitor proxmox-01 dlx-docker capacity
|
|
|
|
### Short-term (1-2 months)
|
|
1. Expand dlx-docker partition or migrate high-usage containers
|
|
2. Archive SonarQube data or increase disk allocation
|
|
3. Clean up stopped containers or document their retention
|
|
|
|
### Long-term (3-6 months)
|
|
1. Implement automated capacity monitoring
|
|
2. Standardize storage backend configuration across cluster
|
|
3. Establish storage lifecycle policies (snapshots, backups, retention)
|
|
4. Consider tiered storage strategy (fast NVME vs. slow SATA)
|
|
|
|
---
|
|
|
|
## Storage Performance Tiers
|
|
|
|
Based on hardware analysis:
|
|
|
|
| Tier | Storage | Speed | Use Case |
|
|
|------|---------|-------|----------|
|
|
| **Tier 1 (Fast)** | nvme0n1 (proxmox-02) | NVMe | OS, critical services |
|
|
| **Tier 2 (Medium)** | ZFS/LVM pools | HDD/SSD | VMs, container data |
|
|
| **Tier 3 (Shared)** | NFS mounts | Network | Backups, shared data |
|
|
| **Tier 4 (Archive)** | Large local dirs | HDD | Infrequently accessed |
|
|
|
|
**Optimization Opportunity**: Align hot data to Tier 1, cold data to Tier 3
|
|
|
|
---
|
|
|
|
## Appendix: Raw Storage Stats
|
|
|
|
### Storage IDs & Content Types
|
|
- **images** - VM/container disk images
|
|
- **rootdir** - Root filesystem for LXCs
|
|
- **backup** - Backup snapshots
|
|
- **iso** - ISO images
|
|
- **vztmpl** - Container templates
|
|
- **snippets** - Config snippets
|
|
- **import** - Import data
|
|
|
|
### Size Conversions
|
|
- 1 TB = ~1,099 GB
|
|
- 1 GB = ~1,074 MB
|
|
- All sizes in binary (not decimal)
|
|
|
|
---
|
|
|
|
**Report Generated**: 2026-02-08 via Ansible
|
|
**Data Source**: `pvesm status` and `pvesh` API
|
|
**Next Audit Recommended**: 2026-03-08
|