Skip to content
PVE Storage Guard

External trace research

Public storage traces are valuable for workload-shape and policy stress tests, but the sources reviewed so far do not contain synchronized evidence of SSH, PVE API, or an equivalent host-management plane. Storage-only data therefore cannot close the active-control promotion gate.

Propose a source through the metadata-only trace qualification process. Do not attach or link raw production data.

Sources reviewed

Section titled “Sources reviewed”
Source Latency Management evidence Use here
UMass/SPC traces no portable standard field none one derived search prefix accepted for workload-shape research only
Alibaba Block Traces none none one Ultra Disk prefix accepted for workload shape and logical storage-class research
Tencent and CloudPhysics catalog not in listed block formats none candidate workload shape
MSR Cambridge catalog per-I/O response time none held for license review
Google Thesios yes none synthetic tests only

The CacheMon collection labels its work CC BY 4.0, while a read-only inspection of the linked MSR archive found an older Microsoft disclaimer with materially more restrictive redistribution terms. The project bundles neither that archive nor derived MSR samples. Permission must be established independently.

Safe research lane

Section titled “Safe research lane”

The repository contains two attributed CC BY 4.0 derivatives, each covering a 600-second prefix in 60 ten-second buckets. The UMass/SPC search artifact drops ASU, LBA, optional fields, and source timestamps; its storage class remains unknown. The Alibaba artifact drops device IDs, offsets, and absolute arrival times while preserving only the documented logical network-block Ultra Disk class. It does not infer physical HDD, SSD, or NVMe media. Neither raw source is bundled.

Both pass the independent workload-shape research gate. Alibaba also passes a separate storage-class research gate. Both always return active_control_eligible=false because response latency and synchronized management-plane evidence are unavailable.

The repository provides a standard-library-only converter for caller-supplied, authorized CSV data. It accepts timestamp, operation, size, and response time; drops every unspecified source column; and emits relative offsets, nearest-rank write p95, IOPS, and throughput. Timestamp-ordered input is aggregated one interval at a time rather than retained as a full per-I/O copy. The caller must explicitly declare whether latency was measured at the storage domain, block device, virtual disk, application, or an unknown layer.

Terminal window
python3 poc/io_csv_to_replay_trace.py authorized.csv \
  --name licensed-study-a \
  --source-kind observed \
  --independence-group external-study-a \
  --storage-class rotational-hdd \
  --workload-class mixed \
  --write-wait-measurement-layer block-device \
  --confirm-authorized-and-sanitized > candidate.json

Because block-I/O rows do not prove host availability, every converted sample has managementPlaneStatus=unknown. The trace can exercise storage algorithms, but the machine assessor must reject it as production promotion evidence. A block-device p95 also cannot be relabeled as the controller’s storage-domain p95. The confirmation flag is an operator assertion, not an automated license check.

Read the complete review and input contract on GitHub.