Measuring Peak Memory of a PDAL Pipeline

TL;DR: For a command-line run, /usr/bin/time -v pdal pipeline p.json reports “Maximum resident set size”. For a pipeline run in a subprocess from Python, read resource.getrusage(resource.RUSAGE_CHILDREN).ru_maxrss afterwards. For in-process pdal.Pipeline.execute(), sample RSS with psutil in a background thread. tracemalloc will not help — PDAL’s allocations happen in C++.

# Context and Motivation

This guide is part of Memory Management in PDAL. An estimate tells you what memory should be; a measurement tells you what it is. You need the measurement to size batch workers, to confirm that a change such as streaming or dropping a dimension actually helped, and to find which stage sets the peak. Python’s own memory tools are the wrong instrument here: PDAL allocates its point tables and spatial indexes in native code, invisible to tracemalloc and to Python object counting. What matters is the operating system’s view — the process’s resident set size, and specifically its maximum over the run.

# Prerequisites and Assumptions

• Linux, where /usr/bin/time -v and ru_maxrss in kilobytes behave as described. On macOS ru_maxrss is in bytes and time -l replaces time -v.
• Python 3.10+ with psutil for sampling.
• A representative tile — preferably the largest in the batch, since that is what the worker must fit.

# Step-by-Step Implementation

# Step 1 — Measure a CLI run

GNU time reports the maximum RSS of the process and its children:

/usr/bin/time -v pdal pipeline dtm.json 2> time.log
grep "Maximum resident set size" time.log

# Step 2 — Measure a subprocess from Python

Run PDAL as a child process and read the children’s maximum RSS after it exits. This is exact and costs nothing during the run.

# Step 3 — Measure an in-process run

When the pipeline runs inside your Python process, sample psutil.Process().memory_info().rss every 50–100 ms in a thread and keep the maximum. Subtract the baseline taken before execution.

# Step 4 — Attribute the peak to a stage

Run cumulative prefixes of the pipeline — reader only, reader plus first filter, and so on — and measure each. The stage whose addition causes the largest jump is the one to optimize.

# Step 5 — Record alongside the tile’s point count

Store peak memory with the point count, so you can fit bytes per point and predict memory for any tile.

# Complete Working Example

"""Peak memory of a PDAL pipeline: subprocess, in-process sampling, per-stage prefixes."""
from __future__ import annotations

import json
import resource
import subprocess
import tempfile
import threading
import time
from pathlib import Path

import pdal
import psutil


def peak_subprocess(spec: dict) -> float:
    """Peak RSS in GB of `pdal pipeline` run as a child process (Linux: ru_maxrss in KB)."""
    with tempfile.NamedTemporaryFile("w", suffix=".json", delete=False) as f:
        json.dump(spec, f)
    before = resource.getrusage(resource.RUSAGE_CHILDREN).ru_maxrss
    subprocess.run(["pdal", "pipeline", f.name], check=True)
    after = resource.getrusage(resource.RUSAGE_CHILDREN).ru_maxrss
    Path(f.name).unlink()
    return max(after, before) / 1e6


def peak_in_process(spec: dict, interval: float = 0.05) -> float:
    proc = psutil.Process()
    base = proc.memory_info().rss
    peak = base
    done = threading.Event()

    def sample() -> None:
        nonlocal peak
        while not done.is_set():
            peak = max(peak, proc.memory_info().rss)
            time.sleep(interval)

    t = threading.Thread(target=sample, daemon=True)
    t.start()
    try:
        pdal.Pipeline(json.dumps(spec)).execute()
    finally:
        done.set()
        t.join()
    return (peak - base) / 1e9


def per_stage(spec: dict) -> list[tuple[str, float]]:
    stages = spec["pipeline"]
    results = []
    for i in range(1, len(stages) + 1):
        prefix = [s for s in stages[:i] if not str(s.get("type", "")).startswith("writers.")]
        if not prefix:
            continue
        gb = peak_subprocess({"pipeline": prefix})
        results.append((stages[i - 1].get("type", "reader"), gb))
    return results


if __name__ == "__main__":
    spec = json.loads(Path("dtm.json").read_text())
    print(f"subprocess peak: {peak_subprocess(spec):.2f} GB")
    for stage, gb in per_stage(spec):
        print(f"  up to {stage:<24} {gb:6.2f} GB")

Note that RUSAGE_CHILDREN reports the maximum over all children so far, which is why per_stage works best when each prefix runs in a fresh Python process or when prefixes are run in increasing order of expected memory. For clean per-stage numbers, wrap each call in its own short-lived Python process.

# Key Parameter Table

# Verification

• Two methods agree. For one tile, the subprocess and in-process measurements should be within a few percent once the in-process baseline is subtracted.
• Linear in points. Measure three tiles of different sizes; peak memory against point count should be close to a straight line. A curve upward points to a stage with super-linear memory, usually a raster writer at fine resolution.
• Matches the estimate. Compare with the prediction from estimating PDAL memory from point layout.

# Gotchas and Edge Cases

Sampling misses short spikes. A 100 ms interval can miss a spike that lasts 50 ms. Use the subprocess method when you need the true maximum; use sampling for a memory-over-time picture.

Freed memory is not always returned. The allocator may keep freed pages mapped, so RSS after a stage can stay high even though PDAL released the memory. The peak is still correct; the “after” value is not a measure of what is in use.

Containers report differently. In Docker, the container’s cgroup counts page cache as well. A container can be killed for exceeding its limit even when the process RSS looks fine, because reading a large LAZ fills the cache. Check memory.peak (cgroup v2) inside the container.

Threads and OMP. Stages that use OpenMP allocate per-thread buffers. Peak memory can rise with OMP_NUM_THREADS, so measure with the thread setting you will run in production.

# Frequently Asked Questions

Why does tracemalloc show almost no memory for a PDAL run?

Because PDAL allocates its point tables and indexes in C++, outside Python’s allocator. tracemalloc only tracks Python allocations. Measure the process’s resident set size instead.

What is the simplest way to get the peak memory of a PDAL command?

Run it under GNU time with the verbose flag and read the maximum resident set size line. It needs no code and has no overhead.

How do I find which stage uses the most memory?

Measure peaks for cumulative prefixes of the pipeline: reader alone, reader plus the first filter, and so on. The largest increase between consecutive prefixes identifies the stage.

Why was my container killed when the process used less than the limit?

Container memory accounting includes page cache from file reads. Large inputs fill the cache, pushing the cgroup total past the limit. Check the cgroup’s peak value and give the container headroom above the process peak.

# Related

• Memory Management in PDAL — how PDAL allocates
• Estimating PDAL Memory from Point Layout — the prediction to check
• Diagnosing PDAL Out-of-Memory Failures — when measurement comes too late
• Running a PDAL Pipeline in Streaming Mode — the biggest lever on peak memory
• Building a Slim PDAL Docker Image — containers where the cgroup limit applies