Reordering PDAL Stages for Speed

TL;DR: Move every stage that removes points as early as correctness allows, and every stage that adds a dimension as late as possible — then measure, because the ordering that is fastest on a dense urban tile is often not the one that is fastest on sparse rural coverage.

# Context and Motivation

This guide is part of PDAL Stage Chaining, which covers how stages hand buffers to one another. Here the question is narrower and entirely practical: given a set of stages that must all run, in which order should they run?

The naive answer — the order in which you thought of them — is usually two to five times slower than the best one, and the reason is arithmetic rather than anything subtle. Every stage costs roughly its per-point cost multiplied by the number of points reaching it. A stage that removes eighty percent of the cloud therefore makes every subsequent stage five times cheaper, and it does so whether it runs first or last. Put it first and the saving applies to everything; put it last and it applies to nothing. The complication is that not every reordering preserves the result, and the stages that reduce the most are often the ones with dependencies.

# Prerequisites and Assumptions

# Step-by-Step Implementation

# Step 1 — Write down each stage’s selectivity and cost

Run the chain once with logging and record two numbers per stage: seconds spent, and points removed. pdal pipeline --verbose 4 reports timings; point counts come from a filters.stats inserted temporarily, or from the difference in output counts when you truncate the chain.

# Step 2 — Sort by cheap-and-selective first

Rank the stages by removed-points per second and put the highest first. Elevation limits and crops almost always win this ranking; classification and neighbourhood filters almost always lose it.

# Step 3 — Apply the dependency constraints

Three hard rules survive any reordering:

• A stage that reads a dimension must run after the stage that creates it. filters.range on HeightAboveGround cannot precede filters.hag_nn.
• A stage whose parameters are in CRS units must run after the reprojection that establishes those units — or before it, consistently, but never with the units of the other side.
• A stage that needs spatial context must not run after something that removed that context. A crop without buffer ahead of a ground classifier is the classic mistake.

# Step 4 — Re-measure, and keep the measurement

Ordering choices decay: a new stage gets added, the input density changes, and the order that was optimal is not. Record the timing in the repository next to the pipeline so the next person can see why the order is what it is.

# Complete Working Example

"""Time a PDAL pipeline under several stage orderings and report the best."""
from __future__ import annotations

import itertools
import json
import time
from typing import Any

import pdal

READER = {"type": "readers.las", "filename": "tile_0431.laz"}
WRITER = {"type": "writers.las", "filename": "/tmp/out.laz", "compression": "laszip"}

MOVABLE: list[dict[str, Any]] = [
    {"type": "filters.range", "limits": "Z[-30:5000]"},
    {"type": "filters.crop", "polygon": "POLYGON((512000 4783000, 513000 4783000, "
                                        "513000 4784000, 512000 4784000, 512000 4783000))"},
    {"type": "filters.outlier", "method": "statistical", "mean_k": 12, "multiplier": 2.5},
]

# Reprojection must stay after the crop, whose polygon is in source CRS units.
FIXED_LAST = {"type": "filters.reprojection", "out_srs": "EPSG:6318"}


def timed(stages: list[dict[str, Any]]) -> tuple[float, int]:
    spec = json.dumps({"pipeline": [READER, *stages, FIXED_LAST, WRITER]})
    started = time.perf_counter()
    n = pdal.Pipeline(spec).execute()
    return time.perf_counter() - started, n


def main() -> None:
    results = []
    for order in itertools.permutations(MOVABLE):
        seconds, kept = timed(list(order))
        names = " → ".join(s["type"].split(".", 1)[1] for s in order)
        results.append((seconds, kept, names))
        print(f"{seconds:7.2f}s  {kept:>10,}  {names}")

    results.sort()
    best, kept, names = results[0]
    worst = results[-1][0]
    print(f"\nbest: {names} at {best:.2f}s — {worst / best:.1f}x faster than the worst order")
    counts = {r[1] for r in results}
    assert len(counts) == 1, f"orderings disagree on the result: {counts}"


if __name__ == "__main__":
    main()

The assertion at the end matters as much as the timing: if two orderings produce different point counts, one of them is wrong and the fast one is not automatically the right one.

# Key Parameter Table

# Verification

Every ordering yields the same output. That is the assertion in the example. Run it on at least one tile with unusual content — a tile that is mostly water, or one that straddles a crop boundary.

The speedup is real, not noise. Time each ordering three times and compare medians. A twenty percent difference on a single run is usually the page cache, not the pipeline.

Metadata still matches. Reordering can change which stage last touched the header. Compare pdal info --metadata output between orderings; the CRS and bounding box should be identical.

# Gotchas and Edge Cases

A faster order that changes the answer is not faster. The most common instance is moving filters.sample earlier: thinning the cloud before classification changes which points are local minima, and the ground surface moves. The point counts will differ and the assertion will catch it — if you wrote the assertion.

Selectivity is data-dependent. A crop that removes 95% of an urban tile may remove 5% of the rural tile next to it. If one order must serve a whole campaign, tune it on the least favourable tile rather than the most.

Reordering interacts with streaming. Some orders are streamable and others are not, and a chain that streams is often worth more than a chain that is nominally faster — see streaming mode execution.

# Frequently Asked Questions

How much can reordering actually save?

On a chain containing one selective stage and one expensive stage, two to five times is typical. The saving is simply the expensive stage running on the reduced cloud instead of the full one, so it scales with how selective the reducing stage is on your data.

Which stage should almost always run first?

An elevation range filter. It is nearly free, it needs nothing from any other stage, and on real acquisitions it removes the blunders that would otherwise distort every later statistic. A crop usually comes next, buffered if a classifier follows.

Can reordering change the output?

Yes, and that is why the timing script asserts the point counts agree. The usual culprits are moving a thinning filter ahead of a classifier, or cropping without a buffer ahead of a stage that needs neighbours at the tile edge.

Does the best order depend on the data?

Strongly. A crop that removes ninety-five percent of an urban tile may remove five percent of a rural one, which changes the ranking entirely. When a single order has to serve a whole campaign, tune it against the least favourable tile.

# Related

• PDAL Stage Chaining — the parent guide to how stages pass buffers
• Chaining PDAL Stages for Data Cleaning — the cleaning chain this ordering advice is usually applied to
• Pipeline Filtering Logic — what each filter costs and how selective it is
• Streaming Mode Execution in PDAL — why a streamable order can beat a nominally faster one
• PDAL Pipeline Architecture and Execution — the section overview