Mapping Custom Attributes in PDAL Pipelines

TL;DR: Declare each custom dimension in readers.las extra_dims, compute its value with filters.assign, then mirror the same name and type in writers.las extra_dims — omitting either declaration causes silent data loss.

# Context and Motivation

This guide is part of the Attribute Mapping cluster, which sits inside the broader PDAL Pipeline Architecture & Execution documentation. The specific problem this page solves is a common source of invisible data loss: PDAL does not auto-persist arbitrary point attributes. You can compute a dimension inside a filter and see it in pipeline.arrays[0] only to find it absent in the output file — because the writer was never told the dimension existed.

Production LiDAR workflows routinely need dimensions beyond the LAS standard: normalized intensity scores for vegetation classifiers, per-point survey confidence flags, height-above-ground proxies derived from return ratios, or machine-learning feature vectors computed on the fly. Getting these dimensions from expression to disk reliably requires understanding exactly where PDAL allocates buffer slots and which stages must declare an explicit type contract.

# Prerequisites and Assumptions

Before implementing custom attribute mapping, verify:

• PDAL 2.5 or later — array-style value in filters.assign (multiple expressions in one stage) requires 2.5+; verify with pdal --version
• Python 3.10+ with the pdal package installed (pip install pdal or conda install -c conda-forge python-pdal)
• Input file in LAS 1.2–1.4 or LAZ format; E57 and PLY require an additional reader stage and may not support Extra Bytes on output
• Standard dimensions present: X, Y, Z, Intensity (uint16), ReturnNumber, NumberOfReturns, and Classification — confirm with pdal info --schema input.laz before writing expressions
• LAS 1.4 output target when custom dimensions must round-trip to the file; Extra Bytes in LAS 1.4 with point format 6 have the widest reader support across downstream tools
• Write permission on the output directory; PDAL will not create directories automatically

# Step-by-Step Implementation

# Step 1 — Audit the input schema

Run a schema inspection before writing any expressions. Dimension names are case-sensitive in PDAL; using returnNumber in an expression when the actual name is ReturnNumber produces a silent zero rather than an error:

pdal info --schema input_cloud.laz

From Python:

import json
import pdal

pipeline = pdal.Pipeline(json.dumps([{"type": "readers.las", "filename": "input_cloud.laz"}]))
pipeline.execute()
arr = pipeline.arrays[0]
print(dict(zip(arr.dtype.names, [str(arr.dtype[n]) for n in arr.dtype.names])))

Record the exact names and types. This drives the expression strings and type choices in steps 2 and 3.

# Step 2 — Choose the right filter for each dimension

Three filters cover the full range of custom attribute derivation. Choose based on the complexity of the rule:

Reserve filters.python for operations that cannot be expressed in native PDAL arithmetic. The Python interpreter adds 10–50× overhead over filters.assign on large point clouds. For pipeline filtering logic that combines dimension assignment with point removal, use filters.expression with an explicit where clause rather than a separate filters.range stage — this avoids two buffer passes.

# Step 3 — Declare the dimension in the reader

Add extra_dims to readers.las for every non-standard dimension the pipeline will produce. PDAL must allocate a buffer slot before any filter can write to it:

{
  "type": "readers.las",
  "filename": "input_cloud.laz",
  "extra_dims": "norm_intensity=float,survey_confidence=uint8"
}

For inbound files that already carry custom Extra Bytes — vendor data from a scanner — this declaration also tells PDAL to preserve those incoming values rather than dropping them at read time.

# Step 4 — Compute values with filters.assign

Use a separate stage per logical group of dimensions. PDAL evaluates stages in declaration order, so a dimension computed in stage N is available as a variable in stage N+1:

{
  "type": "filters.assign",
  "value": [
    "norm_intensity = (Intensity - 100.0) / 1500.0",
    "survey_confidence = 128"
  ],
  "where": "Intensity >= 100 && Intensity <= 1600"
}

The where clause restricts the assignment to matching points. Points outside the predicate retain the default value of 0 for newly allocated dimensions — a common source of unexpected zeros. If you need a different default, add an unconditional filters.assign before the conditional one to initialize the dimension across all points first.

# Step 5 — Re-declare every dimension in the writer

writers.las requires an explicit extra_dims declaration for each custom field. Omitting it produces no error but drops the dimension from the file. The type string in the writer must match — or be safely widened from — the type used in the expression:

{
  "type": "writers.las",
  "filename": "output_mapped.laz",
  "compression": "laszip",
  "minor_version": 4,
  "dataformat_id": 6,
  "extra_dims": "norm_intensity=float,survey_confidence=uint8"
}

# Complete Working Example

The following pipeline is self-contained. Save it as mapping_pipeline.json and run it against any LAZ file that has standard Intensity, ReturnNumber, and NumberOfReturns dimensions. It maps two custom attributes and writes LAS 1.4 with verified Extra Bytes:

{
  "pipeline": [
    {
      "type": "readers.las",
      "filename": "input_cloud.laz",
      "extra_dims": "norm_intensity=float,survey_confidence=uint8"
    },
    {
      "type": "filters.assign",
      "value": "norm_intensity = 0.0"
    },
    {
      "type": "filters.assign",
      "value": [
        "norm_intensity = (Intensity - 100.0) / 1500.0",
        "survey_confidence = 128"
      ],
      "where": "Intensity >= 100 && Intensity <= 1600"
    },
    {
      "type": "writers.las",
      "filename": "output_mapped.laz",
      "compression": "laszip",
      "minor_version": 4,
      "dataformat_id": 6,
      "extra_dims": "norm_intensity=float,survey_confidence=uint8"
    }
  ]
}

Run from the CLI:

pdal pipeline mapping_pipeline.json

Stage breakdown:

1. Reader — ingests the LAZ and allocates norm_intensity (float) and survey_confidence (uint8) buffer slots for all points.
2. First assign — initializes norm_intensity to 0.0 for every point unconditionally, so no point carries an uninitialized slot.
3. Second assign — overwrites norm_intensity with the normalized value and sets survey_confidence = 128 only for points in the valid intensity range. Points outside the range keep norm_intensity = 0.0 from step 2.
4. Writer — serializes to LAS 1.4 point format 6 with both custom dimensions registered as Extra Bytes in the file’s VLR.

The equivalent Python invocation with validation, compatible with the patterns in the Attribute Mapping cluster documentation:

import json
import pdal

pipeline_json = {
    "pipeline": [
        {
            "type": "readers.las",
            "filename": "input_cloud.laz",
            "extra_dims": "norm_intensity=float,survey_confidence=uint8",
        },
        {"type": "filters.assign", "value": "norm_intensity = 0.0"},
        {
            "type": "filters.assign",
            "value": [
                "norm_intensity = (Intensity - 100.0) / 1500.0",
                "survey_confidence = 128",
            ],
            "where": "Intensity >= 100 && Intensity <= 1600",
        },
        {
            "type": "writers.las",
            "filename": "output_mapped.laz",
            "compression": "laszip",
            "minor_version": 4,
            "dataformat_id": 6,
            "extra_dims": "norm_intensity=float,survey_confidence=uint8",
        },
    ]
}

p = pdal.Pipeline(json.dumps(pipeline_json))
count = p.execute()
arr = p.arrays[0]

assert "norm_intensity" in arr.dtype.names, "norm_intensity dimension missing"
assert "survey_confidence" in arr.dtype.names, "survey_confidence dimension missing"
print(f"Processed {count:,} points. norm_intensity range: "
      f"{arr['norm_intensity'].min():.4f}–{arr['norm_intensity'].max():.4f}")

# Key Parameter Table

# Verification

After execution, confirm persistence at three levels:

1. Dimension names in the output file:

pdal info --schema output_mapped.laz

Look for norm_intensity and survey_confidence in the schema list. If absent, the writer extra_dims declaration is missing or misspelled.

2. In-memory array inspection:

arr = p.arrays[0]
required = {"norm_intensity", "survey_confidence"}
missing = required - set(arr.dtype.names)
assert not missing, f"Missing from array: {missing}"

3. Value range check:

assert arr["norm_intensity"].min() >= 0.0, "norm_intensity below 0"
assert arr["norm_intensity"].max() <= 1.0, "norm_intensity above 1 — check Intensity range"
assert set(arr["survey_confidence"].tolist()).issubset({0, 128}), "unexpected confidence values"

For pipeline validation before running against large production datasets, add a point-count parity check: compare pdal info --summary input.laz against pdal info --summary output_mapped.laz. filters.assign does not drop or duplicate points; any count mismatch indicates an upstream filter is also active in the pipeline.

# Gotchas and Edge Cases

Silent type truncation without error

PDAL will truncate a double or float result to fit a narrower target type declared in extra_dims. Assigning norm_intensity = Intensity / 65535.0 and declaring norm_intensity=uint8 in the writer converts every fractional value to 0 without raising any warning. Always match the type in extra_dims to the precision the expression produces: use float for normalized ratios and double only when sub-millimetre coordinate precision is required.

Uninitialized slots when where clause is present

When a conditional filters.assign is the first stage to touch a dimension, points that do not satisfy the where predicate carry a default of 0. For a binary confidence flag this may be acceptable; for a normalized float that feeds a classifier, a zero is indistinguishable from a valid measurement at the lower bound. Initialize unconditionally first, then apply the conditional override.

Schema drift during filters.merge

When merging point clouds from different sources — a common pattern before PDAL stage chaining into a unified processing graph — filters.merge drops dimensions that are not present in all input views. If one tile already has norm_intensity mapped and another does not, the merged output loses the dimension. Pre-process all tiles through the same mapping pipeline before merging, or use separate per-tile pipelines and combine outputs at the file level.

filters.python dimension creation requires explicit type registration

When using filters.python to compute a custom dimension, you must call ins.dimension_types in the add_dimension callback to register the type before the filter function runs. Unlike filters.assign, Python filters do not infer dimension types from the expression — the type must be declared programmatically. Omitting this step raises Dimension not found at the writer stage even when the array appears correct in the Python callback.

# Related

• Attribute Mapping — parent cluster covering the full dimension-mapping workflow, schema audit patterns, and production validation strategies
• PDAL Pipeline Architecture & Execution — pillar overview of the reader → filter → writer execution model and how dimensions propagate through the DAG
• PDAL Stage Chaining — how PDAL passes point buffers between stages and the ordering constraints that govern dimension availability
• Pipeline Filtering Logic — conditional expression evaluation and how where predicates interact with dimension assignment
• Pipeline Validation — pre-execution schema and dimension checks to catch configuration errors before running against full datasets