Python LiDAR

Reporting NVA and VVA Accuracy

TL;DR: NVA uses open-terrain checkpoints: report RMSEz and, where the specification asks for it, the legacy 95 % figure 1.96 × RMSEz. VVA uses vegetated checkpoints: report the 95th percentile of absolute errors. Compare both against the thresholds for the target quality level — for USGS QL2, RMSEz ≤ 10 cm, NVA ≤ 19.6 cm and VVA ≤ 30 cm — and state the checkpoint counts, datums and method alongside the numbers.

# Context and Motivation

This guide is part of Vertical Accuracy Assessment for LiDAR. Two statistics are needed because errors behave differently under vegetation. On open, hard ground, LiDAR errors are close to normally distributed, and RMSEz summarizes them well. Under canopy, the laser reaches the ground less often, ground classification is harder, and errors develop a long tail — a few checkpoints may be off by half a metre. A normal-distribution statistic would understate that tail, so vegetated accuracy is reported with a percentile that makes no distribution assumption.

Standards have evolved. The ASPRS Positional Accuracy Standards for Digital Geospatial Data, edition 2 (2023), report accuracy as RMSE and dropped the requirement to publish 95 % confidence values; the USGS Lidar Base Specification and many contracts still express thresholds as NVA and VVA. The practical approach is to compute everything and report what the governing specification asks for.

Two distributions, two statistics Left: a symmetric bell-shaped distribution of open-terrain errors, summarized by RMSEz and, for legacy reporting, 1.96 times RMSEz. Right: a right-skewed distribution of absolute vegetated errors with a long tail, summarized by its 95th percentile, marked with a vertical line well out in the tail. open terrain → NVA vegetated → VVA 95th percentile RMSEz (× 1.96 for legacy 95 %) |ΔZ| with a long tail

# Prerequisites and Assumptions

  • Checkpoint results with ΔZ and a land-cover label, as produced in computing RMSEz against survey checkpoints.
  • Datum issues resolved: the mean open-terrain error should be small relative to its standard deviation before reporting.
  • The governing specification and target quality level.
  • Python with pandas and NumPy.

# Step-by-Step Implementation

# Step 1 — Split by land cover

Use only open-terrain checkpoints for NVA and only vegetated checkpoints for VVA. Never pool them.

# Step 2 — Compute NVA

RMSEz of open ΔZ, and 1.96 × RMSEz if the specification uses the 95 % confidence form.

# Step 3 — Compute VVA

The 95th percentile of |ΔZ| for vegetated checkpoints. With few vegetated checkpoints, the percentile is effectively the second- or third-largest error — say so.

# Step 4 — Compare with thresholds

Look up the thresholds for the target quality level and record pass or fail for each statistic.

# Step 5 — Write the report table

Include counts, datums, geoid model, method (TIN of class 2 returns), statistics, thresholds and outcome. Keep the per-checkpoint table as an appendix.

# Complete Working Example

python
"""NVA/VVA report against USGS quality-level thresholds."""
from __future__ import annotations

import numpy as np
import pandas as pd

# Thresholds in metres (USGS Lidar Base Specification quality levels; confirm the edition).
QL = {
    "QL0": {"rmsez": 0.050, "nva95": 0.098, "vva95": 0.147},
    "QL1": {"rmsez": 0.100, "nva95": 0.196, "vva95": 0.300},
    "QL2": {"rmsez": 0.100, "nva95": 0.196, "vva95": 0.300},
    "QL3": {"rmsez": 0.200, "nva95": 0.392, "vva95": 0.588},
}


def report(res: pd.DataFrame, level: str = "QL2") -> pd.DataFrame:
    ok = res[res.status == "ok"]
    open_dz = ok.loc[ok.cover == "open", "dz"].to_numpy()
    veg_dz = ok.loc[ok.cover == "vegetated", "dz"].to_numpy()
    t = QL[level]

    rmse = float(np.sqrt(np.mean(open_dz ** 2)))
    nva95 = 1.96 * rmse
    vva95 = float(np.percentile(np.abs(veg_dz), 95)) if veg_dz.size else np.nan

    rows = [
        ("Open checkpoints (n)", len(open_dz), "", ""),
        ("Vegetated checkpoints (n)", len(veg_dz), "", ""),
        ("Mean error, open (m)", round(open_dz.mean(), 3), "", ""),
        ("RMSEz, open (m)", round(rmse, 3), t["rmsez"], rmse <= t["rmsez"]),
        ("NVA at 95 % (1.96 × RMSEz) (m)", round(nva95, 3), t["nva95"], nva95 <= t["nva95"]),
        ("VVA, 95th percentile |ΔZ| (m)", round(vva95, 3), t["vva95"], vva95 <= t["vva95"]),
    ]
    table = pd.DataFrame(rows, columns=["Statistic", "Value", f"{level} threshold", "Pass"])
    table["Pass"] = table["Pass"].map({True: "yes", False: "no", "": ""})
    return table


if __name__ == "__main__":
    res = pd.read_csv("qa/checkpoint_results.csv")
    res["dz"] = res.z_lidar - res.z_check
    tbl = report(res, "QL2")
    print(tbl.to_markdown(index=False))
    tbl.to_csv("qa/accuracy_report_ql2.csv", index=False)

Rendered as Markdown, the table drops straight into a delivery report:

text
| Statistic                       |  Value | QL2 threshold | Pass |
|:--------------------------------|-------:|--------------:|:-----|
| Open checkpoints (n)            | 42     |               |      |
| Vegetated checkpoints (n)       | 28     |               |      |
| Mean error, open (m)            | 0.004  |               |      |
| RMSEz, open (m)                 | 0.052  | 0.1           | yes  |
| NVA at 95 % (1.96 × RMSEz) (m)  | 0.102  | 0.196         | yes  |
| VVA, 95th percentile |ΔZ| (m)   | 0.214  | 0.3           | yes  |
Where the result sits against each quality level Two horizontal scales in centimetres. On the NVA scale, thresholds are marked for QL0 at 9.8, QL1 and QL2 at 19.6, and QL3 at 39.2; the measured 10.2 falls between QL0 and QL2. On the VVA scale, thresholds are marked at 14.7, 30 and 58.8; the measured 21.4 falls between QL0 and QL2. The project meets QL2 on both. NVA QL0 9.8QL1/2 19.6QL3 39.2 measured 10.2 VVA QL0 14.7QL1/2 30QL3 58.8 measured 21.4 centimetres; thresholds from the USGS Lidar Base Specification quality levels

# Key Parameter Table

Quality level RMSEz (cm) NVA 95 % (cm) VVA 95th pct (cm) Typical use
QL0 ≤ 5.0 ≤ 9.8 ≤ 14.7 Engineering, dense corridor work
QL1 ≤ 10.0 ≤ 19.6 ≤ 30.0 High-density regional mapping
QL2 ≤ 10.0 ≤ 19.6 ≤ 30.0 National programme baseline
QL3 ≤ 20.0 ≤ 39.2 ≤ 58.8 Older or lower-density collections

These values follow the USGS Lidar Base Specification’s quality-level tables; check the edition your contract cites, because thresholds and wording are revised periodically.

# Verification

  • Counts first. A VVA from eight checkpoints is the largest error in all but name. State counts in the report and flag statistics computed from fewer than 20 checkpoints.
  • Reproduce from the appendix. Anyone should be able to recompute every number from the per-checkpoint table you ship. Test that by computing the report from the saved CSV, not from in-memory data.
  • Consistency with relative accuracy. A project that passes NVA but shows large swath-to-swath differences probably has checkpoints concentrated in well-behaved areas; see measuring swath-to-swath relative accuracy.

# Gotchas and Edge Cases

1.96 assumes normal, unbiased errors. When mean error is not near zero, 1.96 × RMSEz is not a 95 % bound. That is one reason the newer ASPRS edition reports RMSE directly. Fix the bias, or report RMSE and bias separately.

Percentiles on small samples. NumPy’s default percentile interpolates between order statistics. With 20 vegetated checkpoints, the 95th percentile lies between the 19th and 20th largest values. Document the method (NumPy “linear”) so others reproduce the number.

Land cover labels. Label checkpoints in the field, not afterwards from imagery. A checkpoint called “open” that sits under a thin canopy silently inflates NVA.

Small samples and the 95th percentile Two rows of sorted absolute errors. With 10 vegetated checkpoints, the 95th percentile lands between the 9th and 10th values, essentially the worst checkpoint. With 60 checkpoints, it lands near the 57th value, leaving the three worst above it and giving a more stable statistic. n = 10 p95 ≈ the worst checkpoint n = 60 p95 below the three worst

Specification language. “Meets QL2” is a statement about the whole specification — density, classification, accuracy and more — not accuracy alone. Say “meets QL2 vertical accuracy requirements” unless every part has been checked.

# Frequently Asked Questions

What does NVA mean in LiDAR accuracy reporting?

Non-vegetated vertical accuracy: accuracy measured with checkpoints on open terrain such as bare ground, short grass and pavement. It is based on RMSEz and, in legacy reporting, expressed at 95 percent confidence as 1.96 times RMSEz.

What does VVA mean?

Vegetated vertical accuracy: accuracy measured with checkpoints under vegetation, reported as the 95th percentile of absolute vertical errors because those errors are not normally distributed.

What are the QL2 vertical accuracy thresholds?

Under the USGS Lidar Base Specification, QL2 requires RMSEz of 10 centimetres or better, NVA at 95 percent of 19.6 centimetres or better, and VVA of 30 centimetres or better. Confirm against the edition your project cites.

Do the current ASPRS standards still use NVA and VVA?

Edition 2 of the ASPRS Positional Accuracy Standards reports vertical accuracy as RMSE and no longer requires 95 percent confidence figures. Many specifications and contracts still use NVA and VVA, so compute both and report what your governing document requires.

Related in Vertical Accuracy Assessment for LiDAR

↑ Vertical Accuracy Assessment for LiDAR overview --- title: "Vertical Accuracy Assessment for LiDAR" description: "Measure and report LiDAR vertical accuracy in Python: surveyed... Interpolating LiDAR Elevations at Checkpoints --- title: "Interpolating LiDAR Elevations at Checkpoints" description: "Get the LiDAR elevation at a surveyed checkpoint the way accuracy... Computing RMSEz Against Survey Checkpoints --- title: "Computing RMSEz Against Survey Checkpoints" description: "Calculate RMSEz, mean error and standard deviation of LiDAR elevations... Measuring Swath-to-Swath Relative Accuracy --- title: "Measuring Swath-to-Swath Relative Accuracy" description: "Check internal consistency between overlapping LiDAR flightlines...