Convert: SICD → geocoded COG¶
Turn a complex SICD product into a north-up, map-ready cloud-optimized GeoTIFF
using SICD's own image-projection model. Optional DEM terrain
orthorectification, geoid correction, and radiometric terrain flattening are
documented in Terrain. Requires the [convert] extra.
Both converters also take calibration= — one of sigma0, beta0, gamma0
or rcs — which scales pixel power by the SICD's own Radiometric scale
factors so the output is a physical backscatter coefficient (or an absolute
radar cross-section) rather than relative brightness. It composes with the
terrain flattening: rtc_model="facet" with calibration="gamma0" is a
terrain-flattened gamma-nought product. Ask sicd_calibration_types() what a
given file supports — products that carry no scale factors, which includes most
of Umbra's open data, raise rather than returning an uncalibrated number that
looks calibrated.
That refusal — and the matching one for noise_model="measured" on a product
that states no floor — is a
UnsupportedMeasurementError, the family of errors that are
facts about a product rather than about the request, and it is raised off the
metadata before any pixels are read. A scene that could never have been
calibrated therefore costs its header rather than a multi-gigabyte complex read
followed by an amplitude detection. A malformed request — an unknown
calibration name, an even filter window — stays a plain ValueError, because
the caller can fix that one.
sicd_to_geocoded_cog also takes bbox= (umbra convert --clip-bbox) — a
lon/lat rectangle to convert instead of the whole scene. A SICD is tens of
square kilometres at 16–25 cm, and every step above is proportional to it, so
keeping a site out of a collect otherwise costs the whole collect. With bbox=
the ground rectangle is turned back into the image window that covers it, only
that window is read from the product, and the output is cropped to the request —
the same pixels the whole-scene conversion would have produced there, for a
fraction of the memory, warp and disk. The download stays whole-product: a
slant-plane NITF has no map grid to range-read.
sicd_to_geocoded_cog(src, dst, *, decibels=True, gcp_grid=15, resolution=None, resampling='bilinear', projection_type='HAE', dem=None, geoid=None, rtc=False, rtc_reference_deg=None, rtc_model='cosine', calibration=None, noise_subtract=False, noise_model='measured', speckle_filter=None, speckle_window=SPECKLE_WINDOW_DEFAULT, bbox=None, clip_report=None)
Geocode a SICD to a north-up EPSG:4326 cloud-optimized GeoTIFF.
Reads the complex SICD, detects amplitude, and warps it onto a regular
geographic grid using SICD's own image-projection model (a gcp_grid ×
gcp_grid lattice of ground control points). The result opens on a web
map, in QGIS, or as a georeferenced :class:xarray.DataArray (via
:func:umbra_py.to_xarray) with no further work.
By default the geocoding is flat-earth: pixels are placed on the scene's
height-above-ellipsoid plane (projection_type="HAE"), which is exact
over flat terrain and adequate for map placement elsewhere. Pass dem — a
path to any rasterio-readable digital elevation model — to terrain-
orthorectify instead: each control point is walked onto the DEM surface
(project → sample the DEM → reproject, until it converges), so hilltops and
valley floors land in their true ground position. dem supersedes
projection_type. Pass dem="auto" to fetch the covering Copernicus
GLO-30 DEM tiles for the scene automatically (see :mod:umbra_py.dem).
Pass bbox to convert only an area of interest. A SICD scene is tens
of square kilometres at 16–25 cm, so converting all of it to keep a corner
of it costs the whole warp, a scene-sized float raster in memory, and a
scene-sized COG on disk. With bbox only the image window covering that
ground is read from the product, everything downstream is sized to it, and
the output is cropped to the requested rectangle — so the cost of a
conversion follows the area someone asked for rather than the area the
satellite happened to collect. The download is whole-product either way (a
slant-plane NITF has no map grid to range-read), which is why the saving is
in the processing rather than in the bytes fetched. Pass clip_report to
learn how large that saving was: it is called once, before the read, with a
:class:ClipSavings pricing the window read against the whole scene (what
umbra convert's clipped line prints).
Parameters¶
src:
Path to a SICD NITF file.
dst:
Output GeoTIFF path (written as a COG).
decibels:
If true, write the decibel (log-amplitude) scale; otherwise raw
magnitude.
gcp_grid:
Edge of the square lattice of ground control points sampled across the
image (clamped to the image size). More points track the sensor
geometry more faithfully at a small projection cost.
resolution:
Output pixel size in degrees. None picks the finer of the two
per-axis ground sample distances so no resolution is thrown away.
resampling:
Warp kernel, one of :data:RESAMPLING_METHODS.
projection_type:
SICD image-projection type when dem is not given: "HAE"
(flat-earth, the default), "PLANE", or "DEM".
dem:
Optional path to a digital elevation model (any raster rasterio can
open, e.g. a Copernicus/SRTM COG), or the literal "auto" to
auto-fetch the covering Copernicus GLO-30 tiles for the scene
(:func:umbra_py.dem.fetch_dem_for_bbox). When given, the scene is
terrain-orthorectified against it and projection_type is ignored;
heights are read in the DEM's own vertical datum. None keeps the
flat-earth projection.
geoid:
Optional path to a geoid-undulation grid (any raster rasterio can open,
giving the ellipsoid-minus-geoid separation N in metres, e.g. an
EGM96/EGM2008 undulation GeoTIFF), or the literal "auto" to fetch a
global EGM geoid grid for the scene automatically
(:func:umbra_py.geoid.fetch_geoid_grid). Global DEMs quote orthometric
height above the geoid, but SICD projects against the ellipsoid, so with
a DEM this adds N at each point to convert the sampled height to HAE
before projecting — survey-grade geolocation over relief. Requires dem
(it corrects DEM heights); passing it without dem is an error. Where
the grid has no coverage the undulation is taken as 0 (the DEM height
is used uncorrected). None reads DEM heights as-is (correct to within
the local geoid–ellipsoid separation, ample for map placement).
rtc:
If true, radiometrically terrain-flatten the output: after geocoding,
scale each pixel by a terrain-geometry correction (see rtc_model)
computed from the DEM slope and the scene look geometry, so slopes tilted
toward or away from the radar no longer look artificially bright or dark.
Requires dem (the correction needs terrain); passing it without
dem is an error. This is a geometric normalisation of detected
amplitude, not a calibrated gamma-nought product.
rtc_reference_deg:
Reference incidence angle (degrees) the flattening normalises to. None
uses the scene incidence angle, so flat terrain is left unchanged and only
slopes are corrected.
rtc_model:
Which terrain-flattening model to apply when rtc is true, one of
:data:RTC_MODELS:
* ``"cosine"`` (the default) scales power by
``cos(reference) / cos(local_incidence)`` using the full 3-D local
incidence angle — the standard geometric cosine correction.
* ``"area"`` scales power by
``sin(local_range_incidence) / sin(reference)``, the projected-area /
foreshortening correction. It measures incidence in the range–vertical
plane, so it targets the range-direction foreshortening and layover that
dominate radiometric terrain distortion — separating them from the
azimuth-direction tilt the per-pixel cosine model folds in. It is an
honest first-order step toward area-based gamma-nought normalisation.
* ``"gamma"`` scales power by ``cos(reference) * nz / cos(local_incidence)``,
the per-pixel facet-area (gamma-nought) normalisation. It normalises by
the local illuminated facet area projected into the plane perpendicular
to the look direction, using the full 3-D facet normal and adding the
true tilted-facet-area term ``nz`` (cosine of the slope from horizontal)
that the ground-referenced ``"cosine"`` and range-plane ``"area"``
models both omit.
* ``"facet"`` normalises each pixel by the illuminated area accumulated
in the radar cell it images into — the image-space illuminated-area
integration (Small 2011). Every facet is projected into the scene's
``(slant_range, azimuth)`` geometry and its true tilted area,
projected perpendicular to the look direction, is binned there; a
pixel's factor is the flat-terrain reference over the **total** area
in its cell. The other three correct each pixel from its own slope, so
only this one measures **layover** — where terrain folds several
facets into one cell, their areas sum and all of them are suppressed
together. Over a planar slope it reduces to the product of the
``"area"`` and ``"gamma"`` factors.
On their own all four are a normalisation of *detected amplitude*, in
whatever arbitrary units the product's pixels carry. Pair them with
``calibration=`` to make the result a physical backscatter coefficient:
``calibration="gamma0"`` with ``rtc_model="facet"`` is the terrain-
flattened gamma-nought product.
calibration:
Optional radiometric calibration, one of :data:CALIBRATION_TYPES,
applied to the detected amplitude before geocoding — the SICD scale
factors are polynomials in image coordinates, so image space is where
they are defined. It multiplies pixel power by the scale factor the
product's own Radiometric metadata supplies, which is what turns a
relative brightness into a physical quantity: "sigma0" /
"beta0" / "gamma0" are the backscatter coefficients referenced
to unit ground, slant-plane and perpendicular-to-look area respectively,
and "rcs" is the absolute radar cross-section in m². With
decibels=True the output is that coefficient in dB directly
(10*log10); in linear magnitude it is the calibrated amplitude,
whose square is the coefficient. Composes with rtc: both are
power-domain factors, so a calibrated and terrain-flattened scene is
the two applied together.
``None`` (the default) leaves the output uncalibrated, which is what
Umbra's open products generally require — they usually ship without a
``Radiometric`` block, and asking for a calibration the metadata cannot
support is a self-describing error rather than a plausible-looking
number. :func:`sicd_calibration_types` reports what a given file
supports. MultiRTC interop remains deferred (`STRATEGY.md` 5.5).
noise_subtract:
If true, subtract the receiver's own thermal-noise floor — the SICD's
Radiometric.NoiseLevel.NoisePoly, evaluated per pixel — from the
detected power before anything scales it. A measured pixel is the
ground's echo plus that floor, and over a low-backscatter surface
(calm water, radar shadow, dry sand) the floor is most of it, so an
uncorrected value there reports the sensor's sensitivity as if it were
the scene's brightness — and, because the floor varies across the swath,
varies with the geometry rather than the ground. Subtraction is the one
correction here that is not a multiplicative factor, so it is applied
first, in image space, on raw power: calibration and rtc then
scale what is left.
``False`` (the default) leaves the noise floor in, which is what an
uncalibrated relative image already assumes. Where the floor comes from
is ``noise_model``.
noise_model:
Where the subtracted floor comes from, one of :data:NOISE_MODELS.
``"measured"`` (the default) reads the product's own
``Radiometric.NoiseLevel.NoisePoly``, evaluated per pixel, so the floor
tracks the across-swath variation the sensor's metadata describes. Only
an ``ABSOLUTE`` noise level qualifies: a ``RELATIVE`` one describes the
floor's shape without its level, and a product may declare no noise
level at all; both raise a self-describing error rather than subtracting
a guess, and :func:`sicd_noise_level` reports which case a file is in
before you ask.
``"estimated"`` infers one constant floor from the scene's own detected
power — its :data:`NOISE_ESTIMATE_PERCENTILE`-th percentile, on the
argument that a SAR scene's darkest surfaces (calm water, radar shadow,
smooth roads) return essentially nothing, so what is recorded there is
the receiver. It needs no metadata, which is the whole point: Umbra's
open products generally ship without a ``Radiometric`` block, so
``"measured"`` refuses on most of the archive this library exists for.
The trade is real and is why the models are named separately — the
estimate is one scalar, so it cannot follow the swath, and it assumes the
scene contains dark ground at all (over imagery that is bright everywhere
it removes signal).
``"estimated-range"`` answers the first half of that trade. It takes the
same low-tail read *per range line* — SICD stores range along the image
rows — and fits those per-line floors against range
(:func:`_estimate_noise_profile`), so the subtracted floor follows the
swath the way the measured one does while still needing no metadata. The
fit is what makes it work on real imagery: it interpolates over the lines
that had no dark ground to read, and it drops the lines whose tail sits
far above it, since bright ground can only push a line's low tail up. It
remains an inference and assumes the scene has dark ground *somewhere*
along range, so it is recorded as its own value rather than as a better
``"estimated"``, and it reports the swing it found in
``UMBRA_NOISE_FLOOR_SPREAD_DB`` — near zero means there was nothing here
the constant model was missing.
The raster records which one ran (``UMBRA_NOISE_SUBTRACTION`` of
``"absolute"``, ``"estimated"`` or ``"estimated-range"``, plus
``UMBRA_NOISE_FLOOR_DB`` for the inferred ones), and
:func:`umbra_py.load.to_stack` refuses to difference a series that mixes
any two of them.
speckle_filter:
Optional speckle filter, one of :data:SPECKLE_FILTERS, applied to
detected power in image space. Speckle is not sensor noise and no floor
subtraction touches it: it is the interference pattern coherent
illumination produces on a rough surface, so a single-look pixel's power
is exponentially distributed about the surface's true backscatter — its
standard deviation equals its mean. That is why a single Umbra pixel is
a poor measurement of a surface even after calibration, why a
pixel-by-pixel difference between two passes is dominated by speckle
rather than by change, and why every SAR workflow averages before it
measures. Averaging is also the only correction available, because
speckle is a property of the illumination rather than an additive error to
remove.
``"boxcar"`` averages the window unconditionally — the multilook, maximum
variance reduction, blind to edges. ``"lee"`` averages only where the
window's variability is what speckle alone would produce and keeps the
pixel where it is more variable than that (Lee 1980), so edges and points
survive at the cost of smoothing less. Both run in the power domain, after
the noise subtraction and the calibration: the noise estimators read the
scene's own low tail, whose *distribution* a filter would narrow, and a
smooth multiplicative scale factor commutes with a local average anyway,
so the filter goes last in image space — before the warp, which is where
the pixel grid stops being the radar's.
What it costs is resolution, and that cost is the reason it is opt-in: a
window that averages ``N`` pixels reports ground ``N`` pixels across. The
raster records the filter and its window (``UMBRA_SPECKLE_FILTER`` /
``UMBRA_SPECKLE_WINDOW``) and :func:`umbra_py.load.to_stack` refuses to
difference a series that mixes two of them, since the smoothing would
otherwise be read as change. It also records what the filter *achieved* on
this scene — the equivalent number of looks before and after
(``UMBRA_SPECKLE_ENL_BEFORE`` / ``_AFTER``), which is the honest answer to
"how much speckle did that remove?" and is generally well below the
window's pixel count, because Umbra samples finer than it resolves.
speckle_window:
Edge of the odd, centred window speckle_filter averages over
(:data:SPECKLE_WINDOW_DEFAULT). Larger windows remove more speckle and
more detail; the cost is independent of the size
(:func:_box_sum), so the choice is about the imagery rather than the
runtime. Ignored when speckle_filter is None.
bbox:
Optional area of interest (min_lon, min_lat, max_lon, max_lat) in
WGS-84 degrees. Only the image window covering that ground is read,
amplitude-detected, calibrated, projected and warped, and the output is
cropped to the rectangle (intersected with the scene, so asking for more
ground than the scene holds returns the overlap rather than a nodata
margin). Because a SICD lies in the slant plane, the window found is a
deliberate superset — the smallest axis-aligned image rectangle that
contains the rotated ground region, padded by one search step — so the
pixels read exceed the area kept, and both are a small fraction of the
scene for a small area of interest. None (the default) converts the
whole scene. The clip is not recorded in the provenance tags: the
output's own geotransform states exactly which ground it covers, and the
tags record what a pixel value means rather than where it is.
clip_report:
Optional callback, invoked once with a :class:ClipSavings when a
bbox clip is applied (never otherwise), pricing the image window read
against the whole scene. It is how umbra convert prints its
clipped line; a caller who does not pass it sees no behaviour change.
sicd_to_amplitude_geotiff(src, dst, *, decibels=True, calibration=None, noise_subtract=False, noise_model='measured', speckle_filter=None, speckle_window=SPECKLE_WINDOW_DEFAULT)
Read a SICD (complex) image and write its detected amplitude as a GeoTIFF.
This is an inspection-quality product in the slant plane: the output is
not geocoded. For a map-ready raster use :func:sicd_to_geocoded_cog, or
use the item's GEC asset directly when one exists.
Parameters¶
src:
Path to a SICD NITF file.
dst:
Output GeoTIFF path.
decibels:
If true, scale amplitude to dB (20*log10); otherwise raw magnitude.
calibration:
Optional radiometric calibration, one of :data:CALIBRATION_TYPES,
applied from the SICD's own Radiometric scale-factor polynomial (see
:func:sicd_to_geocoded_cog for what the calibrated values mean).
None writes the uncalibrated amplitude. Raises when the product
carries no scale factor for the requested calibration — ask
:func:sicd_calibration_types first to check.
noise_subtract:
If true, subtract the receiver's own thermal-noise floor from pixel
power before any calibration (see :func:sicd_to_geocoded_cog).
noise_model:
Where that floor comes from, one of :data:NOISE_MODELS:
"measured" (the default) reads the SICD's own
Radiometric.NoiseLevel and raises when the product declares no
absolute level — ask :func:sicd_noise_level first to check;
"estimated" infers one constant floor from the scene's own pixels
and needs no metadata; "estimated-range" infers one per range line
and fits it against range. Ignored when noise_subtract is false.
speckle_filter:
Optional speckle filter, one of :data:SPECKLE_FILTERS, applied to
detected power after any calibration (see
:func:sicd_to_geocoded_cog for what each one trades). None leaves
the scene's full speckle in.
speckle_window:
Edge of the odd, centred window that filter averages over.
Which radiometric calibrations a SICD product's metadata supports.
Returns the subset of :data:CALIBRATION_TYPES whose scale-factor
polynomial the file's Radiometric metadata actually carries — empty for
an uncalibrated product. Ask this before passing calibration= to
:func:sicd_to_geocoded_cog when you want to check rather than handle the
error, e.g. when deciding whether a scene can enter a calibrated stack.
Parameters¶
src: Path to a SICD NITF file.
Noise floor¶
A measured pixel is the ground's echo plus the receiver's own thermal noise, so
over a dark surface a calibrated value reports the sensor rather than the scene.
noise_subtract=True (umbra convert --subtract-noise) removes it in the power
domain, before anything scales it, and noise_model= says where the floor comes
from: "measured" reads the product's own Radiometric.NoiseLevel polynomial,
"estimated" infers one constant from the scene's own darkest pixels, and
"estimated-range" infers one per range line and fits it against range so the
inferred floor follows the swath. The inferred models need no metadata, which is
the point — most of Umbra's open products carry no Radiometric block at all, so
"measured" refuses on exactly the archive this library exists for. Ask
sicd_noise_level() which kind a file declares.
compare_noise_models() (umbra convert --noise-check) is how those inferences
get checked. On a product that does state an absolute floor there is a truth to
score against, so it runs the estimators over the product's pixels and differences
each result against the product's own NoisePoly — reporting the offset the
estimate reads low by and, once that offset is granted, how well it follows the
real floor across the image. It writes nothing and converts nothing.
Speckle¶
Every correction above targets something the sensor added. Speckle is not one of them: coherent illumination of a rough surface interferes with itself, so a single-look pixel's power scatters about the surface's true backscatter with a standard deviation equal to its mean. That is the dominant uncertainty in a calibrated number, the reason a pixel-by-pixel difference between two passes is mostly speckle, and it cannot be subtracted — averaging is the only correction.
speckle_filter= (umbra convert --speckle-filter) does the averaging in the
power domain, last in image space: "boxcar" averages the speckle_window
window unconditionally — the multilook, maximum variance reduction, blind to
edges — and "lee" averages only where the window is no more variable than
speckle alone would explain, keeping edges and points (Lee 1980). No filter is
the default because what it spends is resolution: a window that averages N
pixels reports ground N pixels across, and 25 cm is the reason to use this
archive.
So the raster records both what it did (UMBRA_SPECKLE_FILTER,
UMBRA_SPECKLE_WINDOW — both refused-on-mix by
to_stack, since averaging one pass and not another shows up as
change) and what it achieved: the equivalent number of looks before and after
(UMBRA_SPECKLE_ENL_BEFORE / _AFTER). That pair is the honest answer to "how
much speckle did that remove?", and on a product sampled finer than it resolves
it lands below the window's pixel count — which is a fact about the product, not
a fault in the filter.
dataclass
¶
What one speckle filter did, beyond changing the pixels.
Speckle is not sensor noise: it is the interference pattern coherent illumination produces on a rough surface, so it is multiplicative, it is the same physics on every SAR image, and averaging is the only thing that removes it. Which means a filtered raster differs from an unfiltered one in two ways that both matter downstream — its variance and its effective resolution — and neither is visible in the pixel values afterwards. Hence the record.
Attributes¶
filter:
Which of :data:SPECKLE_FILTERS ran.
window:
Edge of the (odd, centred) window it ran over, in pixels.
enl_before, enl_after:
The scene's equivalent number of looks before and after, as
:func:_estimate_enl reads it: the median block's mean² / variance
of detected power, which is the standard measure of how much speckle is
left. Single-look imagery sits at about 1.0 and every filter's job is to
raise it, so the pair is the filter's own effect measured on the scene it
ran on rather than claimed from its window size. Both read low on a
textured scene, so the ratio is the number to trust rather than either
level. None where the raster was smaller than one measuring block or
no block held enough finite pixels (:data:_ENL_MIN_VALID).
The gain is worth reading rather than assuming: a ``window²``-pixel
boxcar averages ``window²`` *pixels* but only as many independent
*looks* as the product's sampling provides, and Umbra samples finer than
its resolution, so the achieved ENL lands below the window's pixel count.
That gap is a fact about the product, and this is where it shows up.
looks:
The ENL "lee" assumed for the speckle it was separating from scene
structure — :attr:enl_before, i.e. read off the scene rather than
assumed, clamped at single-look (no product has fewer looks than one, so a
lower read is the estimator meeting texture) and falling back to 1.0 where
the scene gave no block to read. None for "boxcar", which needs no
such parameter. It is recorded because the filter's output depends on it,
so a pixel value is not reproducible without it.
Notes¶
Like :class:NoiseSubtraction's diagnostics, enl_before / enl_after
/ looks describe this scene rather than what a pixel value means, so
they are recorded (see :func:conversion_tags) but stay out of
:data:umbra_py.load.MEASUREMENT_PROVENANCE_KEYS: two passes of one site
legitimately differ on them, and refusing a stack over that would end every
series. filter and window are the opposite case and are in that key
set — a 5×5-averaged pass differenced against an unfiltered one reports the
filter as change.
Which noise level a SICD product's metadata declares, if any.
Returns "ABSOLUTE" (the floor is stated, so noise_subtract=True
works), "RELATIVE" (the floor's variation is described but not its
level, so it cannot be subtracted), or None for a product with no noise
metadata at all. Ask this before passing noise_subtract=True when you
want to check rather than handle the error — the same role
:func:sicd_calibration_types plays for the scale factors.
Only the "measured" noise model depends on this: noise_model=
"estimated" infers the floor from the scene's own pixels, so it works on
the products this returns None for — which is most of Umbra's open
archive.
Parameters¶
src: Path to a SICD NITF file.
compare_noise_models(src, *, models=INFERRED_NOISE_MODELS, percentile=NOISE_ESTIMATE_PERCENTILE, bbox=None)
Score the inferred noise floors against the one a product measures.
noise_model="estimated" and "estimated-range" infer the receiver's
floor from the scene's own darkest pixels, which is what lets the correction
run on Umbra's open archive at all — those products carry no Radiometric
block to read a floor from. That is also why the inference was, until this,
only ever argued: the archive it was built for has no truth to check it
against.
A product that does state an ABSOLUTE noise level has both. This runs the
inferred models over such a product's pixels and differences each result
against its own NoisePoly, which turns "does the estimator work?" into
two numbers per model: the offset it reads low by, and — after granting that
offset — how well it follows the real floor across the image
(:class:NoiseModelAgreement). Nothing is written and no conversion is
performed; this is the measurement, not a correction.
What to expect, and what each departure means:
bias_dbnegative on both models, by roughly the same amount. A percentile of a speckled noise-only population sits below that population's mean, so both estimators read low and under-subtract, which leaves a little of the receiver in rather than taking real backscatter out. Positive is the failure the margin diagnostic warns about: too little dark ground, so the low tail was ground.shape_error_dbmuch smaller for"estimated-range"than for"estimated"whenmeasured_spread_dbis wide. That is the whole claim of the fitted profile, and where the floor is genuinely flat there is nothing to separate the two models and they should score alike.
Parameters¶
src:
Path to a SICD NITF file that declares an ABSOLUTE noise level. One
that does not raises, naming what it carries — the same refusal
noise_model="measured" makes, for the same reason: there is no truth
here to score against. :func:sicd_noise_level answers it ahead of time.
models:
Which inferred models to score, a subset of
:data:INFERRED_NOISE_MODELS.
percentile:
The low-tail percentile the estimators read, defaulting to the one they
use in a conversion (:data:NOISE_ESTIMATE_PERCENTILE). Exposed here and
nowhere else on purpose: this is the surface where the number is being
measured rather than trusted, so sweeping it is the point.
bbox:
Optional (min_lon, min_lat, max_lon, max_lat) window to compare over
instead of the whole scene, resolved exactly as bbox= on
:func:sicd_to_geocoded_cog — including evaluating the NoisePoly at
the image coordinates the window actually occupies, without which the
measured floor would be read off the wrong part of the swath.
dataclass
¶
The measured noise floor of one product, and how the estimators did on it.
Attributes¶
source:
File name of the product compared on (the name only, for the same reason
:func:conversion_tags records only that).
shape:
(rows, cols) of the image window the comparison ran over — the whole
scene, or the bbox= window when one was given.
measured_floor_db:
Median of the product's own NoisePoly over that window: the truth
the estimates are scored against.
measured_spread_db:
Peak-to-peak swing of that measured floor across the window. This is the
premise of the whole range-profile model: where it is near zero, a
constant estimate had nothing to miss and the two inferred models should
score alike; where it is wide, it is the artefact a scalar floor leaves
behind.
models:
One :class:NoiseModelAgreement per model scored, in the order asked
for.
dataclass
¶
How closely one inferred noise floor matches the product's measured one.
The two inferred models (:func:_estimate_noise_power,
:func:_estimate_noise_profile) make a documented pair of claims: that the
floor they read is biased low but consistently so, and that the range
profile adds the across-swath shape a constant cannot. Both claims are
about the difference between the inferred floor and the true one, which is
unobservable on a product that states no floor — so this splits the measured
difference into exactly those two parts.
Attributes¶
model:
The model scored, one of :data:INFERRED_NOISE_MODELS.
floor_db:
The level it inferred (the profile's median for "estimated-range"),
the same number the conversion records in UMBRA_NOISE_FLOOR_DB.
spread_db:
Peak-to-peak swing of the inferred floor — 0.0 for the constant
estimate by construction, and UMBRA_NOISE_FLOOR_SPREAD_DB for the
profile. Compare against :attr:NoiseModelComparison.measured_spread_db,
which is how much swing was actually there to find.
bias_db:
Median of inferred − measured in decibels: the offset. Expected
negative, because a percentile of a speckled noise-only population sits
below that population's mean — the estimators read low, which
under-subtracts, which is the safe direction. A positive bias is the
interesting failure: the scene had too little dark ground and the low
tail read was backscatter.
shape_error_db:
RMS of inferred − measured after removing the bias: what is left
once the offset above is granted, i.e. how well the inferred floor
follows the true one across the image. This is the number the range
profile exists to reduce, and on a scene whose floor genuinely varies a
constant estimate cannot score better here than the measured floor's own
RMS deviation about its mean.
residual_db:
RMS of inferred − measured with the bias included — how wrong the
subtracted floor was in absolute terms, which is what the pixels
actually got.
Preflight: ask before downloading¶
sicd_calibration_types() and sicd_noise_level() answer "can this product
support that?" from a file you already have. The expensive half was getting the
file: a SICD's metadata lives inside the NITF, so learning that a pass cannot be
calibrated meant downloading the pass, and learning it about a site's twenty
passes meant downloading twenty.
sicd_capabilities() (umbra preflight) asks the same question over the wire. A
NITF states its own layout in a fixed-width file header, so the SICD XML — a data
extension segment near the end of the file — is located by arithmetic on the
segment table and fetched with two HTTP range requests: tens of kilobytes of a
multi-gigabyte product. It reports what the product declares (which calibrations,
which noise level, the scene's identity), what the answer cost, and the product
size it did not download.
The verdict is not a second opinion. SicdCapabilities.refusal() hands the
parsed metadata to the conversion's own support check, calling the same
coefficient readers, so a preflight that clears a product and a conversion that
then refuses it cannot disagree — only where the metadata came from differs.
preflight_items() asks it of a whole search result, recording an unreadable
acquisition as its own verdict rather than ending the walk, and reads several
products at once (workers=, default DEFAULT_PREFLIGHT_WORKERS) — because once
the answer costs kilobytes, the round trip is the only part left that scales with
the number of passes. The concurrency is a schedule and not an answer: the reads
are independent, the verdicts come back in the order they were asked in (the chip
run pairs them against its own selection positionally), and progress is called
from the calling thread in that same order.
The NITF walk and the XML parse are stdlib, so this needs no extra at all —
including no [convert]. Only confirming a positive answer on a product that
does carry the scale factors reads their coefficients, which needs numpy; every
refusal, which is what most of Umbra's open archive returns, is answerable from a
core install.
Read a SICD product's metadata over the wire and report what it supports.
src is an HTTPS URL or a local NITF path. Only the NITF header and the
XML data extension segment are read — typically a few tens of kilobytes of a
multi-gigabyte product — so this is the question to ask before deciding
which acquisitions are worth downloading.
preflight_items(items, *, asset=PREFLIGHT_ASSET, calibration=None, noise_subtract=False, noise_model='measured', rtc=False, session=None, progress=None, workers=None)
Ask a whole selection which passes can support a measurement.
Reads each acquisition's SICD metadata by range request (see
:func:sicd_capabilities) and applies the conversion's own support check, so
umbra chips --asset SICD --calibrate gamma0 over the survivors is a run
with no refusals in it — decided before a single product is downloaded.
rtc=True adds the third metadata-dependent correction to the question:
whether the product states the collection geometry the flattening tilts by
the terrain's slope.
An acquisition whose metadata cannot be read at all is recorded as
unsupported with the error on :attr:PreflightResult.error rather than
ending the walk: a preflight that dies on the nineteenth scene has failed at
the one thing it is for. :attr:PreflightResult.error_scope then says which
kind of failure it was, because the two are not the same news. A missing
asset, an object that is not there, bytes that are not a NITF and a NITF with
no SICD XML in it are all facts about the product (:data:SCOPE_PRODUCT) —
final, and grounds to drop the pass exactly as a refusal is. A timeout or a
dropped connection is a fact about the wire (:data:SCOPE_TRANSPORT) —
no verdict at all, so a caller should keep the pass. Anything unforeseen
counts as transport, so the cautious branch is the default one.
workers is how many products are read at once
(:data:DEFAULT_PREFLIGHT_WORKERS; None takes that default, 1 walks
serially). What the check costs is two range requests per acquisition, so
what a selection spends is round trips rather than bytes — and that is the
one part of the preflight that scaled with the number of passes rather than
with the answer. Reading several at once removes it without moving a single
verdict: the reads are independent, the shared session's connection pool is
sized for the fan-out, and the results are consumed in the order they were
asked in.
progress is called (index, total, item, result) after each
acquisition, in selection order, from the calling thread — so a CLI progress
line stays one line per pass in the order they were given, and a callback
never has to be thread-safe.
dataclass
¶
SicdCapabilities(source, calibrations, noise_level, look_geometry=None, core_name=None, rows=None, cols=None, polarization=None, bytes_read=0, product_bytes=None, sicd=None)
What one complex product's own metadata says it can support.
Attributes¶
source:
The URL or path the metadata was read from.
calibrations:
The subset of :data:umbra_py.convert.CALIBRATION_TYPES whose
scale-factor polynomial this product carries — empty for an uncalibrated
product, which is most of Umbra's open archive.
noise_level:
"ABSOLUTE" (a floor that can be subtracted), "RELATIVE" (its
variation is described but not its level) or None.
look_geometry:
(incidence_deg, azimuth_deg) from the product's SCPCOA block, or
None where it states neither — the scene-centre geometry radiometric
terrain flattening (--rtc) tilts by the DEM's own slope. Unlike the
two Radiometric answers this is present on most products, which is
precisely why it is worth reporting: a missing one is the exception,
and the run it would have refused is an expensive one.
core_name / rows / cols / polarization:
Identity, for a report that names the scene rather than only its URL.
bytes_read:
Bytes transferred to answer, all of it header and XML.
product_bytes:
The whole product's size, when the source stated one — the download this
did not do.
The refusal a conversion with these settings would raise, or None.
Runs :mod:umbra_py.convert's own _check_measurement_support against
the parsed metadata, so the answer is the conversion's answer rather than
a restatement of it.
dataclass
¶
PreflightResult(item_id, datetime, href, supported, reason=None, hint=None, capabilities=None, error=None, error_scope=SCOPE_NONE)
Whether one acquisition can support the measurement being planned.
supported is false in two quite different situations, and
error_scope is what tells them apart. A product that was read and
refused carries a reason (its own metadata's words) and no error. A
product that could not be read carries an error and an error_scope of
:data:SCOPE_PRODUCT (there is nothing readable at that href — final) or
:data:SCOPE_TRANSPORT (the wire failed — says nothing about the product).
That distinction is the whole reason a caller can act on a failed read at all. Without it every read failure had to take the cautious branch, so a selection whose SICD assets are simply absent was preflighted pass by pass and then downloaded pass by pass anyway — a check that had already found the answer and was not allowed to use it.
property
¶
Whether this verdict is one no later attempt can change.
True for a refusal (the product answered, and the answer is no) and for an unreadable product; false for a transport failure, which is the one outcome that is about the moment rather than about the archive.
dataclass
¶
What a selection of acquisitions can support, and what asking cost.
bytes_read against product_bytes is the whole point of the report:
the second is what discovering the same thing by conversion would have
downloaded.
class-attribute
instance-attribute
¶
How many acquisitions were read at once -- the effective lane count, never more than there were products to read. It says how the answer was obtained rather than what it is: the verdicts, their order and the bytes read are identical at any width.
property
¶
Acquisitions with no readable product behind them.
A verdict, not a mishap: nothing is at the href, or what is there is not
a SICD. Grouped separately from :attr:unreadable because only these two
groups differ in what a batch should do, and this is the group it can
act on.
property
¶
Acquisitions whose metadata read failed on the wire.
No verdict at all — the pass is undecided, so a batch keeps it and finds
out the expensive way. Separated from :attr:missing for exactly that
reason.
JSON-safe form, for umbra preflight --json.
The document is public API, pinned by
docs/schemas/preflight.schema.json.
Fetch a SICD product's XML metadata without reading its pixels.
Returns (xml, bytes_read, product_bytes) — the SICD XML document, how
many bytes were transferred to get it, and the product's total size when the
source states one (Content-Range for a URL, the file size for a path).
src is an HTTPS URL (read with two range requests) or a local NITF path.