Auxiliary Data Tools

dem_autoload

Obtain all relevant DEM tiles for selected geometries.

dem_create

Create a new DEM GeoTIFF file and optionally convert heights from geoid to ellipsoid.

DEMHandler

An interface to obtain DEM data for selected geometries.

get_dem_options

Get the names of all supported DEM type options.

get_egm_lookup

Download lookup tables for converting EGM geoid heights to WGS84 ellipsoid heights.

getasse30_hdr

Create an ENVI HDR file for zipped GETASSE30 DEM tiles.

ImplicitFTP_TLS

FTP_TLS subclass that automatically wraps sockets in SSL to support implicit FTPS.

vrt_check_sources

Check the sanity of all source files of a given VRT.

class pyroSAR.auxdata.DEMHandler(vectorobject=None, buffer=None)[source]

Bases: object

An interface to obtain DEM data for selected geometries. The files are downloaded into the ESA SNAP auxiliary data directory structure. This class is the foundation for the convenience functions dem_autoload() and dem_create().

Parameters:
  • vectorobject (Vector | None) – A vector geometry object. The extent of all features is used for DEM data search and mosaicking. Default None: use the global extent.

  • buffer (int | float | None) – A buffer in degrees to add around the vector object’s extent.

property config: dict[str, Any]

Get DEM configuration options.

create(src, dst, t_srs=None, tr=None, threads=None, geoid_convert=True, nodata=None, resampleAlg=None, dtype=None, pbar=False, **kwargs)[source]

Create a new DEM GeoTIFF file and optionally convert heights from geoid to ellipsoid. This is basically a convenience wrapper around osgeo.gdal.Warp() via spatialist.auxil.gdalwarp().

If src is a list of file names, the extent of the output DEM is determined by the DEMHandler argument vectorobject and the **kwargs argument outputBounds (passed to gdal.Warp). If neither vectorobject nor outputBounds are set, the bounding box extent of all input DEM tiles is used (reprojected to t_srs if necessary). IF both are provided, outputBounds is prioritized because it is in coordinates of t_srs and thus more accurate.

Parameters:
  • dem_type – The input DEM product type. See dem_autoload() for options.

  • product – The input DEM sub-product type. See dem_autoload() for options.

  • src (str | list[str]) – The input dataset(s) as returned by dem_autoload(). A string is expected to point to a VRT file. A list is interpreted as 1..n GDAL-readable datasets. dem_autoload() must be run with return_fnames=True to provide the correct format.

  • dst (str) – The output GeoTIFF file name.

  • t_srs (int | str | SpatialReference | None) – The target geographic reference system. See function spatialist.auxil.crsConvert() for details. Default None: use the crs of src.

  • tr (tuple[int | float, int | float] | None) – The target resolution as (xres, yres). Default None: use the resolution of src.

  • threads (int | str | None) –

    The number of threads to use. Possible values:

    • Default None: use the value of GDAL_NUM_THREADS without modification. If GDAL_NUM_THREADS is None, multi-threading is still turned on and two threads are used, one for I/O and one for computation.

    • integer value: temporarily modify GDAL_NUM_THREADS and reset it once done. If 1, multithreading is turned off.

    • ALL_CPUS: special string to use all cores/CPUs of the computer; will also temporarily modify GDAL_NUM_THREADS.

  • geoid_convert (bool) – Convert geoid heights to WGS84 ellipsoid heights? Only applied if the vertical datum isn’t already WGS84 and product='dem'. The geoid model is inferred from demType. See dem_autoload() for details.

  • nodata (int | float | str | None) –

    The nodata value of dst. Default None: use these values per data type:

    • Byte: 255

    • Int16: -32768

    • Float32: -9999.0

  • resampleAlg (str | None) – The resampling algorithm to be used. See here for options: https://gdal.org/programs/gdalwarp.html#cmdoption-gdalwarp-r. Default None: use mode if the data type of src is Byte (for categorical value masks) and bilinear (for DEM and floating point error masks) otherwise.

  • dtype (str | None) – The data type of dst. Default None: use the value of the source file(s). Data type notations of GDAL (e.g. Float32) and numpy (e.g. int8) are supported. See spatialist.raster.Dtype.

  • pbar (bool) – Add a progressbar?

  • **kwargs –

    Additional keyword arguments to be passed to spatialist.auxil.gdalwarp(). See osgeo.gdal.WarpOptions() for options. The following arguments cannot be set as they are controlled internally:

    • xRes, yRes: controlled via argument tr

    • srcSRS, dstSRS: controlled via arguments t_srs and geoid_convert

    • srcNodata: controlled via nodata value of src

    • dstNodata: controlled via argument nodata

    • outputType: controlled via argument dtype

    • multithread: controlled via argument threads

    • targetAlignedPixels: fixed to True

    • warpOptions: currently used for setting the number of threads. Can be exposed if necessary.

    The following arguments are set if they are not defined in **kwargs:

    • outputBounds: determined from user input. See above.

    • format: set to GTiff

Return type:

None

info_from_filenames(filenames)[source]

Derive DEM type and product from a list of filenames.

Parameters:

filenames (list[str]) – The file names as returned by load() (with vrt=None and return_fname=True) or vrt_check_sources().

Return type:

tuple[str, str]

Returns:

(DEM type, product)

intrange(step)[source]

Generate a sequence of integer coordinates marking the tie points of the individual DEM tiles.

Parameters:

step (int) – The sequence steps.

Return type:

tuple[list[int], list[int]]

Returns:

The integer sequences as (latitude, longitude).

load(dem_type, vrt=None, username=None, password=None, product='dem', crop=True, lock_timeout=600, offline=False, return_fname=True)[source]

Download DEM tiles. The result is either returned in a list of file names or combined into a VRT mosaic. The VRT is cropped to the combined extent of the geometries, but the pixel grid of the source files is preserved and no resampling/shifting is applied.

Parameters:
  • dem_type (str) – The type fo DEM to be used

  • vrt (str | None) – An optional GDAL VRT file created from the obtained DEM tiles. NOTE: VRTs are not suited for geometries crossing the antimeridian.

  • username (str | None) – The download account username.

  • password (str | None) – The download account password.

  • product (str) –

    The sub-product to extract from the DEM product. Options:

    • ’AW3D30’

    • ’dem’: the actual Digital Elevation Model

    • ’msk’: mask information for each pixel (Cloud/Snow Mask, Land water and low correlation mask, Sea mask, Information of elevation dataset used for the void-filling processing)

    • ’stk’: number of DSM-scene files which were used to produce the 5m resolution DSM

    • ’Copernicus 10m EEA DEM’

    • ’dem’: the actual Digital Elevation Model

    • ’edm’: Editing Mask

    • ’flm’: Filling Mask

    • ’hem’: Height Error Mask

    • ’wbm’: Water Body Mask

    • ’Copernicus 30m Global DEM’

    • ’dem’: the actual Digital Elevation Model

    • ’edm’: Editing Mask

    • ’flm’: Filling Mask

    • ’hem’: Height Error Mask

    • ’wbm’: Water Body Mask

    • ’Copernicus 30m Global DEM II’

    • ’dem’: the actual Digital Elevation Model

    • ’edm’: Editing Mask

    • ’flm’: Filling Mask

    • ’hem’: Height Error Mask

    • ’wbm’: Water Body Mask

    • ’Copernicus 90m Global DEM’

    • ’dem’: the actual Digital Elevation Model

    • ’edm’: Editing Mask

    • ’flm’: Filling Mask

    • ’hem’: Height Error Mask

    • ’wbm’: Water Body Mask

    • ’Copernicus 90m Global DEM II’

    • ’dem’: the actual Digital Elevation Model

    • ’edm’: Editing Mask

    • ’flm’: Filling Mask

    • ’hem’: Height Error Mask

    • ’wbm’: Water Body Mask

    • ’GETASSE30’

    • ’dem’: the actual Digital Elevation Model

    • ’SRTM 1Sec HGT’

    • ’dem’: the actual Digital Elevation Model

    • ’SRTM 3Sec’

    • ’dem’: the actual Digital Elevation Model

    • ’TDX90m’

    • ’dem’: the actual Digital Elevation Model

    • ’am2’: Amplitude Mosaic representing the minimum value

    • ’amp’: Amplitude Mosaic representing the mean value

    • ’com’: Consistency Mask

    • ’cov’: Coverage Map

    • ’hem’: Height Error Map

    • ’lsm’: Layover and Shadow Mask, based on SRTM C-band and Globe DEM data

    • ’wam’: Water Indication Mask

  • crop (bool) – If a VRT is created, crop it to the spatial extent of the provided geometries or return the full extent of the DEM tiles? In the latter case, the common bounding box of the geometries is expanded so that the coordinates are multiples of the tile size of the respective DEM option.

  • lock_timeout (int) – How long to wait to acquire a lock on the downloaded files?

  • offline (bool) – Work offline? If True, only locally existing files are considered and no online check is performed. If a file is missing, an error is raised. For this to work, the function needs to be run in online mode once to create a local index.

  • return_fname (bool) – Return the file name including GDAL VSI directive (or just the path to the downloaded product)? E.g. /vsizip/srtm_72_02.zip/srtm_72_02.tif vs. /srtm_72_02.zip. Only applies if vrt=None.

Return type:

list[str] | None

Returns:

The names of the obtained files or None if a VRT file was defined with vrt.

remote_ids(dem_type, product='dem', username=None, password=None)[source]

Parse the names/URLs of the needed product files.

Parameters:
  • dem_type (str) – The type of DEM to be used.

  • product (str) – The sub-product to extract from the DEM product.

  • username (str | None) – The download account username.

  • password (str | None) – The download account password.

Return type:

list[str]

Returns:

The sorted product names.

class pyroSAR.auxdata.ImplicitFTP_TLS(*args, **kwargs)[source]

Bases: FTP_TLS

FTP_TLS subclass that automatically wraps sockets in SSL to support implicit FTPS. taken from https://stackoverflow.com/a/36049814

property sock: SSLSocket | None

Return the socket.

Setter accepts:

socket | SSLSocket | None

pyroSAR.auxdata.dem_autoload(vectorobject, demType, vrt=None, buffer=None, username=None, password=None, product='dem', crop=True, lock_timeout=600, offline=False, return_fname=True)[source]

Obtain all relevant DEM tiles for selected geometries. The tiles are optionally mosaicked into a VRT file. Otherwise, the tiles are returned as file path list. With return_fname it can be controlled whether the zip file path or a path inside the zip including GDAL VSI directive is returned.

Parameters:
  • vectorobject (Vector | None) – A spatialist.vector.Vector object to obtain DEM data for. CRS must be WGS84 LatLon (EPSG 4326). Can be set to None for global extent.

  • demType (str) –

    The type of DEM to be used. Options:

  • vrt (str | None) –

    An optional GDAL VRT file created from the obtained DEM tiles. Setting this to None lets the function return the file paths.

    Note

    VRTs do not support antimeridian splitting. For this, return the file paths and pass them to dem_create() directly.

    Note

    If the geometries are entirely over ocean it might be possible that no DEM tiles are found. In this case a sidecar file is created, which covers the geometries’ extent and contains one pixel with a value of 0 (Naming: vrt.replace('.vrt', '_tmp.tif')). This file is then used as source in the VRT.

  • buffer (int | float | None) – A buffer in degrees to add around the vector object’s extent.

  • username (str | None) – (optional) The username for services requiring registration.

  • password (str | None) – (optional) The password for the registration account.

  • product (str) –

    The sub-product to extract from the DEM product. The following options are available for the respective DEM types:

    • ’AW3D30’

      • ’dem’: the actual Digital Elevation Model

      • ’msk’: mask information for each pixel (Cloud/Snow Mask, Land water and low correlation mask, Sea mask, Information of elevation dataset used for the void-filling processing)

      • ’stk’: number of DSM-scene files which were used to produce the 5 m resolution DSM

    • ’Copernicus 10m EEA DEM’

      • ’dem’: the actual Digital Elevation Model

      • ’edm’: editing mask

      • ’flm’: filling mask

      • ’hem’: height error mask

      • ’wbm’: water body mask

    • ’Copernicus 30m Global DEM’

      • ’dem’: the actual Digital Elevation Model

      • ’edm’: Editing Mask

      • ’flm’: Filling Mask

      • ’hem’: Height Error Mask

      • ’wbm’: Water Body Mask

    • ’Copernicus 30m Global DEM II’

      • ’dem’: the actual Digital Elevation Model

      • ’edm’: editing mask

      • ’flm’: filling mask

      • ’hem’: height error mask

      • ’wbm’: water body mask

    • ’Copernicus 90m Global DEM’

      • ’dem’: the actual Digital Elevation Model

      • ’edm’: Editing Mask

      • ’flm’: Filling Mask

      • ’hem’: Height Error Mask

      • ’wbm’: Water Body Mask

    • ’Copernicus 90m Global DEM II’

      • ’dem’: the actual Digital Elevation Model

      • ’edm’: editing mask

      • ’flm’: filling mask

      • ’hem’: height error mask

      • ’wbm’: water body mask

    • ’GETASSE30’

      • ’dem’: the actual Digital Elevation Model

    • ’SRTM 1Sec HGT’

      • ’dem’: the actual Digital Elevation Model

    • ’SRTM 3Sec’

      • ’dem’: the actual Digital Elevation Model

  • crop (bool) – Crop to the provided geometries (or return the full extent of the DEM tiles)? Only applies if vrt!=None.

  • lock_timeout (int) – How long to wait to acquire a lock on the downloaded files?

  • offline (bool) – Work offline? If True, only locally existing files are considered and no online check is performed. If a file is missing, an error is raised. For this to work, the function needs to be run in online mode once to create a local index.

  • return_fname (bool) – Return the file name including GDAL VSI directive (or just the path to the downloaded product)? E.g. /vsizip/srtm_72_02.zip/srtm_72_02.tif vs. /srtm_72_02.zip. Only applies if vrt=None.

Return type:

list[str] | None

Returns:

The names of the obtained files or None if a VRT file was defined.

Examples

Download all SRTM 1 arcsec DEMs overlapping with a Sentinel-1 scene and mosaic them to a single GeoTIFF file:

from pyroSAR import identify
from pyroSAR.auxdata import dem_autoload
from spatialist import gdalwarp

# identify the SAR scene
filename = 'S1A_IW_SLC__1SDV_20150330T170734_20150330T170801_005264_006A6C_DA69.zip'
scene = identify(filename)

# extract the bounding box as spatialist.Vector object
bbox = scene.bbox()

# download the tiles and virtually combine them in an in-memory
# VRT file subsetted to the extent of the SAR scene plus a buffer of 0.01 degrees
vrt = '/vsimem/srtm1.vrt'
dem_autoload(vectorobject=bbox, demType='SRTM 1Sec HGT',
             vrt=vrt, buffer=0.01)

# write the final GeoTIFF file
outname = scene.outname_base() + 'srtm1.tif'
gdalwarp(src=vrt, dst=outname, options={'format': 'GTiff'})

# alternatively use function dem_create and warp the DEM to UTM
# including conversion from geoid to ellipsoid heights
from pyroSAR.auxdata import dem_create
outname = scene.outname_base() + 'srtm1_ellp.tif'
dem_create(vectorobject=bbox, src=vrt, dst=outname, t_srs=32632,
           tr=(30, 30), geoid_convert=True)

# close the Vector object
bbox.close()
pyroSAR.auxdata.dem_create(src, dst, vectorobject=None, buffer=None, t_srs=None, tr=None, threads=None, geoid_convert=True, nodata=None, resampleAlg=None, dtype=None, pbar=False, **kwargs)[source]

Create a new DEM GeoTIFF file and optionally convert heights from geoid to ellipsoid. This is basically a convenience wrapper around osgeo.gdal.Warp() via spatialist.auxil.gdalwarp().

Parameters:
  • src (str | list[str]) – The input dataset(s) as returned by dem_autoload().

  • dst (str) – The output GeoTIFF file name.

  • vectorobject (Vector | None) – A spatialist.vector.Vector geometry object to define the output extent. CRS must be WGS84 LatLon (EPSG 4326). Default None: use the extent of src.

  • buffer (int | float | None) – A buffer in degrees to add around the vector object’s extent.

  • t_srs (int | str | SpatialReference | None) – The target geographic reference system. See function spatialist.auxil.crsConvert() for details. Default None: use the crs of src.

  • tr (tuple[int | float, int | float] | None) – The target resolution as (xres, yres). Default None: use the resolution of src.

  • threads (int | str | None) –

    The number of threads to use. Possible values:

    • Default None: use the value of GDAL_NUM_THREADS without modification. If GDAL_NUM_THREADS is None, multi-threading is still turned on, and two threads are used, one for I/O and one for computation.

    • integer value: temporarily modify GDAL_NUM_THREADS and reset it once done. If 1, multithreading is turned off.

    • ALL_CPUS: special string to use all cores/CPUs of the computer; will also temporarily modify GDAL_NUM_THREADS.

  • geoid_convert (bool) – Convert geoid heights to WGS84 ellipsoid heights? Only applied if the vertical datum isn’t already WGS84 and product='dem'. The geoid model is inferred from demType. See dem_autoload() for details.

  • nodata (int | float | str | None) –

    the nodata value of dst. Default None: use these values per data type:

    • Byte: 255

    • Int16: -32768

    • Float32: -9999.0

  • resampleAlg (str | None) – The resampling algorithm to be used. See here for options: https://gdal.org/programs/gdalwarp.html#cmdoption-gdalwarp-r. Default None: use mode if the data type of src is Byte (for categorical value masks) and bilinear (for DEM and floating point error masks) otherwise.

  • dtype (str | None) – The data type of dst. Default None: use the value of the source file(s). Data type notations of GDAL (e.g. Float32) and numpy (e.g. int8) are supported. See spatialist.raster.Dtype.

  • pbar (bool) – Add a progressbar?

  • **kwargs –

    Additional keyword arguments to be passed to spatialist.auxil.gdalwarp(). See osgeo.gdal.WarpOptions() for options. The following arguments cannot be set as they are controlled internally:

    • xRes, yRes: controlled via argument tr

    • srcSRS, dstSRS: controlled via arguments t_srs and geoid_convert

    • srcNodata: controlled via nodata value of src

    • dstNodata: controlled via argument nodata

    • outputType: controlled via argument dtype

    • multithread: controlled via argument threads

    • targetAlignedPixels: fixed to True

    • warpOptions: currently used for setting the number of threads. Can be exposed if necessary.

    The following arguments are set if they are not defined in kwargs:

    • outputBounds: set to the common extent of geometries projected to t_srs if isinstance(src, list).

    • format: set to GTiff

Return type:

None

pyroSAR.auxdata.get_dem_options(require_auth=None)[source]

Get the names of all supported DEM type options.

Parameters:

require_auth (bool | None) – Only return options that do/don’t require authentication. Default None: return all options.

Return type:

list[str]

Returns:

The names of the DEM options.

pyroSAR.auxdata.get_egm_lookup(geoid, software)[source]

Download lookup tables for converting EGM geoid heights to WGS84 ellipsoid heights.

Parameters:
Return type:

None

pyroSAR.auxdata.getasse30_hdr(fname)[source]

Create an ENVI HDR file for zipped GETASSE30 DEM tiles.

Parameters:

fname (str) – The name of the zipped tile.

Return type:

None

pyroSAR.auxdata.vrt_check_sources(fname)[source]

Check the sanity of all source files of a given VRT.

Parameters:

fname (str) – The VRT file name.

Return type:

list[str]

Returns:

The source file names.

Raises:

RuntimeError –