Auxiliary Data Tools
Obtain all relevant DEM tiles for selected geometries. |
|
Create a new DEM GeoTIFF file and optionally convert heights from geoid to ellipsoid. |
|
An interface to obtain DEM data for selected geometries. |
|
Get the names of all supported DEM type options. |
|
Download lookup tables for converting EGM geoid heights to WGS84 ellipsoid heights. |
|
Create an ENVI HDR file for zipped GETASSE30 DEM tiles. |
|
FTP_TLS subclass that automatically wraps sockets in SSL to support implicit FTPS. |
|
Check the sanity of all source files of a given VRT. |
- class pyroSAR.auxdata.DEMHandler(vectorobject=None, buffer=None)[source]
Bases:
objectAn interface to obtain DEM data for selected geometries. The files are downloaded into the ESA SNAP auxiliary data directory structure. This class is the foundation for the convenience functions
dem_autoload()anddem_create().- Parameters:
- create(src, dst, t_srs=None, tr=None, threads=None, geoid_convert=True, nodata=None, resampleAlg=None, dtype=None, pbar=False, **kwargs)[source]
Create a new DEM GeoTIFF file and optionally convert heights from geoid to ellipsoid. This is basically a convenience wrapper around
osgeo.gdal.Warp()viaspatialist.auxil.gdalwarp().If
srcis a list of file names, the extent of the output DEM is determined by the DEMHandler argumentvectorobjectand the **kwargs argument outputBounds (passed togdal.Warp). If neithervectorobjectnor outputBounds are set, the bounding box extent of all input DEM tiles is used (reprojected to t_srs if necessary). IF both are provided,outputBoundsis prioritized because it is in coordinates oft_srsand thus more accurate.- Parameters:
dem_type – The input DEM product type. See
dem_autoload()for options.product – The input DEM sub-product type. See
dem_autoload()for options.src (
str|list[str]) – The input dataset(s) as returned bydem_autoload(). A string is expected to point to a VRT file. A list is interpreted as 1..n GDAL-readable datasets.dem_autoload()must be run with return_fnames=True to provide the correct format.dst (
str) – The output GeoTIFF file name.t_srs (
int|str|SpatialReference|None) – The target geographic reference system. See functionspatialist.auxil.crsConvert()for details. DefaultNone: use the crs ofsrc.tr (
tuple[int|float,int|float] |None) – The target resolution as (xres, yres). DefaultNone: use the resolution ofsrc.The number of threads to use. Possible values:
Default
None: use the value ofGDAL_NUM_THREADSwithout modification. IfGDAL_NUM_THREADSis None, multi-threading is still turned on and two threads are used, one for I/O and one for computation.integer value: temporarily modify
GDAL_NUM_THREADSand reset it once done. If 1, multithreading is turned off.ALL_CPUS: special string to use all cores/CPUs of the computer; will also temporarily modifyGDAL_NUM_THREADS.
geoid_convert (
bool) – Convert geoid heights to WGS84 ellipsoid heights? Only applied if the vertical datum isn’t already WGS84 andproduct='dem'. The geoid model is inferred fromdemType. Seedem_autoload()for details.nodata (
int|float|str|None) –The nodata value of dst. Default
None: use these values per data type:Byte: 255Int16: -32768Float32: -9999.0
resampleAlg (
str|None) – The resampling algorithm to be used. See here for options: https://gdal.org/programs/gdalwarp.html#cmdoption-gdalwarp-r. DefaultNone: usemodeif the data type ofsrcisByte(for categorical value masks) andbilinear(for DEM and floating point error masks) otherwise.dtype (
str|None) – The data type ofdst. DefaultNone: use the value of the source file(s). Data type notations of GDAL (e.g.Float32) and numpy (e.g.int8) are supported. Seespatialist.raster.Dtype.pbar (
bool) – Add a progressbar?**kwargs –
Additional keyword arguments to be passed to
spatialist.auxil.gdalwarp(). Seeosgeo.gdal.WarpOptions()for options. The following arguments cannot be set as they are controlled internally:xRes,yRes: controlled via argumenttrsrcSRS,dstSRS: controlled via argumentst_srsandgeoid_convertsrcNodata: controlled via nodata value ofsrcdstNodata: controlled via argumentnodataoutputType: controlled via argumentdtypemultithread: controlled via argumentthreadstargetAlignedPixels: fixed toTruewarpOptions: currently used for setting the number of threads. Can be exposed if necessary.
The following arguments are set if they are not defined in
**kwargs:outputBounds: determined from user input. See above.format: set toGTiff
- Return type:
- intrange(step)[source]
Generate a sequence of integer coordinates marking the tie points of the individual DEM tiles.
- load(dem_type, vrt=None, username=None, password=None, product='dem', crop=True, lock_timeout=600, offline=False, return_fname=True)[source]
Download DEM tiles. The result is either returned in a list of file names or combined into a VRT mosaic. The VRT is cropped to the combined extent of the geometries, but the pixel grid of the source files is preserved and no resampling/shifting is applied.
- Parameters:
dem_type (
str) – The type fo DEM to be usedvrt (
str|None) – An optional GDAL VRT file created from the obtained DEM tiles. NOTE: VRTs are not suited for geometries crossing the antimeridian.product (
str) –The sub-product to extract from the DEM product. Options:
’AW3D30’
’dem’: the actual Digital Elevation Model
’msk’: mask information for each pixel (Cloud/Snow Mask, Land water and low correlation mask, Sea mask, Information of elevation dataset used for the void-filling processing)
’stk’: number of DSM-scene files which were used to produce the 5m resolution DSM
’Copernicus 10m EEA DEM’
’dem’: the actual Digital Elevation Model
’edm’: Editing Mask
’flm’: Filling Mask
’hem’: Height Error Mask
’wbm’: Water Body Mask
’Copernicus 30m Global DEM’
’dem’: the actual Digital Elevation Model
’edm’: Editing Mask
’flm’: Filling Mask
’hem’: Height Error Mask
’wbm’: Water Body Mask
’Copernicus 30m Global DEM II’
’dem’: the actual Digital Elevation Model
’edm’: Editing Mask
’flm’: Filling Mask
’hem’: Height Error Mask
’wbm’: Water Body Mask
’Copernicus 90m Global DEM’
’dem’: the actual Digital Elevation Model
’edm’: Editing Mask
’flm’: Filling Mask
’hem’: Height Error Mask
’wbm’: Water Body Mask
’Copernicus 90m Global DEM II’
’dem’: the actual Digital Elevation Model
’edm’: Editing Mask
’flm’: Filling Mask
’hem’: Height Error Mask
’wbm’: Water Body Mask
’GETASSE30’
’dem’: the actual Digital Elevation Model
’SRTM 1Sec HGT’
’dem’: the actual Digital Elevation Model
’SRTM 3Sec’
’dem’: the actual Digital Elevation Model
’TDX90m’
’dem’: the actual Digital Elevation Model
’am2’: Amplitude Mosaic representing the minimum value
’amp’: Amplitude Mosaic representing the mean value
’com’: Consistency Mask
’cov’: Coverage Map
’hem’: Height Error Map
’lsm’: Layover and Shadow Mask, based on SRTM C-band and Globe DEM data
’wam’: Water Indication Mask
crop (
bool) – If a VRT is created, crop it to the spatial extent of the provided geometries or return the full extent of the DEM tiles? In the latter case, the common bounding box of the geometries is expanded so that the coordinates are multiples of the tile size of the respective DEM option.lock_timeout (
int) – How long to wait to acquire a lock on the downloaded files?offline (
bool) – Work offline? If True, only locally existing files are considered and no online check is performed. If a file is missing, an error is raised. For this to work, the function needs to be run in online mode once to create a local index.return_fname (
bool) – Return the file name including GDAL VSI directive (or just the path to the downloaded product)? E.g. /vsizip/srtm_72_02.zip/srtm_72_02.tif vs. /srtm_72_02.zip. Only applies if vrt=None.
- Return type:
- Returns:
The names of the obtained files or None if a VRT file was defined with vrt.
- class pyroSAR.auxdata.ImplicitFTP_TLS(*args, **kwargs)[source]
Bases:
FTP_TLSFTP_TLS subclass that automatically wraps sockets in SSL to support implicit FTPS. taken from https://stackoverflow.com/a/36049814
- pyroSAR.auxdata.dem_autoload(vectorobject, demType, vrt=None, buffer=None, username=None, password=None, product='dem', crop=True, lock_timeout=600, offline=False, return_fname=True)[source]
Obtain all relevant DEM tiles for selected geometries. The tiles are optionally mosaicked into a VRT file. Otherwise, the tiles are returned as file path list. With return_fname it can be controlled whether the zip file path or a path inside the zip including GDAL VSI directive is returned.
- Parameters:
vectorobject (
Vector|None) – Aspatialist.vector.Vectorobject to obtain DEM data for. CRS must be WGS84 LatLon (EPSG 4326). Can be set to None for global extent.demType (
str) –The type of DEM to be used. Options:
’AW3D30’ (ALOS Global Digital Surface Model “ALOS World 3D - 30m”)
url: ftp://ftp.eorc.jaxa.jp/pub/ALOS/ext1/AW3D30/release_v1804
height reference: EGM96
’Copernicus 10m EEA DEM’ (Copernicus 10 m DEM available over EEA-39 countries)
registration: https://spacedata.copernicus.eu/web/cscda/data-access/registration
url: ftps://cdsdata.copernicus.eu/DEM-datasets/COP-DEM_EEA-10-DGED/2021_1
height reference: EGM2008
’Copernicus 30m Global DEM’
height reference: EGM2008
’Copernicus 30m Global DEM II’
registration: https://spacedata.copernicus.eu/web/cscda/data-access/registration
url: ftps://cdsdata.copernicus.eu/DEM-datasets/COP-DEM_GLO-30-DGED/2021_1
height reference: EGM2008
’Copernicus 90m Global DEM’
height reference: EGM2008
’Copernicus 90m Global DEM II’
registration: https://spacedata.copernicus.eu/web/cscda/data-access/registration
url: ftps://cdsdata.copernicus.eu/DEM-datasets/COP-DEM_GLO-90-DGED/2021_1
height reference: EGM2008
’GETASSE30’
info: https://seadas.gsfc.nasa.gov/help-8.1.0/desktop/GETASSE30ElevationModel.html
height reference: WGS84
’SRTM 1Sec HGT’
height reference: EGM96
’SRTM 3Sec’
height reference: EGM96
An optional GDAL VRT file created from the obtained DEM tiles. Setting this to None lets the function return the file paths.
Note
VRTs do not support antimeridian splitting. For this, return the file paths and pass them to
dem_create()directly.Note
If the geometries are entirely over ocean it might be possible that no DEM tiles are found. In this case a sidecar file is created, which covers the geometries’ extent and contains one pixel with a value of 0 (Naming:
vrt.replace('.vrt', '_tmp.tif')). This file is then used as source in the VRT.buffer (
int|float|None) – A buffer in degrees to add around the vector object’s extent.username (
str|None) – (optional) The username for services requiring registration.password (
str|None) – (optional) The password for the registration account.product (
str) –The sub-product to extract from the DEM product. The following options are available for the respective DEM types:
’AW3D30’
’dem’: the actual Digital Elevation Model
’msk’: mask information for each pixel (Cloud/Snow Mask, Land water and low correlation mask, Sea mask, Information of elevation dataset used for the void-filling processing)
’stk’: number of DSM-scene files which were used to produce the 5 m resolution DSM
’Copernicus 10m EEA DEM’
’dem’: the actual Digital Elevation Model
’edm’: editing mask
’flm’: filling mask
’hem’: height error mask
’wbm’: water body mask
’Copernicus 30m Global DEM’
’dem’: the actual Digital Elevation Model
’edm’: Editing Mask
’flm’: Filling Mask
’hem’: Height Error Mask
’wbm’: Water Body Mask
’Copernicus 30m Global DEM II’
’dem’: the actual Digital Elevation Model
’edm’: editing mask
’flm’: filling mask
’hem’: height error mask
’wbm’: water body mask
’Copernicus 90m Global DEM’
’dem’: the actual Digital Elevation Model
’edm’: Editing Mask
’flm’: Filling Mask
’hem’: Height Error Mask
’wbm’: Water Body Mask
’Copernicus 90m Global DEM II’
’dem’: the actual Digital Elevation Model
’edm’: editing mask
’flm’: filling mask
’hem’: height error mask
’wbm’: water body mask
’GETASSE30’
’dem’: the actual Digital Elevation Model
’SRTM 1Sec HGT’
’dem’: the actual Digital Elevation Model
’SRTM 3Sec’
’dem’: the actual Digital Elevation Model
crop (
bool) – Crop to the provided geometries (or return the full extent of the DEM tiles)? Only applies if vrt!=None.lock_timeout (
int) – How long to wait to acquire a lock on the downloaded files?offline (
bool) – Work offline? If True, only locally existing files are considered and no online check is performed. If a file is missing, an error is raised. For this to work, the function needs to be run in online mode once to create a local index.return_fname (
bool) – Return the file name including GDAL VSI directive (or just the path to the downloaded product)? E.g. /vsizip/srtm_72_02.zip/srtm_72_02.tif vs. /srtm_72_02.zip. Only applies if vrt=None.
- Return type:
- Returns:
The names of the obtained files or
Noneif a VRT file was defined.
Examples
Download all SRTM 1 arcsec DEMs overlapping with a Sentinel-1 scene and mosaic them to a single GeoTIFF file:
from pyroSAR import identify from pyroSAR.auxdata import dem_autoload from spatialist import gdalwarp # identify the SAR scene filename = 'S1A_IW_SLC__1SDV_20150330T170734_20150330T170801_005264_006A6C_DA69.zip' scene = identify(filename) # extract the bounding box as spatialist.Vector object bbox = scene.bbox() # download the tiles and virtually combine them in an in-memory # VRT file subsetted to the extent of the SAR scene plus a buffer of 0.01 degrees vrt = '/vsimem/srtm1.vrt' dem_autoload(vectorobject=bbox, demType='SRTM 1Sec HGT', vrt=vrt, buffer=0.01) # write the final GeoTIFF file outname = scene.outname_base() + 'srtm1.tif' gdalwarp(src=vrt, dst=outname, options={'format': 'GTiff'}) # alternatively use function dem_create and warp the DEM to UTM # including conversion from geoid to ellipsoid heights from pyroSAR.auxdata import dem_create outname = scene.outname_base() + 'srtm1_ellp.tif' dem_create(vectorobject=bbox, src=vrt, dst=outname, t_srs=32632, tr=(30, 30), geoid_convert=True) # close the Vector object bbox.close()
- pyroSAR.auxdata.dem_create(src, dst, vectorobject=None, buffer=None, t_srs=None, tr=None, threads=None, geoid_convert=True, nodata=None, resampleAlg=None, dtype=None, pbar=False, **kwargs)[source]
Create a new DEM GeoTIFF file and optionally convert heights from geoid to ellipsoid. This is basically a convenience wrapper around
osgeo.gdal.Warp()viaspatialist.auxil.gdalwarp().- Parameters:
src (
str|list[str]) – The input dataset(s) as returned bydem_autoload().dst (
str) – The output GeoTIFF file name.vectorobject (
Vector|None) – Aspatialist.vector.Vectorgeometry object to define the output extent. CRS must be WGS84 LatLon (EPSG 4326). DefaultNone: use the extent ofsrc.buffer (
int|float|None) – A buffer in degrees to add around the vector object’s extent.t_srs (
int|str|SpatialReference|None) – The target geographic reference system. See functionspatialist.auxil.crsConvert()for details. DefaultNone: use the crs ofsrc.tr (
tuple[int|float,int|float] |None) – The target resolution as (xres, yres). DefaultNone: use the resolution ofsrc.The number of threads to use. Possible values:
Default
None: use the value ofGDAL_NUM_THREADSwithout modification. IfGDAL_NUM_THREADSis None, multi-threading is still turned on, and two threads are used, one for I/O and one for computation.integer value: temporarily modify
GDAL_NUM_THREADSand reset it once done. If 1, multithreading is turned off.ALL_CPUS: special string to use all cores/CPUs of the computer; will also temporarily modifyGDAL_NUM_THREADS.
geoid_convert (
bool) – Convert geoid heights to WGS84 ellipsoid heights? Only applied if the vertical datum isn’t already WGS84 andproduct='dem'. The geoid model is inferred fromdemType. Seedem_autoload()for details.nodata (
int|float|str|None) –the nodata value of dst. Default
None: use these values per data type:Byte: 255Int16: -32768Float32: -9999.0
resampleAlg (
str|None) – The resampling algorithm to be used. See here for options: https://gdal.org/programs/gdalwarp.html#cmdoption-gdalwarp-r. DefaultNone: usemodeif the data type ofsrcisByte(for categorical value masks) andbilinear(for DEM and floating point error masks) otherwise.dtype (
str|None) – The data type ofdst. DefaultNone: use the value of the source file(s). Data type notations of GDAL (e.g.Float32) and numpy (e.g.int8) are supported. Seespatialist.raster.Dtype.pbar (
bool) – Add a progressbar?**kwargs –
Additional keyword arguments to be passed to
spatialist.auxil.gdalwarp(). Seeosgeo.gdal.WarpOptions()for options. The following arguments cannot be set as they are controlled internally:xRes,yRes: controlled via argumenttrsrcSRS,dstSRS: controlled via argumentst_srsandgeoid_convertsrcNodata: controlled via nodata value ofsrcdstNodata: controlled via argumentnodataoutputType: controlled via argumentdtypemultithread: controlled via argumentthreadstargetAlignedPixels: fixed toTruewarpOptions: currently used for setting the number of threads. Can be exposed if necessary.
The following arguments are set if they are not defined in kwargs:
outputBounds: set to the common extent ofgeometriesprojected tot_srsifisinstance(src, list).format: set toGTiff
- Return type:
- pyroSAR.auxdata.get_dem_options(require_auth=None)[source]
Get the names of all supported DEM type options.
- pyroSAR.auxdata.get_egm_lookup(geoid, software)[source]
Download lookup tables for converting EGM geoid heights to WGS84 ellipsoid heights.
- Parameters:
geoid (
Literal['EGM96','EGM2008']) –The geoid model; current options:
SNAP: ‘EGM96’
PROJ: ‘EGM96’, ‘EGM2008’
software (
Literal['PROJ','SNAP']) –The software for which to download the EGM lookup. Options:
PROJ: requires
PROJ_DATAorPROJ_LIBenvironment variable to be set as download directory; URLs:SNAP: default directory:
~/.snap/auxdata/dem/egm96; URL:
- Return type: