Drivers

This is the core module of package pyroSAR. It contains the drivers for the different SAR image formats and offers functionality for retrieving metadata, unpacking images, downloading ancillary files like DEMs and Orbit State Vector files as well as archiving scenes in a database. The ID class and its subclasses allow easy and standardized access to the metadata of images from different SAR sensors.

classes

ID	Abstract class for SAR metadata handlers
BEAM_DIMAP	Handler class for BEAM-DIMAP data
CEOS_PSR	Handler class for ALOS-PALSAR data in CEOS format
CEOS_ERS	Handler class for ERS data in CEOS format
EORC_PSR	Handler class for ALOS-2/PALSAR-2 data in EORC (Earth Observation Research Center) Path format
ESA	Handler class for SAR data in ESA format (Envisat ASAR, ERS-1/2)
SAFE	Handler class for Sentinel-1 data
TSX	Handler class for TerraSAR-X and TanDEM-X data
TDM	Handler class for TerraSAR-X and TanDEM-X experimental data

functions

identify	identify a SAR scene and return the appropriate metadata handler object
identify_many	wrapper function for returning metadata handlers of all valid scenes in a list, similar to function identify().
filter_processed	Filter a list of pyroSAR objects to those that have not yet been processed and stored in the defined directory.
getFileObj	Load a file in a SAR scene archive into a readable file object.
parse_date	this function gathers known time formats provided in the different SAR products and converts them to a common standard of the form YYYYMMDDTHHMMSS

class pyroSAR.drivers.BEAM_DIMAP(scene)

Bases: ID

Handler class for BEAM-DIMAP data

Sensors:

• SNAP supported sensors

scanMetadata()

scan SAR scenes for metadata attributes. The returned dictionary is registered as attribute meta by the class upon object initialization. This dictionary furthermore needs to return a set of standardized attribute keys, which are directly registered as object attributes.

Return type:

dict[str, Any]

Returns:

the derived attributes

unpack(directory, overwrite=False, exist_ok=False)

Unpack the SAR scene into a defined directory.

Parameters:

• directory (str) – the base directory into which the scene is unpacked

• overwrite (bool) – overwrite an existing unpacked scene?

• exist_ok (bool) – allow existing output files and do not create new ones?

Return type:

None

class pyroSAR.drivers.CEOS_ERS(scene)

Bases: ID

Handler class for ERS data in CEOS format

Sensors:

• ERS1

• ERS2

Reference:

ER-IS-EPO-GS-5902-3: Annex C. ERS SAR.SLC/SLC-I. CCT and EXABYTE (ESA 1998)

scanMetadata()

scan SAR scenes for metadata attributes. The returned dictionary is registered as attribute meta by the class upon object initialization. This dictionary furthermore needs to return a set of standardized attribute keys, which are directly registered as object attributes.

Return type:

dict[str, Any]

Returns:

the derived attributes

unpack(directory, overwrite=False, exist_ok=False)

Unpack the SAR scene into a defined directory.

Parameters:

• directory (str) – the base directory into which the scene is unpacked

• overwrite (bool) – overwrite an existing unpacked scene?

• exist_ok (bool) – allow existing output files and do not create new ones?

Return type:

None

class pyroSAR.drivers.CEOS_PSR(scene)

Bases: ID

Handler class for ALOS-PALSAR data in CEOS format

Sensors:

• PSR1

• PSR2

PALSAR-1:

References:

• NEB-01006: ALOS/PALSAR Level 1 Product Format Description (JAXA 2006)

• NEB-070062B: ALOS/PALSAR Level 1.1/1.5 Product Format Description (JAXA 2009)

Products / processing levels:

• 1.0

• 1.1

• 1.5

Acquisition modes:

• AB: [SP][HWDPC]

• A: supplemental remarks of the sensor type:

  • S: Wide observation mode

  • P: all other modes

• B: observation mode

  • H: Fine mode

  • W: ScanSAR mode

  • D: Direct downlink mode

  • P: Polarimetry mode

  • C: Calibration mode

PALSAR-2:

Reference:

ALOS-2/PALSAR-2 Level 1.1/1.5/2.1/3.1 CEOS SAR Product Format Description (JAXA 2014).

Products / processing levels:

• 1.0

• 1.1

• 1.5

Acquisition modes:

• SBS: Spotlight mode

• UBS: Ultra-fine mode Single polarization

• UBD: Ultra-fine mode Dual polarization

• HBS: High-sensitive mode Single polarization

• HBD: High-sensitive mode Dual polarization

• HBQ: High-sensitive mode Full (Quad.) polarimetry

• FBS: Fine mode Single polarization

• FBD: Fine mode Dual polarization

• FBQ: Fine mode Full (Quad.) polarimetry

• WBS: Scan SAR nominal [14MHz] mode Single polarization

• WBD: Scan SAR nominal [14MHz] mode Dual polarization

• WWS: Scan SAR nominal [28MHz] mode Single polarization

• WWD: Scan SAR nominal [28MHz] mode Dual polarization

• VBS: Scan SAR wide mode Single polarization

• VBD: Scan SAR wide mode Dual polarization

property led_filename: str

scanMetadata()

scan SAR scenes for metadata attributes. The returned dictionary is registered as attribute meta by the class upon object initialization. This dictionary furthermore needs to return a set of standardized attribute keys, which are directly registered as object attributes.

Return type:

dict[str, Any]

Returns:

the derived attributes

unpack(directory, overwrite=False, exist_ok=False)

Unpack the SAR scene into a defined directory.

Parameters:

• directory (str) – the base directory into which the scene is unpacked

• overwrite (bool) – overwrite an existing unpacked scene?

• exist_ok (bool) – allow existing output files and do not create new ones?

Return type:

None

class pyroSAR.drivers.EORC_PSR(scene)

Bases: ID

Handler class for ALOS-2/PALSAR-2 data in EORC (Earth Observation Research Center) Path format

Sensors:

• PALSAR-2

PALSAR-2:

Reference:

NDX-150019: ALOS-2/PALSAR-2 EORC Path Product Format Description (JAXA 2016)

Products / processing levels:

• 1.5

Acquisition modes:

• FBD: Fine mode Dual polarization

• WBD: Scan SAR nominal [14MHz] mode Dual polarization

property header_filename: str

scanMetadata()

scan SAR scenes for metadata attributes. The returned dictionary is registered as attribute meta by the class upon object initialization. This dictionary furthermore needs to return a set of standardized attribute keys, which are directly registered as object attributes.

Return type:

dict[str, Any]

Returns:

the derived attributes

unpack(directory, overwrite=False, exist_ok=False)

Unpack the SAR scene into a defined directory.

Parameters:

• directory (str) – the base directory into which the scene is unpacked

• overwrite (bool) – overwrite an existing unpacked scene?

• exist_ok (bool) – allow existing output files and do not create new ones?

Return type:

None

class pyroSAR.drivers.ESA(scene)

Bases: ID

Handler class for SAR data in ESA format (Envisat ASAR, ERS-1/2)

Sensors:

• ASAR

• ERS1

• ERS2

geo_grid(outname=None, driver=None, overwrite=True)

get the geo grid as vector geometry

Parameters:

• outname (str | None) – the name of the vector file to be written

• driver (str | None) – the output file format; needs to be defined if the format cannot be auto-detected from the filename extension

• overwrite (bool) – overwrite an existing vector file?

Return type:

Vector | None

Returns:

the vector object if outname is None, None otherwise

See also

spatialist.vector.Vector.write

scanMetadata()

scan SAR scenes for metadata attributes. The returned dictionary is registered as attribute meta by the class upon object initialization. This dictionary furthermore needs to return a set of standardized attribute keys, which are directly registered as object attributes.

Return type:

dict[str, Any]

Returns:

the derived attributes

unpack(directory, overwrite=False, exist_ok=False)

Unpack the SAR scene into a defined directory.

Parameters:

• directory (str) – the base directory into which the scene is unpacked

• overwrite (bool) – overwrite an existing unpacked scene?

• exist_ok (bool) – allow existing output files and do not create new ones?

Return type:

None

class pyroSAR.drivers.ID(metadict)

Bases: object

Abstract class for SAR metadata handlers

bbox(outname=None, driver=None, overwrite=True, buffer=None)

get the bounding box of a scene. The result is either returned as vector object or written to a file.

Parameters:

• outname (str | None) – the name of the vector file to be written

• driver (str | None) – the output file format; needs to be defined if the format cannot be auto-detected from the filename extension

• overwrite (bool) – overwrite an existing vector file?

• buffer (int | float | tuple[int | float, int | float] | None) – a buffer to add around coordinates. Default None: do not add a buffer. A tuple is interpreted as (x buffer, y buffer).

Return type:

Vector | None

Returns:

the vector object if outname is None and None otherwise

See also

spatialist.vector.Vector.bbox

property compression: Literal['zip', 'tar'] | None

check whether a scene is compressed into a tarfile or zipfile or not at all

Return type:

either ‘zip’, ‘tar’ or None

examine(include_folders=False)

check whether any items in the SAR scene structure (i.e. files/folders) match the regular expression pattern defined by the class. On success the item is registered in the object as attribute file.

Parameters:

include_folders (bool) – also match folder (or just files)?

Raises:

RuntimeError –

Return type:

None

export2dict()

Return the uuid and the metadata that is defined in self.locals as a dictionary

Return type:

dict[str, Any]

findfiles(pattern, include_folders=False)

find files in the scene archive, which match a pattern.

Parameters:

• pattern (str) – the regular expression to match

• include_folders (bool) – also match folders (or just files)?

Return type:

str | list[str]

Returns:

the matched file name(s)

See also

spatialist.ancillary.finder()

gdalinfo()

read metadata directly from the GDAL SAR image drivers

Return type:

dict[str, Any]

Returns:

the metadata attributes

geometry(outname=None, driver=None, overwrite=True)

get the footprint geometry of a scene either as a vector object or written to a file

Parameters:

• outname (str | None) – the name of the vector file to be written

• driver (str | None) – the output file format; needs to be defined if the format cannot be auto-detected from the filename extension

• overwrite (bool) – overwrite an existing vector file?

Return type:

Vector | None

Returns:

the vector object if outname is None, None otherwise

See also

spatialist.vector.Vector.write

getCorners()

Get the bounding box corner coordinates

Return type:

dict[Literal['xmin', 'xmax', 'ymin', 'ymax'], float]

Returns:

the corner coordinates as a dictionary with keys xmin, ymin, xmax, ymax

getFileObj(filename)

Load a file into a readable file object.

Parameters:

filename (str) – the name of a file in the scene archive, easiest to get with method findfiles()

Return type:

BytesIO

Returns:

a file pointer object

getGammaImages(directory=None)

list all files processed by GAMMA

Parameters:

directory (str | None) – the directory to be scanned; if left empty the object attribute gammadir is scanned

Return type:

list[str]

Returns:

the file names of the images processed by GAMMA

Raises:

RuntimeError –

getHGT()

get the names of all SRTM HGT tiles overlapping with the SAR scene

Return type:

list[str]

Returns:

names of the SRTM HGT tiles

is_processed(outdir, recursive=False)

check whether a scene has already been processed and stored in the defined output directory (and subdirectories if scanned recursively)

Parameters:

• outdir (str) – the directory to be checked

• recursive (bool) – also scan subdirectories for output?

Return type:

bool

Returns:

does an image matching the scene pattern exist?

outname_base(extensions=None)

parse a string containing basic information about the scene in standardized format. Currently, this id contains the sensor (4 digits), acquisition mode (4 digits), orbit (1 digit) and acquisition start time (15 digits)., e.g. S1A__IW___A_20150523T122350.

Parameters:

extensions (list[str] | None) – the names of additional parameters to append to the basename, e.g. ['orbitNumber_rel']

Return type:

str

Returns:

a standardized name unique to the scene

static parse_date(x)

this function gathers known time formats provided in the different SAR products and converts them to a common standard of the form YYYYMMDDTHHMMSS.

Parameters:

x (str | datetime) – the time stamp

Return type:

str

Returns:

the converted time stamp in format YYYYmmddTHHMMSS

abstractmethod quicklook(outname, format='kmz')

export a quick look image of the scene

Parameters:

• outname (str) – the name of the output file

• format (str) – the format of the file to write; currently only kmz is supported

Return type:

None

Examples

>>> from pyroSAR import identify
>>> scene = identify('S1A_IW_GRDH_1SDV_20180101T170648_20180101T170713_019964_021FFD_DA78.zip')
>>> scene.quicklook('S1A__IW___A_20180101T170648.kmz')

abstractmethod scanMetadata()

scan SAR scenes for metadata attributes. The returned dictionary is registered as attribute meta by the class upon object initialization. This dictionary furthermore needs to return a set of standardized attribute keys, which are directly registered as object attributes.

Return type:

dict[str, Any]

Returns:

the derived attributes

property start_dt: datetime

Return type:

the acquisition start time as timezone-aware datetime object

property stop_dt: datetime

Return type:

the acquisition stop time as timezone-aware datetime object

summary()

print the set of standardized scene metadata attributes

Return type:

None

abstractmethod unpack(directory, overwrite=False, exist_ok=False)

Unpack the SAR scene into a defined directory.

Parameters:

• directory (str) – the base directory into which the scene is unpacked

• overwrite (bool) – overwrite an existing unpacked scene?

• exist_ok (bool) – allow existing output files and do not create new ones?

Return type:

None

class pyroSAR.drivers.SAFE(scene)

Bases: ID

Handler class for Sentinel-1 data

Sensors:

• S1A

• S1B

• S1C

• S1D

References:

• S1-RS-MDA-52-7443 Sentinel-1 IPF Auxiliary Product Specification

• MPC-0243 Masking “No-value” Pixels on GRD Products generated by the Sentinel-1 ESA IPF

geo_grid(outname=None, driver=None, overwrite=True)

get the geo grid as vector geometry

Parameters:

• outname (str | None) – the name of the vector file to be written

• driver (str | None) – the output file format; needs to be defined if the format cannot be auto-detected from the filename extension

• overwrite (bool) – overwrite an existing vector file?

Return type:

Vector | None

Returns:

the vector object if outname is None, None otherwise

See also

spatialist.vector.Vector.write

getOSV(osvdir=None, osvType='POE', returnMatch=False, useLocal=True, timeout=300, url_option=1)

download Orbit State Vector files for the scene

Parameters:

• osvdir (str | None) – the directory of OSV files; subdirectories POEORB and RESORB are created automatically; if no directory is defined, the standard SNAP auxdata location is used

• osvType (Literal['POE', 'RES'] | list[Literal['POE', 'RES']]) – the type of orbit file either ‘POE’, ‘RES’ or a list of both; if both are selected, the best matching file will be retrieved. I.e., POE if available and RES otherwise

• returnMatch (bool) – return the best matching orbit file?

• useLocal (bool) – use locally existing files and do not search for files online if the right file has been found?

• timeout (int | float | tuple[int | float, int | float] | None) – the timeout in seconds for downloading OSV files as provided to requests.get()

• url_option (int) – the OSV download URL option; see pyroSAR.S1.OSV.catch() for options

Return type:

str | None

Returns:

the best matching OSV file if returnMatch is True or None otherwise

See also

pyroSAR.S1.OSV

quicklook(outname, format='kmz', na_transparent=True)

Write a quicklook file for the scene.

Parameters:

• outname (str) – the file to write

• format (Literal['kmz']) –

  the quicklook format. Currently supported options:

  • kmz

• na_transparent (bool) – make NA values transparent?

Return type:

None

removeGRDBorderNoise(method='pyroSAR')

mask out Sentinel-1 image border noise.

Parameters:

method (Literal['pyroSAR', 'ESA'] | str) –

the border noise removal method to be applied; one of the following:

• ’ESA’: the pure implementation as described by ESA

• ’pyroSAR’: the ESA method plus the custom pyroSAR refinement

See also

removeGRDBorderNoise()

Return type:

None

resolution()

Compute the mid-swath resolution of the Sentinel-1 product. For GRD products the resolution is expressed in ground range and in slant range otherwise.

References:

• https://sentinel.esa.int/web/sentinel/user-guides/sentinel-1-sar/resolutions/level-1-single-look-complex

• https://sentinel.esa.int/web/sentinel/user-guides/sentinel-1-sar/resolutions/level-1-ground-range-detected

• https://sentinel.esa.int/web/sentinel/user-guides/sentinel-1-sar/document-library/-/asset_publisher/1dO7RF5fJMbd/content/sentinel-1-product-definition

Return type:

tuple[float, float]

Returns:

the resolution as (range, azimuth)

scanMetadata()

scan SAR scenes for metadata attributes. The returned dictionary is registered as attribute meta by the class upon object initialization. This dictionary furthermore needs to return a set of standardized attribute keys, which are directly registered as object attributes.

Return type:

dict[str, Any]

Returns:

the derived attributes

unpack(directory, overwrite=False, exist_ok=False)

Unpack the SAR scene into a defined directory.

Parameters:

• directory (str) – the base directory into which the scene is unpacked

• overwrite (bool) – overwrite an existing unpacked scene?

• exist_ok (bool) – allow existing output files and do not create new ones?

Return type:

None

class pyroSAR.drivers.TDM(scene)

Bases: TSX

Handler class for TerraSAR-X and TanDEM-X experimental data

Sensors:

• TDM1

References:

• TD-GS-PS-3028 TanDEM-X Experimental Product Description

Acquisition modes:

• HS: High Resolution SpotLight

• SL: SpotLight

• SM: StripMap

Polarisation modes:

• Single (S): all acquisition modes

• Dual (D): High Resolution SpotLight (HS), SpotLight (SL) and StripMap (SM)

• Twin (T): StripMap (SM) (experimental)

• Quad (Q): StripMap (SM) (experimental)

Products:

• CoSSCs: (bi-static) SAR co-registered single look slant range complex products (CoSSCs)

Examples

Ingest all Tandem-X Bistatic scenes in a directory and its sub-directories into the database:

>>> from pyroSAR import Archive, identify
>>> from spatialist.ancillary import finder
>>> dbfile = '/.../scenelist.db'
>>> archive_tdm = '/.../TDM/'
>>> scenes_tdm = finder(archive_tdm, [r'^TDM1.*'], foldermode=2, regex=True, recursive=True)
>>> with Archive(dbfile) as archive:
>>>     archive.insert(scenes_tdm)

scanMetadata()

scan SAR scenes for metadata attributes. The returned dictionary is registered as attribute meta by the class upon object initialization. This dictionary furthermore needs to return a set of standardized attribute keys, which are directly registered as object attributes.

Return type:

dict[str, Any]

Returns:

the derived attributes

class pyroSAR.drivers.TSX(scene)

Bases: ID

Handler class for TerraSAR-X and TanDEM-X data

Sensors:

• TSX1

• TDX1

References:

• TX-GS-DD-3302 TerraSAR-X Basic Product Specification Document

• TX-GS-DD-3303 TerraSAR-X Experimental Product Description

• TD-GS-PS-3028 TanDEM-X Experimental Product Description

• TerraSAR-X Image Product Guide (Airbus Defence and Space)

Acquisition modes:

• ST: Staring Spotlight

• HS: High Resolution SpotLight

• HS300: High Resolution SpotLight 300 MHz

• SL: SpotLight

• SM: StripMap

• SC: ScanSAR

• WS: Wide ScanSAR

Polarisation modes:

• Single (S): all acquisition modes

• Dual (D): High Resolution SpotLight (HS), SpotLight (SL) and StripMap (SM)

• Twin (T): StripMap (SM) (experimental)

• Quad (Q): StripMap (SM) (experimental)

Products:

• SSC: Single Look Slant Range Complex

• MGD: Multi Look Ground Range Detected

• GEC: Geocoded Ellipsoid Corrected

• EEC: Enhanced Ellipsoid Corrected

scanMetadata()

scan SAR scenes for metadata attributes. The returned dictionary is registered as attribute meta by the class upon object initialization. This dictionary furthermore needs to return a set of standardized attribute keys, which are directly registered as object attributes.

Return type:

dict[str, Any]

Returns:

the derived attributes

unpack(directory, overwrite=False, exist_ok=False)

Unpack the SAR scene into a defined directory.

Parameters:

• directory (str) – the base directory into which the scene is unpacked

• overwrite (bool) – overwrite an existing unpacked scene?

• exist_ok (bool) – allow existing output files and do not create new ones?

Return type:

None

pyroSAR.drivers.filter_processed(scenelist, outdir, recursive=False)

Filter a list of pyroSAR objects to those that have not yet been processed and stored in the defined directory. The search for processed scenes is either done in the directory only or recursively into subdirectories. The scenes must have been processed with pyroSAR in order to follow the right naming scheme.

Parameters:

• scenelist (list[ID]) – a list of pyroSAR objects

• outdir (str) – the processing directory

• recursive (bool) – scan outdir recursively into subdirectories?

Return type:

list[ID]

Returns:

a list of those scenes, which have not been processed yet

pyroSAR.drivers.getFileObj(scene, filename)

Load a file in a SAR scene archive into a readable file object.

Parameters:

• scene (str) – the scene archive. Can be either a directory or a compressed archive of type zip or tar.gz.

• filename (str) – the name of a file in the scene archive, easiest to get with method findfiles()

Return type:

BytesIO

Returns:

a file object

pyroSAR.drivers.identify(scene)

identify a SAR scene and return the appropriate metadata handler object

Parameters:

scene (str) – a file or directory name

Return type:

ID

Returns:

a pyroSAR metadata handler

Examples

>>> from pyroSAR import identify
>>> filename = 'S1A_IW_GRDH_1SDV_20180829T170656_20180829T170721_023464_028DE0_F7BD.zip'
>>> scene = identify(filename)
>>> print(scene)
pyroSAR ID object of type SAFE
acquisition_mode: IW
cycleNumber: 148
frameNumber: 167392
lines: 16703
orbit: A
orbitNumber_abs: 23464
orbitNumber_rel: 117
polarizations: ['VV', 'VH']
product: GRD
projection: +proj=longlat +datum=WGS84 +no_defs
samples: 26056
sensor: S1A
spacing: (10.0, 10.0)
start: 20180829T170656
stop: 20180829T170721

pyroSAR.drivers.identify_many(scenes, pbar=False, sortkey=None, cores=1)

wrapper function for returning metadata handlers of all valid scenes in a list, similar to function identify().

Parameters:

• scenes (list[str | ID]) – the file names of the scenes to be identified

• pbar (bool) – adds a progressbar if True

• sortkey (str | None) – sort the handler object list by an attribute

• cores (int) – the number of cores to parallelize identification

Return type:

list[ID]

Returns:

a list of pyroSAR metadata handlers

Examples

>>> from pyroSAR import identify_many
>>> files = finder('/path', ['S1*.zip'])
>>> ids = identify_many(files, pbar=False, sortkey='start')

pyroSAR.drivers.parse_date(x)

this function gathers known time formats provided in the different SAR products and converts them to a common standard of the form YYYYMMDDTHHMMSS

Parameters:

x (str | datetime) – the time stamp to be converted

Return type:

str

Returns:

the converted time stamp in format YYYYmmddTHHMMSS