Drivers

This is the core module of package pyroSAR. It contains the drivers for the different SAR image formats and offers functionality for retrieving metadata, unpacking images, downloading ancillary files like DEMs and Orbit State Vector files as well as archiving scenes in a database. The ID class and its subclasses allow easy and standardized access to the metadata of images from different SAR sensors.

classes

ID	Abstract class for SAR meta data handlers
BEAM_DIMAP	Handler class for BEAM-DIMAP data
CEOS_PSR	Handler class for ALOS-PALSAR data in CEOS format
CEOS_ERS	Handler class for ERS data in CEOS format
EORC_PSR	Handler class for ALOS-2/PALSAR-2 data in EORC (Earth Observation Research Center) Path format
ESA	Handler class for SAR data in ESA format (Envisat ASAR, ERS-1/2)
SAFE	Handler class for Sentinel-1 data
TSX	Handler class for TerraSAR-X and TanDEM-X data
TDM	Handler class for TerraSAR-X and TanDEM-X experimental data

functions

identify	identify a SAR scene and return the appropriate metadata handler object
identify_many	wrapper function for returning metadata handlers of all valid scenes in a list, similar to function identify().
filter_processed	Filter a list of pyroSAR objects to those that have not yet been processed and stored in the defined directory.
getFileObj	Load a file in a SAR scene archive into a readable file object.
parse_date	this function gathers known time formats provided in the different SAR products and converts them to a common standard of the form YYYYMMDDTHHMMSS

class pyroSAR.drivers.BEAM_DIMAP(scene)

Bases: ID

Handler class for BEAM-DIMAP data

Sensors:

• SNAP supported sensors

scanMetadata()

scan SAR scenes for metadata attributes. The returned dictionary is registered as attribute meta by the class upon object initialization. This dictionary furthermore needs to return a set of standardized attribute keys, which are directly registered as object attributes.

Returns:

the derived attributes

Return type:

dict

unpack(directory, overwrite=False, exist_ok=False)

Unpack the SAR scene into a defined directory.

Parameters:

• directory (str) – the base directory into which the scene is unpacked

• overwrite (bool) – overwrite an existing unpacked scene?

• exist_ok (bool) – allow existing output files and do not create new ones?

class pyroSAR.drivers.CEOS_ERS(scene)

Bases: ID

Handler class for ERS data in CEOS format

Sensors:

• ERS1

• ERS2

Reference:

ER-IS-EPO-GS-5902-3: Annex C. ERS SAR.SLC/SLC-I. CCT and EXABYTE (ESA 1998)

scanMetadata()

scan SAR scenes for metadata attributes. The returned dictionary is registered as attribute meta by the class upon object initialization. This dictionary furthermore needs to return a set of standardized attribute keys, which are directly registered as object attributes.

Returns:

the derived attributes

Return type:

dict

unpack(directory, overwrite=False, exist_ok=False)

Unpack the SAR scene into a defined directory.

Parameters:

• directory (str) – the base directory into which the scene is unpacked

• overwrite (bool) – overwrite an existing unpacked scene?

• exist_ok (bool) – allow existing output files and do not create new ones?

class pyroSAR.drivers.CEOS_PSR(scene)

Bases: ID

Handler class for ALOS-PALSAR data in CEOS format

Sensors:

• PSR1

• PSR2

PALSAR-1:

References:

• NEB-01006: ALOS/PALSAR Level 1 Product Format Description (JAXA 2006)

• NEB-070062B: ALOS/PALSAR Level 1.1/1.5 Product Format Description (JAXA 2009)

Products / processing levels:

• 1.0

• 1.1

• 1.5

Acquisition modes:

• AB: [SP][HWDPC]

• A: supplemental remarks of the sensor type:

  • S: Wide observation mode

  • P: all other modes

• B: observation mode

  • H: Fine mode

  • W: ScanSAR mode

  • D: Direct downlink mode

  • P: Polarimetry mode

  • C: Calibration mode

PALSAR-2:

Reference:

ALOS-2/PALSAR-2 Level 1.1/1.5/2.1/3.1 CEOS SAR Product Format Description (JAXA 2014).

Products / processing levels:

• 1.0

• 1.1

• 1.5

Acquisition modes:

• SBS: Spotlight mode

• UBS: Ultra-fine mode Single polarization

• UBD: Ultra-fine mode Dual polarization

• HBS: High-sensitive mode Single polarization

• HBD: High-sensitive mode Dual polarization

• HBQ: High-sensitive mode Full (Quad.) polarimetry

• FBS: Fine mode Single polarization

• FBD: Fine mode Dual polarization

• FBQ: Fine mode Full (Quad.) polarimetry

• WBS: Scan SAR nominal [14MHz] mode Single polarization

• WBD: Scan SAR nominal [14MHz] mode Dual polarization

• WWS: Scan SAR nominal [28MHz] mode Single polarization

• WWD: Scan SAR nominal [28MHz] mode Dual polarization

• VBS: Scan SAR wide mode Single polarization

• VBD: Scan SAR wide mode Dual polarization

property led_filename

scanMetadata()

scan SAR scenes for metadata attributes. The returned dictionary is registered as attribute meta by the class upon object initialization. This dictionary furthermore needs to return a set of standardized attribute keys, which are directly registered as object attributes.

Returns:

the derived attributes

Return type:

dict

unpack(directory, overwrite=False, exist_ok=False)

Unpack the SAR scene into a defined directory.

Parameters:

• directory (str) – the base directory into which the scene is unpacked

• overwrite (bool) – overwrite an existing unpacked scene?

• exist_ok (bool) – allow existing output files and do not create new ones?

class pyroSAR.drivers.EORC_PSR(scene)

Bases: ID

Handler class for ALOS-2/PALSAR-2 data in EORC (Earth Observation Research Center) Path format

Sensors:

• PALSAR-2

PALSAR-2:

Reference:

NDX-150019: ALOS-2/PALSAR-2 EORC Path Product Format Description (JAXA 2016)

Products / processing levels:

• 1.5

Acquisition modes:

• FBD: Fine mode Dual polarization

• WBD: Scan SAR nominal [14MHz] mode Dual polarization

property header_filename

scanMetadata()

scan SAR scenes for metadata attributes. The returned dictionary is registered as attribute meta by the class upon object initialization. This dictionary furthermore needs to return a set of standardized attribute keys, which are directly registered as object attributes.

Returns:

the derived attributes

Return type:

dict

unpack(directory, overwrite=False, exist_ok=False)

Unpack the SAR scene into a defined directory.

Parameters:

• directory (str) – the base directory into which the scene is unpacked

• overwrite (bool) – overwrite an existing unpacked scene?

• exist_ok (bool) – allow existing output files and do not create new ones?

class pyroSAR.drivers.ESA(scene)

Bases: ID

Handler class for SAR data in ESA format (Envisat ASAR, ERS-1/2)

Sensors:

• ASAR

• ERS1

• ERS2

geo_grid(outname=None, driver=None, overwrite=True)

get the geo grid as vector geometry

Parameters:

• outname (str) – the name of the vector file to be written

• driver (str) – the output file format; needs to be defined if the format cannot be auto-detected from the filename extension

• overwrite (bool) – overwrite an existing vector file?

Returns:

the vector object if outname is None, None otherwise

Return type:

spatialist.vector.Vector or None

See also

spatialist.vector.Vector.write

scanMetadata()

scan SAR scenes for metadata attributes. The returned dictionary is registered as attribute meta by the class upon object initialization. This dictionary furthermore needs to return a set of standardized attribute keys, which are directly registered as object attributes.

Returns:

the derived attributes

Return type:

dict

unpack(directory, overwrite=False, exist_ok=False)

Unpack the SAR scene into a defined directory.

Parameters:

• directory (str) – the base directory into which the scene is unpacked

• overwrite (bool) – overwrite an existing unpacked scene?

• exist_ok (bool) – allow existing output files and do not create new ones?

class pyroSAR.drivers.ID(metadict)

Bases: object

Abstract class for SAR meta data handlers

bbox(outname=None, driver=None, overwrite=True, buffer=None)

get the bounding box of a scene. The result is either returned as vector object or written to a file.

Parameters:

• outname (str) – the name of the vector file to be written

• driver (str) – the output file format; needs to be defined if the format cannot be auto-detected from the filename extension

• overwrite (bool) – overwrite an existing vector file?

• buffer (None or int or float or tuple[int or float]) – a buffer to add around coordinates. Default None: do not add a buffer. A tuple is interpreted as (x buffer, y buffer).

Returns:

the vector object if outname is None and None otherwise

Return type:

Vector or None

See also

spatialist.vector.Vector.bbox

property compression

check whether a scene is compressed into an tarfile or zipfile or not at all

Returns:

either ‘zip’, ‘tar’ or None

Return type:

str or None

examine(include_folders=False)

check whether any items in the SAR scene structure (i.e. files/folders) match the regular expression pattern defined by the class. On success the item is registered in the object as attribute file.

Parameters:

include_folders (bool) – also match folder (or just files)?

Raises:

RuntimeError –

export2dict()

Return the uuid and the metadata that is defined in self.locals as a dictionary

findfiles(pattern, include_folders=False)

find files in the scene archive, which match a pattern.

Parameters:

• pattern (str) – the regular expression to match

• include_folders (bool) – also match folders (or just files)?

Returns:

the matched file names

Return type:

list[str]

See also

spatialist.ancillary.finder()

gdalinfo()

read metadata directly from the GDAL SAR image drivers

Returns:

the metadata attributes

Return type:

dict

geometry(outname=None, driver=None, overwrite=True)

get the footprint geometry of a scene either as a vector object or written to a file

Parameters:

• outname (str) – the name of the vector file to be written

• driver (str) – the output file format; needs to be defined if the format cannot be auto-detected from the filename extension

• overwrite (bool) – overwrite an existing vector file?

Returns:

the vector object if outname is None, None otherwise

Return type:

Vector or None

See also

spatialist.vector.Vector.write

getCorners()

Get the bounding box corner coordinates

Returns:

the corner coordinates as a dictionary with keys xmin, ymin, xmax, ymax

Return type:

dict

getFileObj(filename)

Load a file into a readable file object.

Parameters:

filename (str) – the name of a file in the scene archive, easiest to get with method findfiles()

Returns:

a file pointer object

Return type:

io.BytesIO

getGammaImages(directory=None)

list all files processed by GAMMA

Parameters:

directory (str or None) – the directory to be scanned; if left empty the object attribute gammadir is scanned

Returns:

the file names of the images processed by GAMMA

Return type:

list[str]

Raises:

RuntimeError –

getHGT()

get the names of all SRTM HGT tiles overlapping with the SAR scene

Returns:

names of the SRTM HGT tiles

Return type:

list[str]

is_processed(outdir, recursive=False)

check whether a scene has already been processed and stored in the defined output directory (and subdirectories if scanned recursively)

Parameters:

outdir (str) – the directory to be checked

Returns:

does an image matching the scene pattern exist?

Return type:

bool

outname_base(extensions=None)

parse a string containing basic information about the scene in standardized format. Currently, this id contains the sensor (4 digits), acquisition mode (4 digits), orbit (1 digit) and acquisition start time (15 digits)., e.g. S1A__IW___A_20150523T122350.

Parameters:

extensions (list[str] or None) – the names of additional parameters to append to the basename, e.g. ['orbitNumber_rel']

Returns:

a standardized name unique to the scene

Return type:

str

static parse_date(x)

this function gathers known time formats provided in the different SAR products and converts them to a common standard of the form YYYYMMDDTHHMMSS.

Parameters:

x (str) – the time stamp

Returns:

the converted time stamp in format YYYYmmddTHHMMSS

Return type:

str

abstractmethod quicklook(outname, format='kmz')

export a quick look image of the scene

Parameters:

• outname (str) – the name of the output file

• format (str) – the format of the file to write; currently only kmz is supported

Examples

>>> from pyroSAR import identify
>>> scene = identify('S1A_IW_GRDH_1SDV_20180101T170648_20180101T170713_019964_021FFD_DA78.zip')
>>> scene.quicklook('S1A__IW___A_20180101T170648.kmz')

abstractmethod scanMetadata()

scan SAR scenes for metadata attributes. The returned dictionary is registered as attribute meta by the class upon object initialization. This dictionary furthermore needs to return a set of standardized attribute keys, which are directly registered as object attributes.

Returns:

the derived attributes

Return type:

dict

property start_dt: datetime

Return type:

the acquisition start time as timezone-aware datetime object

property stop_dt: datetime

Return type:

the acquisition stop time as timezone-aware datetime object

summary()

print the set of standardized scene metadata attributes

abstractmethod unpack(directory, overwrite=False, exist_ok=False)

Unpack the SAR scene into a defined directory.

Parameters:

• directory (str) – the base directory into which the scene is unpacked

• overwrite (bool) – overwrite an existing unpacked scene?

• exist_ok (bool) – allow existing output files and do not create new ones?

class pyroSAR.drivers.SAFE(scene)

Bases: ID

Handler class for Sentinel-1 data

Sensors:

• S1A

• S1B

• S1C

• S1D

References:

• S1-RS-MDA-52-7443 Sentinel-1 IPF Auxiliary Product Specification

• MPC-0243 Masking “No-value” Pixels on GRD Products generated by the Sentinel-1 ESA IPF

geo_grid(outname=None, driver=None, overwrite=True)

get the geo grid as vector geometry

Parameters:

• outname (str) – the name of the vector file to be written

• driver (str) – the output file format; needs to be defined if the format cannot be auto-detected from the filename extension

• overwrite (bool) – overwrite an existing vector file?

Returns:

the vector object if outname is None, None otherwise

Return type:

Vector or None

See also

spatialist.vector.Vector.write

getOSV(osvdir=None, osvType='POE', returnMatch=False, useLocal=True, timeout=300, url_option=1)

download Orbit State Vector files for the scene

Parameters:

• osvdir (str) – the directory of OSV files; subdirectories POEORB and RESORB are created automatically; if no directory is defined, the standard SNAP auxdata location is used

• osvType (str or list[str]) – the type of orbit file either ‘POE’, ‘RES’ or a list of both; if both are selected, the best matching file will be retrieved. I.e., POE if available and RES otherwise

• returnMatch (bool) – return the best matching orbit file?

• useLocal (bool) – use locally existing files and do not search for files online if the right file has been found?

• timeout (int or tuple or None) – the timeout in seconds for downloading OSV files as provided to requests.get()

• url_option (int) – the OSV download URL option; see pyroSAR.S1.OSV.catch() for options

Returns:

the best matching OSV file if returnMatch is True or None otherwise

Return type:

str or None

See also

pyroSAR.S1.OSV

quicklook(outname, format='kmz', na_transparent=True)

Write a quicklook file for the scene.

Parameters:

• outname (str) – the file to write

• format (str) –

  the quicklook format. Currently supported options:

  • kmz

• na_transparent (bool) – make NA values transparent?

removeGRDBorderNoise(method='pyroSAR')

mask out Sentinel-1 image border noise.

Parameters:

method (str) –

the border noise removal method to be applied; one of the following:

• ’ESA’: the pure implementation as described by ESA

• ’pyroSAR’: the ESA method plus the custom pyroSAR refinement

See also

removeGRDBorderNoise()

resolution()

Compute the mid-swath resolution of the Sentinel-1 product. For GRD products the resolution is expressed in ground range and in slant range otherwise.

References:

• https://sentinel.esa.int/web/sentinel/user-guides/sentinel-1-sar/resolutions/level-1-single-look-complex

• https://sentinel.esa.int/web/sentinel/user-guides/sentinel-1-sar/resolutions/level-1-ground-range-detected

• https://sentinel.esa.int/web/sentinel/user-guides/sentinel-1-sar/document-library/-/asset_publisher/1dO7RF5fJMbd/content/sentinel-1-product-definition

Returns:

the resolution as (range, azimuth)

Return type:

tuple[float]

scanMetadata()

scan SAR scenes for metadata attributes. The returned dictionary is registered as attribute meta by the class upon object initialization. This dictionary furthermore needs to return a set of standardized attribute keys, which are directly registered as object attributes.

Returns:

the derived attributes

Return type:

dict

unpack(directory, overwrite=False, exist_ok=False)

Unpack the SAR scene into a defined directory.

Parameters:

• directory (str) – the base directory into which the scene is unpacked

• overwrite (bool) – overwrite an existing unpacked scene?

• exist_ok (bool) – allow existing output files and do not create new ones?

class pyroSAR.drivers.TDM(scene)

Bases: TSX

Handler class for TerraSAR-X and TanDEM-X experimental data

Sensors:

• TDM1

References:

• TD-GS-PS-3028 TanDEM-X Experimental Product Description

Acquisition modes:

• HS: High Resolution SpotLight

• SL: SpotLight

• SM: StripMap

Polarisation modes:

• Single (S): all acquisition modes

• Dual (D): High Resolution SpotLight (HS), SpotLight (SL) and StripMap (SM)

• Twin (T): StripMap (SM) (experimental)

• Quad (Q): StripMap (SM) (experimental)

Products:

• CoSSCs: (bi-static) SAR co-registered single look slant range complex products (CoSSCs)

Examples

Ingest all Tandem-X Bistatic scenes in a directory and its sub-directories into the database:

>>> from pyroSAR import Archive, identify
>>> from spatialist.ancillary import finder
>>> dbfile = '/.../scenelist.db'
>>> archive_tdm = '/.../TDM/'
>>> scenes_tdm = finder(archive_tdm, [r'^TDM1.*'], foldermode=2, regex=True, recursive=True)
>>> with Archive(dbfile) as archive:
>>>     archive.insert(scenes_tdm)

scanMetadata()

scan SAR scenes for metadata attributes. The returned dictionary is registered as attribute meta by the class upon object initialization. This dictionary furthermore needs to return a set of standardized attribute keys, which are directly registered as object attributes.

Returns:

the derived attributes

Return type:

dict

class pyroSAR.drivers.TSX(scene)

Bases: ID

Handler class for TerraSAR-X and TanDEM-X data

Sensors:

• TSX1

• TDX1

References:

• TX-GS-DD-3302 TerraSAR-X Basic Product Specification Document

• TX-GS-DD-3303 TerraSAR-X Experimental Product Description

• TD-GS-PS-3028 TanDEM-X Experimental Product Description

• TerraSAR-X Image Product Guide (Airbus Defence and Space)

Acquisition modes:

• ST: Staring Spotlight

• HS: High Resolution SpotLight

• HS300: High Resolution SpotLight 300 MHz

• SL: SpotLight

• SM: StripMap

• SC: ScanSAR

• WS: Wide ScanSAR

Polarisation modes:

• Single (S): all acquisition modes

• Dual (D): High Resolution SpotLight (HS), SpotLight (SL) and StripMap (SM)

• Twin (T): StripMap (SM) (experimental)

• Quad (Q): StripMap (SM) (experimental)

Products:

• SSC: Single Look Slant Range Complex

• MGD: Multi Look Ground Range Detected

• GEC: Geocoded Ellipsoid Corrected

• EEC: Enhanced Ellipsoid Corrected

scanMetadata()

scan SAR scenes for metadata attributes. The returned dictionary is registered as attribute meta by the class upon object initialization. This dictionary furthermore needs to return a set of standardized attribute keys, which are directly registered as object attributes.

Returns:

the derived attributes

Return type:

dict

unpack(directory, overwrite=False, exist_ok=False)

Unpack the SAR scene into a defined directory.

Parameters:

• directory (str) – the base directory into which the scene is unpacked

• overwrite (bool) – overwrite an existing unpacked scene?

• exist_ok (bool) – allow existing output files and do not create new ones?

pyroSAR.drivers.filter_processed(scenelist, outdir, recursive=False)

Filter a list of pyroSAR objects to those that have not yet been processed and stored in the defined directory. The search for processed scenes is either done in the directory only or recursively into subdirectories. The scenes must have been processed with pyroSAR in order to follow the right naming scheme.

Parameters:

• scenelist (list[ID]) – a list of pyroSAR objects

• outdir (str) – the processing directory

• recursive (bool) – scan outdir recursively into subdirectories?

Returns:

a list of those scenes, which have not been processed yet

Return type:

list[ID]

pyroSAR.drivers.getFileObj(scene, filename)

Load a file in a SAR scene archive into a readable file object.

Parameters:

• scene (str) – the scene archive. Can be either a directory or a compressed archive of type zip or tar.gz.

• filename (str) – the name of a file in the scene archive, easiest to get with method findfiles()

Returns:

a file object

Return type:

BytesIO

pyroSAR.drivers.identify(scene)

identify a SAR scene and return the appropriate metadata handler object

Parameters:

scene (str) – a file or directory name

Returns:

a pyroSAR metadata handler

Return type:

pyroSAR.drivers.ID

Examples

>>> from pyroSAR import identify
>>> filename = 'S1A_IW_GRDH_1SDV_20180829T170656_20180829T170721_023464_028DE0_F7BD.zip'
>>> scene = identify(filename)
>>> print(scene)
pyroSAR ID object of type SAFE
acquisition_mode: IW
cycleNumber: 148
frameNumber: 167392
lines: 16703
orbit: A
orbitNumber_abs: 23464
orbitNumber_rel: 117
polarizations: ['VV', 'VH']
product: GRD
projection: +proj=longlat +datum=WGS84 +no_defs
samples: 26056
sensor: S1A
spacing: (10.0, 10.0)
start: 20180829T170656
stop: 20180829T170721

pyroSAR.drivers.identify_many(scenes, pbar=False, sortkey=None, cores=1)

wrapper function for returning metadata handlers of all valid scenes in a list, similar to function identify().

Parameters:

• scenes (list[str or ID]) – the file names of the scenes to be identified

• pbar (bool) – adds a progressbar if True

• sortkey (str or None) – sort the handler object list by an attribute

• cores (int) – the number of cores to parallelize identification

Returns:

a list of pyroSAR metadata handlers

Return type:

list[ID]

Examples

>>> from pyroSAR import identify_many
>>> files = finder('/path', ['S1*.zip'])
>>> ids = identify_many(files, pbar=False, sortkey='start')

pyroSAR.drivers.parse_date(x)

this function gathers known time formats provided in the different SAR products and converts them to a common standard of the form YYYYMMDDTHHMMSS

Parameters:

x (str or datetime) – the time stamp to be converted

Returns:

the converted time stamp in format YYYYmmddTHHMMSS

Return type:

str