Concepts

The GEP provides access to EO data coming from several missions and sensors and provided using different data formats and metadata.

These acquisitions have heterogeneous formats for the data files and associated metadata.

To reduce this heterogeneity, the GEP seeks to pre-process the acquisitions into a common format which, on the one hand, provides a ready-to-inspect and comparable dataset and, on the other, to have downstream processing services that support multi-sensor and multi-mission acquisitions without implementing complex blocks.

Below are described the main pre-processing that support this approach.

STAC as a common metadata model

EO data is handled under the GEP Processing Chains using the SpatioTemporal Asset Catalog (STAC)¹.

STAC consists of a standardized way to catalog and expose multi-source geospatial data.

The basic core of STAC¹ is adopted and extended within the GEP to manage a wide collection of multi-source satellite imagery using multiple STAC “spatiotemporal” Assets.

Each STAC item (e.g. a single Sentinel-2 MSI L1C product for UTM zone 9 and grid square XK) includes multiple assets (e.g. multiple multispectral bands) and consists of a GeoJSON feature with metadata and thumbnail and can be organized under a STAC Catalog (e.g. multitemporal Sentinel-2 L1C MSI products for UTM zone 9 and grid square XK).

Each STAC Catalog can be then grouped under a STAC Collection (e.g. Sentinel-2 L1C) which can also further complement associated metadata information such as producer, processor, host, license, version, temporal extent, GSD, instrument, off-nadir angle, etc. More information can be found at STAC Common Metadata section of STAC Specification documentation².

Common Band Names (CBN)

STAC assets of EO data are catalogued in the GEP using Common Band Names (CBN)². CBN classes refer to common band ranges derived from pre-defined frequency ranges of the Electromagnetic Spectrum made for several popular instruments. Each CBN class is defined by a Band Range in micrometers for optical and in centimeters for SAR data. The CBN classification of the frequency spectrum allows a one-to-one mapping of multi-mission and multi-sensor bands (Optical and SAR). CBN thus ease the handling multi-sensor source EO data.

Optical Common Band Names

To classify a generic band from an optical sensor with the CBN schema (e.g. Worldview-3 Yellow band from 0.584 to 0.632 μm), one way is to derive its centered wavelength in micrometers (0.608 μm) and identify the CBN class having the closest central wavelength (CBN class 05 and centered at 0.6 μm). However, while choosing a CBN class also the bandwidth needs to be considered. Common Band Names "nir" and "lwir" refer, in fact, to wider bands that cover most of the spectral range for NIR (0.75μm to 1.0μm) and TIR (10.5μm to 12.5μm) radiation. On the other hand, narrow bands for "nir" are "nir08" and "nir09", centered at 0.85μm and 0.95μm respectively. This is particularly useful for example to classify sensors having both a wide (e.g. Sentinel-2 MSI Band 8 at 833 nm) and a narrow (Sentinel-2 MSI Band 8a at 865 nm) band over the same portion of the EM spectrum. CBN "nir" will refer then to S2 MSI spectral Band 8 and "nir08" to the S2 MSI spectral band 8a.

Figure 1 - Comparison of Landsat-7 and 8 bands with the ones of Sentinel-2 (Image credit: USGS).

The same applies for "lwir" which is also discretized into two narrow bands "lwir11" and "lwir12", centered at 11μm and 12μm respectively. The CBN schema is not rigid and allows the definition of new classes in case the definition of an additional portion of the spectrum is required (e.g. to deal with future EO missions in the GEP Optical Payload having different spectral resolution that cannot be fully classified with the current CBN schema).

A total of 29 CBNs (pan, coastal, blue, yellow, etc.) are identified for the GEP (see Table 1). This table also includes a sample application of this CBN schema, made for Landsat-7 ETM+, Landsat-8 OLI/TIRS and Sentinel-2 MSI data.

Table 1 - The Common Band Name schema of the GEP and its application for Landsat-7, Landsat-8 and Sentinel-2 missions.

To better explain the advantages in managing multi-sensor EO optical data with CBN, an application of CBN schema, made for a selection of the GEP optical payload is shown in Table 2. This enables a clear mapping of EO data across the EM spectrum with STAC assets.

Table 2 - Common Band Names for a selection of the GEP Optical payload products. P, C, B, G, Y, and R stands for PAN, Cyan, Blue, Green, Yellow, and Red respectively.

SAR Common Band Names

A similar schema is applied for SAR missions. CBN classes for SAR refer to a combination of frequency band and polarization derived from multiple band numbers of several popular SAR instruments for disaster mapping. The CBN classification of microwave radiation enables a one-to-one mapping of multi-mission and multi-sensor SAR products. Table 3 shows a sample application of CBN for radar backscatter which is used in the GEP SAR missions.

Table 3 - Common Band Names schema for the GEP SAR payload products.

CBNs for SAR are meant to classify Sigma Nought in decibel from the multi-mission SAR Payload using this code s0_db_r_pp, where “s0_db” means Sigma Nought in dB, “r” is the SAR-band (X, C or L), and “pp” is the polarization (HH, HV, VH or VV). This schema enables the mapping of the SAR products provided as quad- (HH+HV+VH+VV), dual- (VV+VH or HH+HV), or single-polarization (HH or VV). Cross-pol assets are generally provided within dual- or quad-pol products. The ICEYE mission is represented by only the CBN s0_db_x_vv because the GRD products are given only in VV polarization.

Different types of output products

In the GEP two main types of Product can be derived from both systematic and on-demand processing services:

1. overview assets (full-res browse images as grayscale or RGB composite).

2. single band assets (TOA reflectance, Brightness Temperature, Sigma Nought, interferometric phase and coherence, LOS displacement, spectral indexes, change detection bitmasks, etc.).

Each of them is given by following a dedicated data structure (e.g. unit, data type, scale factor, valid range) with respect to the nature of the product. More details about each of the physical meaning or visual products of the GEP can be found in the specifications of each service.

Visualize an asset in the Geobrowser with Titiler

When opening the feature details panel in the left panel of the Geobrowser, an additional widget called "Titiler options" is shown in the feature's details, inside the collapsible section "Layer styling". From this widget users can manipulate the titiler api options to see the layer in different ways and styles. More precisely, users can:

• choose one asset or select an assets combination;
• set and expression or select a predefinite color map (for one asset)
• set the histogram min and max value;
• apply a formula for color representation;

The visualization and the options of a Titiler layer is basing on the concept of Asset, defined on the STAC specification. The SpatioTemporal Asset Catalog (STAC) specification provides a common language to describe a range of geospatial information, so it can more easily be indexed and discovered. A 'spatiotemporal asset' is any file that represents information about the earth captured in a certain space and time. For more info, see: https://stacspec.org/.

Any feature containing a Titiler data has a set of assets (e.g. a reflectance or spectral index asset) users can choose a single asset to view on the map, or a band-combination of three assets, mapped on the three RGB channel (Red, Green, Blue).

The Single Asset tab allow users to select one asset from the feature's available assets. If the asset is represented by a mono-band source, by default a grayscale image will be plotted. There are 2 options:

• Expression: a mathematical expression to combine more bands in one. For example, if we have these band [red, green, blue, nir] a possible expression could be (red-green)/nir.

• Color map: a select input containing a predefinite set of color mapping functions, only for mono-band assets.

The Combine Assets tab allow users to select three assets for the three RGB band and combine them. This tab doesn't have any option.

Other useful options, common for both single asset or assets combination, are histogram min/max and color formula.

Histogram visualization

A feature exposing a Titiler tiled layer can include a detailed histogram information for each asset, if available. The histogram provides the distribution of the pixel values. The space of pixel values is splitted in 256 buckets, and for each bucket is calculated the number of pixel inside of it. With these data is possible to plot an histogram, allowing the user to:

• See the distribution of pixel values;
• Get the exact amount of pixel inside a bucket range;
• Set the histogram min/max value directly from a histogram interaction;
• Perform an vertical cut (by considering only the 99% of buckets in ascending order) to better show the histogram with a vertical stretch;
• Perform an horizontal cumulative cut ([2%-98%] of distribution), by a visual stretching of the histogram and by setting the right histogram min/max values.

Below the histogram, the histogram min/max fields can be used to define a range of minimum and maximum values of the digital number of the asset selected.

On the EO context, there are several types of assets, each of them has a predefinite corresponding scale, as described in Table 4.

Table 4 - Description, Unit, Bit depth, Scaling factors, and valid ranges for the multiple types of assets available in the GEP.

Color formula

With the color formula, it's possible to apply basic color-oriented image operations to the titiler image layer. The syntax is based on the rio-color plugin for the Cloud Optimized GeoTIFF in TiTiler and is the following:

Gamma RGB 1 Saturation 1 Sigmoidal RGB 3 0.5

where \(Gamma\), \(Saturation\) and \(Sigmoidal\) are the parameters described in Table 5.

Table 5 - Default values for Color Formulas used for the generation of browse images in the PE (Credits: Rio-Color in GitHub). Such parameters are the ones available in the Rio Color color formula expression for COG in TiTiler.

For more information, see https://github.com/mapbox/rio-color.

A common format for output raster products

All the output raster products derived from the GEP Processing Service, which are summarized in here, are given in the Cloud Optimized GeoTIFF (COG)³ format.

COG is an extension of the GeoTIFF format dedicated for data hosting on HTTP file servers. One of the main advantages in using COG format in cloud processing environments is that the single GeoTIFF file can be accessed by multiple clients with no need to copy or cache the desired product.

Furthermore, it allows the client to easily retrieve via HTTP GET range requests just a desired portion of the data required for more efficient workflows on the cloud. As an example, in the GEP the pre-processed datasets are stored in COG format to reduce data transfer during the thematic processing. The COG format is already employed under multiple initiatives such as: OpenAerialMap, INPE, MAXAR/DigitalGlobe, Planet, NASA and Copernicus’s Mundi DIAS.

STAC and COG have been recently adopted by USGS for the provision of Landsat-7 and Landsat-8 products, as described in the USGS Data Format Control Book⁴ which describes the format of the data to be used in Collection 2 processing.