Output file structure

Source: vignettes/outstructure.Rmd

Products generated using sen2r functions follows a precise naming convention, and are organised within a customisable folder structure.

Products

Currently, sen2r can be used to generate the following products:

  1. surface reflectances (TOA or BOA);
  2. Accessory layers (SCL, CLD, SNW, WVP, AOT);
  3. Spectral indices;
  4. true or false colour RGB images.

Surface reflectances

TOA (Top Of Atmosphere) and BOA (Bottom Of Atmosphere) surface reflectances are multiband rasters containing 12 (TOA) or 11 (BOA) bands, here listed:

  1. Band 1 – Aerosol (443 nm)
  2. Band 2 – Blue (490 nm)
  3. Band 3 – Green (560 nm)
  4. Band 4 – Red (665 nm)
  5. Band 5 – Red-edge 1 (705 nm)
  6. Band 6 – Red-edge 2 (740 nm)
  7. Band 7 – Red-edge 3 (783 nm)
    • on rasters with an output resolution < 20m: Band 8 – NIR (842 nm)
    • on rasters with an output resolution >= 20m: Band 8A – narrow NIR (865 nm)
  9. Band 9 – Water vapour (940 nm)
  • on TOA images:
     10. Band 10 – Cirrus (1375 nm)
     11. Band 11 – SWIR1 (1610 nm)
     12. Band 12 – SWIR2 (2190 nm)
  • on BOA images:
     10. Band 11 – SWIR1 (1610 nm)
     11. Band 12 – SWIR2 (2190 nm)

Bands are named if an output ENVI format is chosen.

For details about Sentinel-2 bands, refer to this page.

All bands are reshaped with the same resolution. In case the user chose a native resolution (10, 20 or 60 m), bands with a finest resolution are downscaled using an averaged method, while pixels of bands with a coarse resolution are replicated without any interpolation method; conversely, in case the user chose a custom resolution, the chosen resampling method is used.

Bands 8 and 8A are alternatively included, basing on the required output resolution. That is due to the fact that they cover an overlapping spectral window; so, band 8A (finer radiometric resolution, coarse spatial one) is provided when the output spatial resolution is adequate to include it (>= 20m), otherwise band 8 is used. The possibility to include both NIR bands could be included in a future release.

Reflectances values are provided in integer output type (multiplicated by 10000), as well as original values included in SAFE archives.

Accessory layers

Scene Classification Map

SCL (Scene Classification Maps) are single band rasters provided within Level-2A SAFE archives. Each 20-metres pixel is classified in one of the following classes:

  1.   No data   
  2.   Saturated or defective   
  3.   Dark area pixels   
  4.   Cloud shadows   
  5.   Vegetation   
  6.   Bare soils   
  7.   Water   
  8.   Unclassified   
  9.   Cloud (medium probability)   
  10.   Cloud (high probability)   
  11.   Thin cirrus   
  12.   Snow   

For further details about this product, see this page.

SCL are used as intermediate layers for cloud masking, but they can be also required as output formats. If cloud masking is required by the user, SCL maps are never masked.

Cloud and snow probability

CLD (cloud probability) and SNW (snow probability) are single band rasters provided within Level-2A SAFE archives, representing the percentage probability (0-100 Byte values) of each 20 metres pixel to be cloud- or snow-covered. If cloud masking is required by the user, CLD and SNW maps are never masked. Currently these layers cannot be used for cloud masking (only SCL can).

Water vapour column

WVP is a single band raster provided within Level-2A SAFE archives, representing the water vapour column depth. Data type is Int16 (values must be multiplicated by 1000 to obtain floating values in centimetres). Typical ranges of water vapour columns are (sea-level-to space): tropical conditions: WVP = 3-5 cm; mid-altitude summer: WVP = 2-3 cm; dry summer, spring, fall: WVP = 1-1.5 cm; dry desert or winter: WVP = 0.3-0.8 cm.

For further details about this product, see the Sentinel-2 Level 2A Product Definition document.

Aerosol Optical Thickness

AOT is a single band raster provided within Level-2A SAFE archives, representing the aerosol optical thickness. It is defined as the integrated extinction coefficient over a vertical column of atmosphere of unit cross section. Extinction coefficient is the fractional depletion of radiance per unit path length (also called attenuation for radar frequencies). Data type is Int16 (values must be multiplicated by 1000 to obtain original values). If cloud masking is required by the user, AOT is never masked.

For further details about this product, see the Sentinel-2 Level 2A Product Definition document.

Spectral indices

Spectral indices are single band rasters computed from BOA (by default) or TOA images using function s2_calcindices() (or the main function sen2r()).

The list of the indices which can be computed can be seen in the GUI (fourth sheet). In the GUI, the index formula is usually graphically provided. In order to check the formula which is used for index computation, the function `list_indices()’ can be used. For example, the following command will return the formula of index NDVI:

list_indices("s2_formula", "^NDVI$")
## [1] "(band_8-band_4)/(band_8+band_4)"

See the function documentation for other examples.

The output type can be both floating (in this case, the actual index value is provided) or integer (default; in this case, values are multiplicated by a scaling factor, 10000 by default). So, if the user did not change the arguments outType and scaleFactor of function s2_calcindices(), or index_datatype of function sen2r(), raster values will be multiplicated by 10000 respect to the “expected” index values.

RGB colour images

RGB are three-bands raster in 24-bit format, computed from BOA or TOA images using function s2_rgb() (or the main function sen2r()).

Users can customise the bands used to produce images, as well as the reflectance ranges of each band and the source (BOA or TOA): the easiest way to do this is to use the GUI (fifth sheet), or to properly set the arguments of function s2_rgb().

If cloud masking is required by the user, RGB are never masked (this because they are thought for visualisation).

The names of RGB products contains information about reflectance source and reproduced bands (see the next section).

Naming convention

ESA Sentinel-2 naming convention is particularly long-winded; so, a different convention was adopted, follows this schema:

S2mll_yyyymmdd_rrr_EXTNAME_PROD_rr.fff

(e.g. S2A2A_20190723_022_Barbellino_BOA_10.tif)

  • S2mll (length: 5) shows the mission ID (S2A or S2B) and the product level (1C or 2A);
  • yyyymmdd (length: 8) is the sensing date (e.g.  20190723 for 2019-07-23); the hour is skipped, since a single sensor can not pass two times in a day on the same tile);
  • rrr (length: 3) is the relative orbit number (e.g.  022);
  • EXTNAME (length: variable, see details) is the output extent name (e.g.  Barbellino); it could also be the the tile number (length: 5 – e.g.  32TQQ) for outputs of function s2_translate(), and is kept empty for outputs of function s2_merge();
  • PROD (length: variable) is the output product name, being one of these:
    • an output product: TOA (13-bands Top-Of-Atmosphere Reflectance), BOA (13-bands Bottom-Of-Atmosphere Reflectance) or SCL (Scene Classification Map);
    • a spectral index name (the list of supported index names can be obtained with the function list_indices("name"));
    • the name of a RGB image (length: 7, see details).
  • rr (length: 2) is the original minimum spatial resolution in metres (10, 20 or 60);
  • fff (length: variable, generally 3) is the file extension (e.g. tif).

Extent name

The extent name is defined with the argument extent_name of function sen2r().

The user can choose any non-empty alphanumeric string as extent name, as long as it does not contain spaces, points nor underscores, and as long as it is not a five-length string with the same structure of a tile ID (two numeric and three uppercase character values), because this structure is reserved for intermediate layers.

RGB names

The names of RGB images are 7-length strings following the structure RGBrgbX (e.g. RGBb84B), where:

  • r, g and b are the number of the bands to be used respectively for red, green and blue, in hexadecimal format (in the example above, b, 8 and 4 correspond to bands 11, 8 and 4);
  • X is B if index source is BOA, or T if source is TOA.

Folder structure

Processing functions allows setting an output directory using the arguments path_out, path_rgb and path_indices (sen2r()) or outdir (s2_translate(), s2_merge(), s2_mask(), s2_rgb() and s2_calcindices()). These folders are automatically created if their parent directories exist.

By default, functions create a subfolder for each product / index / RGB type, in which files are stored. Each product subfolder includes a sub-subfolder named thumbnails, which contains small JPEG or PNG images which can be used to rapidly monitor what was done (see s2_thumbnails() for further details).

For example, the following command would produce the outputs listed below.

json_path <- build_example_param_file()
out_dir_1 <- tempfile(pattern = "sen2r_out_1_")

library(sen2r)
sen2r(
  json_path, 
  timewindow = c("2019-07-13","2019-07-23"),
  path_out = out_dir_1
)
[2020-02-06 09:42:40] #### Starting sen2r execution. ####
[2020-02-06 09:42:40] Searching for available SAFE products on SciHub...
[2020-02-06 09:42:46] Computing output names...
[2020-02-06 09:42:48] Processing group 1 of 5...

Most of output message were skipped; see this vignette for further details.

╔══════════════════════════════════════════════════════════════════════════════
║ sen2r Processing Report
╟──────────────────────────────────────────────────────────────────────────────
║ Dates to be processed based on processing parameters: 5
║ Processing completed for: all expected dates.
╚══════════════════════════════════════════════════════════════════════════════
[2020-02-06 09:44:45] #### sen2r session terminated. ####
The processing chain can be re-launched with the command:
  sen2r("/home/lranghetti/.sen2r/proc_par/s2proc_20200206_094240.json")
system(paste("tree", out_dir_1)) # working on Linux with binary "tree" installed
/tmp/RtmpUlAdHw/sen2r_out_1_a55119f4ed09
├── BOA
│   ├── S2A2A_20190713_022_sen2r_BOA_10.tif
│   ├── S2A2A_20190716_065_sen2r_BOA_10.tif
│   ├── S2A2A_20190723_022_sen2r_BOA_10.tif
│   ├── S2B2A_20190718_022_sen2r_BOA_10.tif
│   ├── S2B2A_20190721_065_sen2r_BOA_10.tif
│   └── thumbnails
│       ├── S2A2A_20190713_022_sen2r_BOA_10.jpg
│       ├── S2A2A_20190713_022_sen2r_BOA_10.jpg.aux.xml
│       ├── S2A2A_20190716_065_sen2r_BOA_10.jpg
│       ├── S2A2A_20190716_065_sen2r_BOA_10.jpg.aux.xml
│       ├── S2A2A_20190723_022_sen2r_BOA_10.jpg
│       ├── S2A2A_20190723_022_sen2r_BOA_10.jpg.aux.xml
│       ├── S2B2A_20190718_022_sen2r_BOA_10.jpg
│       ├── S2B2A_20190718_022_sen2r_BOA_10.jpg.aux.xml
│       ├── S2B2A_20190721_065_sen2r_BOA_10.jpg
│       └── S2B2A_20190721_065_sen2r_BOA_10.jpg.aux.xml
├── MSAVI2
│   ├── S2A2A_20190713_022_sen2r_MSAVI2_10.tif
│   ├── S2A2A_20190716_065_sen2r_MSAVI2_10.tif
│   ├── S2A2A_20190723_022_sen2r_MSAVI2_10.tif
│   ├── S2B2A_20190718_022_sen2r_MSAVI2_10.tif
│   ├── S2B2A_20190721_065_sen2r_MSAVI2_10.tif
│   └── thumbnails
│       ├── S2A2A_20190713_022_sen2r_MSAVI2_10.jpg
│       ├── S2A2A_20190713_022_sen2r_MSAVI2_10.jpg.aux.xml
│       ├── S2A2A_20190716_065_sen2r_MSAVI2_10.jpg
│       ├── S2A2A_20190716_065_sen2r_MSAVI2_10.jpg.aux.xml
│       ├── S2A2A_20190723_022_sen2r_MSAVI2_10.jpg
│       ├── S2A2A_20190723_022_sen2r_MSAVI2_10.jpg.aux.xml
│       ├── S2B2A_20190718_022_sen2r_MSAVI2_10.jpg
│       ├── S2B2A_20190718_022_sen2r_MSAVI2_10.jpg.aux.xml
│       ├── S2B2A_20190721_065_sen2r_MSAVI2_10.jpg
│       └── S2B2A_20190721_065_sen2r_MSAVI2_10.jpg.aux.xml
├── NDVI
│   ├── S2A2A_20190713_022_sen2r_NDVI_10.tif
│   ├── S2A2A_20190716_065_sen2r_NDVI_10.tif
│   ├── S2A2A_20190723_022_sen2r_NDVI_10.tif
│   ├── S2B2A_20190718_022_sen2r_NDVI_10.tif
│   ├── S2B2A_20190721_065_sen2r_NDVI_10.tif
│   └── thumbnails
│       ├── S2A2A_20190713_022_sen2r_NDVI_10.jpg
│       ├── S2A2A_20190713_022_sen2r_NDVI_10.jpg.aux.xml
│       ├── S2A2A_20190716_065_sen2r_NDVI_10.jpg
│       ├── S2A2A_20190716_065_sen2r_NDVI_10.jpg.aux.xml
│       ├── S2A2A_20190723_022_sen2r_NDVI_10.jpg
│       ├── S2A2A_20190723_022_sen2r_NDVI_10.jpg.aux.xml
│       ├── S2B2A_20190718_022_sen2r_NDVI_10.jpg
│       ├── S2B2A_20190718_022_sen2r_NDVI_10.jpg.aux.xml
│       ├── S2B2A_20190721_065_sen2r_NDVI_10.jpg
│       └── S2B2A_20190721_065_sen2r_NDVI_10.jpg.aux.xml
├── RGB432B
│   ├── S2A2A_20190713_022_sen2r_RGB432B_10.tif
│   ├── S2A2A_20190716_065_sen2r_RGB432B_10.tif
│   ├── S2A2A_20190723_022_sen2r_RGB432B_10.tif
│   ├── S2B2A_20190718_022_sen2r_RGB432B_10.tif
│   ├── S2B2A_20190721_065_sen2r_RGB432B_10.tif
│   └── thumbnails
│       ├── S2A2A_20190713_022_sen2r_RGB432B_10.jpg
│       ├── S2A2A_20190713_022_sen2r_RGB432B_10.jpg.aux.xml
│       ├── S2A2A_20190716_065_sen2r_RGB432B_10.jpg
│       ├── S2A2A_20190716_065_sen2r_RGB432B_10.jpg.aux.xml
│       ├── S2A2A_20190723_022_sen2r_RGB432B_10.jpg
│       ├── S2A2A_20190723_022_sen2r_RGB432B_10.jpg.aux.xml
│       ├── S2B2A_20190718_022_sen2r_RGB432B_10.jpg
│       ├── S2B2A_20190718_022_sen2r_RGB432B_10.jpg.aux.xml
│       ├── S2B2A_20190721_065_sen2r_RGB432B_10.jpg
│       └── S2B2A_20190721_065_sen2r_RGB432B_10.jpg.aux.xml
├── RGB843B
│   ├── S2A2A_20190713_022_sen2r_RGB843B_10.tif
│   ├── S2A2A_20190716_065_sen2r_RGB843B_10.tif
│   ├── S2A2A_20190723_022_sen2r_RGB843B_10.tif
│   ├── S2B2A_20190718_022_sen2r_RGB843B_10.tif
│   ├── S2B2A_20190721_065_sen2r_RGB843B_10.tif
│   └── thumbnails
│       ├── S2A2A_20190713_022_sen2r_RGB843B_10.jpg
│       ├── S2A2A_20190713_022_sen2r_RGB843B_10.jpg.aux.xml
│       ├── S2A2A_20190716_065_sen2r_RGB843B_10.jpg
│       ├── S2A2A_20190716_065_sen2r_RGB843B_10.jpg.aux.xml
│       ├── S2A2A_20190723_022_sen2r_RGB843B_10.jpg
│       ├── S2A2A_20190723_022_sen2r_RGB843B_10.jpg.aux.xml
│       ├── S2B2A_20190718_022_sen2r_RGB843B_10.jpg
│       ├── S2B2A_20190718_022_sen2r_RGB843B_10.jpg.aux.xml
│       ├── S2B2A_20190721_065_sen2r_RGB843B_10.jpg
│       └── S2B2A_20190721_065_sen2r_RGB843B_10.jpg.aux.xml
└── SCL
    ├── S2A2A_20190713_022_sen2r_SCL_10.tif
    ├── S2A2A_20190716_065_sen2r_SCL_10.tif
    ├── S2A2A_20190723_022_sen2r_SCL_10.tif
    ├── S2B2A_20190718_022_sen2r_SCL_10.tif
    ├── S2B2A_20190721_065_sen2r_SCL_10.tif
    └── thumbnails
        ├── S2A2A_20190713_022_sen2r_SCL_10.png
        ├── S2A2A_20190713_022_sen2r_SCL_10.png.aux.xml
        ├── S2A2A_20190716_065_sen2r_SCL_10.png
        ├── S2A2A_20190716_065_sen2r_SCL_10.png.aux.xml
        ├── S2A2A_20190723_022_sen2r_SCL_10.png
        ├── S2A2A_20190723_022_sen2r_SCL_10.png.aux.xml
        ├── S2B2A_20190718_022_sen2r_SCL_10.png
        ├── S2B2A_20190718_022_sen2r_SCL_10.png.aux.xml
        ├── S2B2A_20190721_065_sen2r_SCL_10.png
        └── S2B2A_20190721_065_sen2r_SCL_10.png.aux.xml

12 directories, 90 files

In each of the 6 subfolders BOA, MSAVI2, NDVI, RGB432B, RGB843B and SCL 5 products (one for each date) were created. Sub-subfolders thumbnails contains thumbnail images.

Users can skip the creation of products’ subfolders setting the argument sen2r(..., path_subdirs = FALSE) (default: TRUE), or doing the same with the argument subdir of other processing functions. The creation of thumbnails can be skipped using sen2r(..., thumbnails = FALSE) (default: TRUE). Both the options can be also set using the sen2r() GUI.

For example, the following command would produce the outputs listed below.

out_dir_2 <- tempfile(pattern = "sen2r_out_2_")

sen2r(
  json_path, 
  timewindow = c("2019-07-13","2019-07-23"),
  path_out = out_dir_2,
  path_subdirs = FALSE,
  thumbnails = FALSE
)
[2020-02-06 09:52:30] #### Starting sen2r execution. ####
[2020-02-06 09:52:30] Searching for available SAFE products on SciHub...
[2020-02-06 09:52:35] Computing output names...
[2020-02-06 09:52:36] Processing group 1 of 5...

# most of output message were skipped

╔══════════════════════════════════════════════════════════════════════════════
║ sen2r Processing Report
╟──────────────────────────────────────────────────────────────────────────────
║ Dates to be processed based on processing parameters: 5
║ Processing completed for: all expected dates.
╚══════════════════════════════════════════════════════════════════════════════
[2020-02-06 09:54:16] #### sen2r session terminated. ####
The processing chain can be re-launched with the command:
  sen2r("/home/lranghetti/.sen2r/proc_par/s2proc_20200206_095230.json")
system(paste("tree", out_dir_2))
/tmp/RtmpUlAdHw/sen2r_out_2_a5513ad348b0
├── S2A2A_20190713_022_sen2r_BOA_10.tif
├── S2A2A_20190713_022_sen2r_MSAVI2_10.tif
├── S2A2A_20190713_022_sen2r_NDVI_10.tif
├── S2A2A_20190713_022_sen2r_RGB432B_10.tif
├── S2A2A_20190713_022_sen2r_RGB843B_10.tif
├── S2A2A_20190713_022_sen2r_SCL_10.tif
├── S2A2A_20190716_065_sen2r_BOA_10.tif
├── S2A2A_20190716_065_sen2r_MSAVI2_10.tif
├── S2A2A_20190716_065_sen2r_NDVI_10.tif
├── S2A2A_20190716_065_sen2r_RGB432B_10.tif
├── S2A2A_20190716_065_sen2r_RGB843B_10.tif
├── S2A2A_20190716_065_sen2r_SCL_10.tif
├── S2A2A_20190723_022_sen2r_BOA_10.tif
├── S2A2A_20190723_022_sen2r_MSAVI2_10.tif
├── S2A2A_20190723_022_sen2r_NDVI_10.tif
├── S2A2A_20190723_022_sen2r_RGB432B_10.tif
├── S2A2A_20190723_022_sen2r_RGB843B_10.tif
├── S2A2A_20190723_022_sen2r_SCL_10.tif
├── S2B2A_20190718_022_sen2r_BOA_10.tif
├── S2B2A_20190718_022_sen2r_MSAVI2_10.tif
├── S2B2A_20190718_022_sen2r_NDVI_10.tif
├── S2B2A_20190718_022_sen2r_RGB432B_10.tif
├── S2B2A_20190718_022_sen2r_RGB843B_10.tif
├── S2B2A_20190718_022_sen2r_SCL_10.tif
├── S2B2A_20190721_065_sen2r_BOA_10.tif
├── S2B2A_20190721_065_sen2r_MSAVI2_10.tif
├── S2B2A_20190721_065_sen2r_NDVI_10.tif
├── S2B2A_20190721_065_sen2r_RGB432B_10.tif
├── S2B2A_20190721_065_sen2r_RGB843B_10.tif
└── S2B2A_20190721_065_sen2r_SCL_10.tif

0 directories, 30 files