Tools

The package pyspex comes with several tools, these are desccribed in this section.

Script: spx1_level01a

Generate PACE level-1A product from SPEXone level-0 data.

Usage:

spx1_level01a  [-h] [--debug] [--dump] [--verbose]
               [--outdir OUTDIR] [--yaml YAML]
               [lv0_list ...]

positional arguments:

lv0_list   provide one or more input files:
- raw: CCSDS packages generated by the SPEXone ICU -- provide name
  of one file with extension '.H'. The files with science and
  housekeeping data are collected using Unix filename pattern matching.
- st3: CCSDS packages with ITOS and spacewire headers -- provide name
  of one file with extension '.ST3'.
- dsb: CCSDS packages with PACE headers -- provide list of filenames
  with extension '.spx'.

Options:

-h, --help           show this help message and exit.
--debug              be more verbose and do not write any output products.
--dump               dump CCSDS packet headers in ASCII.
--verbose            be verbose.
--outdir OUTDIR      directory to store the Level-1A product.
--yaml YAML          provide settings via a YAML file.

Return codes:

    Failed to parse command-line parameters.
  Input file does not exist or failed to read from input file.
  Input file not recognized as a SPEXone level-0 product.
  Corrupted SPEXone level-0 data.
  SPEXone level-0 data does not contain Science data, no L1A generated.
  Failed to generate output file due to permission error.
  Failed to generate output file due to netCDF4 library issues.

Content of YAML file:

# define output directory, CWD when empty
outdir: CWD
# define name of output file, will be generated automatically when empty
outfile: ''
# define file-version as nn, neglected when outfile not empty
file_version: 1
# flag to compress netCDF4 variable: '/science_data/detector_images'
# createVariable parameters: compression='zlib', complevel=1, shuffle=True.
compression: True
# flag to indicate measurements taken in eclipse or day-side
eclipse: True
# provide list, directory, file-glob or empty
hkt_list: ''
# must be a list, directory or glob. Fails when empty
l0_list: L0/SPX0000000??.spx

Examples:

Generate L1A from OCAL level-0 data directly from the SPEXone instrument:

   spx1_level01a <Path>/NomSciCal1_20220123T121801.676167.H

   Note that OCAL science & telemetry data is read from the files:
     <Path>/NomSciCal1_20220123T121801.676167.?
     <Path>/NomSciCal1_20220123T121801.676167.??
     <Path>/NomSciCal1_20220123T121801.676167_hk.?

Generate L1A from OCAL level-0 data via ITOS from the PACE platform:

   spx1_level01a <Path>/DIAG_20220124_175458_073.ST3

Generate L1A from inflight level-0 data, store product in directory L1A:

   spx1_level01a --outdir L1A <Path>/SPX*.spx

Generate L1A from inflight level-0 data read settings from a YAML file:

   spx1_level01a --yaml config_l1a_gen.yaml

   An example YAML file:
      outdir: L1A
      outfile: ''
      file_version: 1
      eclipse: False
      hkt_list: HKT/PACE.20220617T011*.HKT.nc
      l0_list: L0/SPX0220000??.spx

Dry-run: be extra verbose without generating data:

   spx1_level01a --debug <Path>/NomSciCal1_20220123T121801.676167.H

Read level-0 data and dump CCSDS packet headers in ASCII:

   spx1_level01a --dump <Path>/NomSciCal1_20220123T121801.676167.H

Environment:

OCVARROOT
  The number of leap seconds for the TAI to UTC conversion are determine
  using the file 'tai-utc.dat'. A copy of this file is included in the
  package `pyspex`. The latest version can be obtained from
        `https://maia.usno.navy.mil/ser7/tai-utc.dat`.
  When OCVARROOT is set the path should be '$OCVARROOT/common/tai-utc.dat'.

Inflight measurements

The SPEXone instrument can generate data in 2 modes:

Science mode: this is binned data according to various binning tables that are present in the flash memory of the detector module.
Diagnostic mode: this is full frame data (2048x2048 pixels).

The data that is taken during an orbit can be divided into 3 types:

Science data: data taken during the dayside of the nomimal orbit, only science mode MPSes.
Dark data: data taken during the eclipse in order to obtain dark measurements (both Earth viewing and during lunar cal) with the same science mode MPSes as used during the dayside of an orbit. These are to be used for direct offset and dark current correction of science data.
Calibration data: data taken during the eclipse using only diagnostic mode MPSes. The purpose is to obtain full frame detector images, both with and without the LED on, which are to be used for detector monitoring and recalibration (offset, dark current, non-linearity, PRNU).

Note:

There is no unique division based on MPSes possible that separates dayside data from eclipse data, since the same science mode MPSes are used in both orbit periods.

So for processing of data during nominal operations, the data types will be stored as follows:

PACE_SPEXONE.YYYYmmddTHHMMSS.L1A.nc for “Science data”
PACE_SPEXONE_DARK.YYYYmmddTHHMMSS.L1A.nc for “Dark data”
PACE_SPEXONE_CAL.YYYYmmddTHHMMSS.L1A.nc for “Calibration data”

This means that if the dayside data is captured into a single L0-file, this L0-file can be processed with a single command to L1A-data (parameter eclipse=False). The eclipse data would also be captured into a single L0-file and distributed bij the level 0->1a processor over two L1A-files: diagnostic mode MPSes to the CAL-file and science mode MPSes to the DARK-file (parameter eclipse=True).

Script: spx1_add_egse2l1a.py

Add ITOS EGSE information of OCAL measurements to a SPEXone level-1A product.

Usage:

spx1_add_egse2l1a.py [-h] [--verbose] [--egse_dir EGSE_DIR]
                     {create_db,add} ...

positional arguments:

{create_db,add}      sub-command help.
 create_db           create new EGSE database.
 add                 add EGSE information to a SPEXone level-1A product.

options:

-h, --help            show this help message and exit.
--verbose             be verbose.
--egse_dir EGSE_DIR   directory with EGSE information.

Script: spx1_add_ogse2l1a.py

Add OGSE information of a OCAL measurement to a SPEXone L1A product.

Possible OGSE information:

Reference diode (Ambient/GSFC polarized, radiometric) located in the integrated sphere.
Wavelength monitor (Ambient/GSFC polarized, radiometric, wavelength).
Wavelength of OPO laser (GSFC spectral-radiometry).

usage:

spx1_add_ogse2l1a.py [-h] [--verbose] [--ogse_dir oGSE_DIR]
                      {create_db,add} ...

positional arguments:

{create_db,add}     sub-command help.
 create_db          create new OGSE database.
 add                add OGSE information to a SPEXone level-1A product.

options:

-h, --help            show this help message and exit.
--verbose             be verbose.
--ogse_dir OGSE_DIR   directory with OGSE information.