pyspex package
Subpackages
Submodules
pyspex.binning_tables module
Tools to read or write definitions of SPEXone binning-tables.
- class pyspex.binning_tables.BinningTables(ckd_dir: str | None = None)[source]
Bases:
objectClass to handle SPEXone binning-table definitions.
- Parameters:
ckd_dir (Path) – Specify the name of directory with SPEXone binning-table files.
- Raises:
FileNotFoundError – Directory with SPEXone binning-table files does not exist.
Notes
Syntax of the file name with SPEXone binning-tables:
SPX1_CKD_BIN_TBL_<yyyymmddTHHMMSS>_<NNN>.nc
where yyyymmddTHHMMSS defines the validity start (UTC) and NNN the release number of the file format.
The binning tables as defined on-ground are supposed to be available during the whole mission at the same on-board memory location. Because these original binning tables are necessary for re-processing and may facilitate instrument performance monitoring. Therefore, it is preferred that a new binning table is added to the current set, without changing the validity start string. However, new binning-table file should be released in case any of the binning tables are overwritten.
Examples
# create new file with binning-table definitions:
> bin_tbl = BinningTables() > bin_tbl.create_if_needed(validity_start) > bin_tbl.add_table(0, lineskip_arr, binning_table) > bin_tbl.add_table(1, lineskip_arr, binning_table)
# add a new binning-table to an existing file:
> bin_tbl BinningTables() > bin_tbl.create_if_needed(validity_start) > bin_tbl.add_table(2, lineskip_arr, binning_table)
# use binning-table ‘130’ to unbin SPEXone detector data:
> bin_tbl BinningTables() > bin_tbl.search(coverage_start) > img = bin_tbl.unbin(130, img_binned)
- add_table(table_id: int, lineskip_arr: ndarray, binning_table: ndarray) None[source]
Add a binning table definition to existing file.
- Parameters:
table_id (int) – Table identifier (integer between 1 and 255)
lineskip_arr (ndarray) – Lineskip array definition
binning_table (ndarray) – Binning table definition
- create_if_needed(validity_start: str, release: int = 1) None[source]
Initialize CKD file for binning tables if not exist.
- Parameters:
validity_start (str) – Validity start of the CKD data, as
yyyymmddTHHMMSSrelease (int, default=1) – Release number, start at 1
pyspex.ccsds_io module
Contains the class CCSDSio to read SPEXone telemetry packets.
- class pyspex.ccsds_io.CCSDSio(file_list: list)[source]
Bases:
objectRead SPEXone telemetry packets.
- Parameters:
file_list (iterator to strings) – list of file-names, where each file contains parts of a measurement
Notes
The formats of the PACE telemetry packets are following the standards: CCSDS-131.0-B-3, CCSDS-132.0-B-2 and CCSDS-133.0-B-1.
This module is currenty restriced to telementry packets with APID: 0x350 (Science), 0x320 (NomHK) and 0x322 (DemHK).
A telemtry packet consist of a PRIMARY HEADER, SECONDARY HEADER (consist of a timestamp) and USER DATA with the actual telemetry packet data.
Doc: TMTC handbook (SPX1-TN-005), issue 12, 15-May-2020
The files with science and telemetry data needs to be in chronological order. However, you may mix science and housekeeping data as long as science data are chronological and housekeeping data are chronological.
Examples
>>> packets = () >>> with CCSDSio(['file1', 'file2', 'file3']) as ccsds: >>> while True: >>> # read one telemetry packet at a time >>> packet = ccsds.read_packet() >>> if packet is None: >>> # now we have read all files >>> break >>> >>> packets += (packet[0],) >>> >>> # combine segmented Science packages >>> science_tm = ccsds.science_tm(packets) >>> # now you may want to collect the engineering packages
- property ap_id: int | None
Returns SPEXone ApID.
- static fix_dem_hk24(dem_hk: ndarray) ndarray[source]
Correct 32-bit values in the DemHk.
Which originate from 24-bit values of the detector register parameters.
- Parameters:
dem_hk (numpy.ndarray) – SPEXone DEM housekeeping packages
- Returns:
SPEXone DEM housekeeping packages
- Return type:
numpy.ndarray
- static fix_sci_hk24(sci_hk: ndarray) ndarray[source]
Correct 32-bit values in the Science HK.
Which originate from 24-bit values in the detector register parameters. In addition:
- copy the first 4 bytes of DET_CHENA to DET_ILVDS - parameter 'REG_BINNING_TABLE_START' was writen in little-endian
- Parameters:
sci_hk (numpy.ndarray) – SPEXone Science telemetry packages
- Returns:
SPEXone Science telemetry packages
- Return type:
numpy.ndarray
- property grouping_flag: int | None
Returns grouping flag.
The meaning of the grouping flag values are:
00 continuation packet-data segment 01 first packet-data segment 10 last packet-data segment 11 packet-data unsegmented
- property packet_length: int | None
Returns size of packet data in bytes.
Value equals secondary header + user data (always odd)
- read_packet() ndarray | None[source]
Read next telemetry packet.
- Returns:
CCSDS packet data
- Return type:
numpy.ndarray
- science_tm(packets_in: tuple) tuple[source]
Combine segmented Science telemetry packages.
- Parameters:
packets_in (tuple) – Science or house-keeping telemetry packages
- Returns:
unsegmented Science telemetry packages
- Return type:
tuple
- property secnd_hdr_flag: bool | None
Returns flag indicating presence of a secondary header.
- static select_tm(packets_in: tuple, ap_id: int) tuple[source]
Select telemetry packages on SPEXone ApID.
- Parameters:
packets_in (tuple) – SPEXone telemetry packages
ap_id (int) – SPEXone ApID
- Returns:
selected telemetry packages
- Return type:
tuple
- property sequence_count: int | None
Returns sequence counter, rollover to zero at 0x3FFF.
- property type_indicator: int | None
Returns type of telemetry packet.
- property version_no: int | None
Returns CCSDS version number.
- pyspex.ccsds_io.hk_sec_of_day(ccsds_sec: ndarray, ccsds_subsec: ndarray, ref_day: datetime | None = None) ndarray[source]
Convert CCSDS timestamp to seconds after midnight.
- Parameters:
ccsds_sec (numpy array (dtype='u4')) – Seconds since 1970-01-01 (or 1958-01-01)
ccsds_subsec (numpy array (dtype='u2')) – Sub-seconds as (1 / 2**16) seconds
ref_day (datetime.datetime, optional) –
- Return type:
numpy.ndarray with sec_of_day
- pyspex.ccsds_io.img_sec_of_day(img_sec: ndarray, img_subsec: ndarray, img_hk: ndarray) tuple[datetime, float | Any][source]
Convert Image CCSDS timestamp to seconds after midnight.
- Parameters:
img_sec (numpy array (dtype='u4')) – Seconds since 1970-01-01 (or 1958-01-01)
img_subsec (numpy array (dtype='u2')) – Sub-seconds as (1 / 2**16) seconds
img_hk (numpy array) – DemHK telemetry packages
- Returns:
reference day: datetime, sec_of_day: numpy.ndarray
- Return type:
tuple
pyspex.ckd_io module
Contains the class CKDio to read SPEXone CKD.
References
- class pyspex.ckd_io.CKDio(ckd_file: Path, verbose: bool = False)[source]
Bases:
objectDefines a class to read SPEXone CKD parameters.
- Parameters:
ckd_file (str) – Name of CKD file
verbose (bool, default=False) – Be verbose
Examples
Read several CKD parameters:
>>> with CKDio(Path('SPX1_CKD.nc')) as ckd: >>> dark = ckd.dark() >>> fov = ckd.fov()
- dark() Dataset | None[source]
Read Dark CKD.
- Returns:
parameters of the SPEXone Dark CKD
- Return type:
xarray.Dataset
- date_created(compact: bool = False) str[source]
Return creation date of the CKD product.
- Parameters:
compact (bool) – return date in iso-format if not compact else return ‘YYYYmmddHHMMSS’
- fov() Dataset | None[source]
Read field-of-view CKD.
- Returns:
parameters of the SPEXone field-of-view CKD
- Return type:
xarray.Dataset
- property git_commit: str
Return git hash of repository spexone_cal, used to generate the CKD.
- nlin() Dataset | None[source]
Read non-linearity CKD.
- Returns:
parameters of the SPEXone non-linearity CKD
- Return type:
xarray.Dataset
- noise() Dataset | None[source]
Read Noise CKD.
- Returns:
parameters of the SPEXone Noise CKD
- Return type:
xarray.Dataset
- polarimetric() Dataset | None[source]
Read Polarimetric CKD.
- Returns:
parameters of the SPEXone Polarimetric CKD
- Return type:
xarray.Dataset
- prnu() DataArray | None[source]
Read PRNU CKD.
- Returns:
parameters of the SPEXone PRNU CKD
- Return type:
xr.DataArray
- property processor_version: str
Return the version of the spexone_cal program.
pyspex.dem_io module
Contains the class DEMio to read SPEXone CMV4000 detector data.
- class pyspex.dem_io.DEMio(flname: str)[source]
Bases:
objectRead SPEXone DEM output.
- Parameters:
flname (str) – filename with header or binary data of DEM measurement
Examples
Read data from a SPEXone DEM binary file:
> dem = DEMio(dem_file) > img_hk = dem.get_sci_hk() > img_data = dem.get_data()
- property exp_control: tuple
(inte_sync, exp_dual, exp_ext).
Register address: 41
- Type:
Exposure time control parameters
- get_data(numlines: int | None = None) ndarray[source]
Return data of a detector frame (numpy uint16 array).
- Parameters:
numlines (int, optional) – Provide number of detector rows when no headerfile is present
- Returns:
data of a detector frame, dtype np.uint16
- Return type:
numpy.ndarray
- get_sci_hk() ndarray[source]
Return Science telemetry.
A subset of MPS and housekeeping parameters.
- Return type:
numpy.ndarray
- property hdr: ndarray | None
Return DEM header as numpy compound array.
- property lvds_clock: bool
enable).
Register address: 82
- Type:
Return flag for LVDS clock (0
- Type:
disable, 1
- property number_channels: int
Return number of LVDS channels used.
- property number_lines: int
Return number of lines (rows).
Register address: [1, 2]
- property offset: int
Return digital offset including ADC offset.
Register address: [100, 101]
- property pga_gain: float
Return PGA gain (Volt).
Register address: 102
- pll_control() tuple[source]
Return PLL control parameters: pll_range, pll_out_fre, pll_div.
PLL_range: range (0 or 1) PLL_out_fre: output frequency (0, 1, 2 or 5) PLL_div: 9 (10 bit) or 11 (12 bit)
Register address: 116
- property temp_detector: int
Return detector temperature as raw counts.
Notes
Uncalibrated conversion: ((1184 - 1066) * 0.3 * 40 / 40Mhz) + offs [K]
- pyspex.dem_io.img_sec_of_day(img_sec: ndarray, img_subsec: ndarray, img_hk: ndarray) tuple[datetime, float | Any][source]
Convert Image CCSDS timestamp to seconds after midnight.
- Parameters:
img_sec (numpy array (dtype='u4')) – Seconds since 1970-01-01 (or 1958-01-01)
img_subsec (numpy array (dtype='u2')) – Sub-seconds as (1 / 2**16) seconds
img_hk (numpy array) – DemHK telemetry packages
- Returns:
reference day: datetime, sec_of_day: numpy.ndarray
- Return type:
tuple
pyspex.egse_db module
Collection of routines to access EGSE data generated by ITOS.
pyspex.hkt_io module
Contains the class HKTio to read PACE HKT products.
- class pyspex.hkt_io.HKTio(filename: Path)[source]
Bases:
objectClass to read housekeeping and navigation data from PACE-HKT products.
- Parameters:
filename (Path) – name of the PACE HKT product
Notes
This class has the following methods:
- reference_date -> datetime - set_reference_date() - coverage() -> tuple[datetime, datetime] - housekeeping(instrument: str) -> tuple[np.ndarray, ...] - navigation() -> dict
- housekeeping(instrument: str = 'spx') tuple[ndarray, ...][source]
Get housekeeping telemetry data.
- Parameters:
instrument ({'spx', 'oci', 'harp', 'sc'}, default='spx') – name of PACE instrument: ‘harp’: HARP2, ‘oci’: OCI, ‘sc’: spacecraft, ‘spx’: SPEXone.
Notes
Current implementation only works for SPEXone.
Get navigation data.
- property reference_date: datetime
Return reference date of all time_of_day variables.
Check time coverage of navigation data.
- Parameters:
l1a_file (Path) – name of the SPEXone level-1A product
xds_nav (xr.Dataset) – xarray dataset with PACE navigation data
Read navigation data from one or more HKT products.
- Parameters:
hkt_list (list[Path, ...]) – list of PACE-HKT products collocated with SPEXone measurements
- Returns:
xarray dataset with PACE navigation data
- Return type:
xr.Dataset
pyspex.lv0_io module
pyspex.lv1_args module
pyspex.lv1_gse module
Collect EGSE/OGSE data and/or add this to a SPEXone Level-1A product.
- class pyspex.lv1_gse.LV1gse(l1a_file: str)[source]
Bases:
objectAdds EGSE/OGSE data to a SPEXone Level-1A product.
- Parameters:
l1a_file (str) – Name of the Level-1A product
- check_egse(egse_data: ndarray) None[source]
Check consistency of OGSE/EGSE information during measurement.
- set_attr(name: str, value: Any) None[source]
Add attribute to group ‘gse_data’.
- Parameters:
name (str) –
value (anything(?)) –
- write_attr_act(angle: float, illumination: float | None = None) None[source]
Add act rotation angle as a group attribute.
- Parameters:
angle (float) –
illumination (float) –
- write_attr_alt(angle: float, illumination: float | None = None) None[source]
Add altitude rotation angle as a group attribute.
- Parameters:
angle (float) –
illumination (float) –
- write_attr_polarization(aolp: float, dolp: float) None[source]
Add polarization parameters AoLP & DoLP as group attributes.
- Parameters:
aolp (float) – Angle of linear polarization
dolp (float) – Degree of linear polarization
- write_egse(egse_time: ndarray, egse_data: ndarray, egse_attrs: dict) None[source]
Add EGSE parameters.
- Parameters:
egse_time (ndarray) –
egse_data (ndarray) –
egse_attrs (dict) –
- write_reference_diode(ref_time: ndarray, ref_data: ndarray, ref_attrs: dict) None[source]
Add data measured by the reference diode during the measurement.
- Parameters:
ref_time (ndarray) –
ref_data (ndarray) –
ref_attrs (dict) – fid[‘/ReferenceDiode/ref_diode’].__dict__
- write_reference_signal(signal: ndarray, stdev: ndarray) None[source]
Write reference detector signal and variance.
- Parameters:
signal (float) – Mean of signal level measured during the measurement
stdev (float) – standard deviation of signal level measured during the measurement
Notes
Used for non-linearity measurements.
pyspex.lv1_io module
pyspex.ogse_db module
Collect OGSE data in a database.
This module contain routines to read reference diode measurements and wavelength monitor data. These data are supposed to be written to a HDF5 database. From these database selected data can be added to a SPEXone Level-1a product.
- pyspex.ogse_db.add_ogse_ref_diode(ref_db: Path, l1a_file: Path) None[source]
Select reference data taken during a measurement and add to a L1A product.
- pyspex.ogse_db.add_ogse_wav_mon(ref_db: Path, l1a_file: Path) None[source]
Select reference data taken during a measurement and add to a L1A product.
pyspex.ogse_dolp module
Defines the spectral dependent DolP of the Moxtek polarizer.
pyspex.ogse_grande module
Defines the Grande spectrum for a given number of lamps, used at NASA GSFC.
pyspex.ogse_helios module
Defines the Helios spectrum, used at SRON.
pyspex.ogse_laser module
Read settings of the OPO laser used at NASA GSFC.
The OPO laser is used for ISRF and stray-light calibration measurements.
pyspex.tif_io module
Contains the class TIFio to read simulated SPEXone measurements.
pyspex.tlm module
SPXtlm can read telemetry house-keeping data from SPEXone.
- class pyspex.tlm.SPXtlm[source]
Bases:
objectAccess/convert parameters of SPEXone Science telemetry data.
Notes
This class has the following methods:
- set_coverage(coverage: tuple[datetime, datetime] | None) -> None - hk_hdr() -> np.ndarray | None - hk_tlm() -> np.ndarray | None - hk_tstamp() -> np.ndarray | None - sci_hdr() -> np.ndarray | None - sci_tlm() -> np.ndarray | None - sci_tstamp() -> np.ndarray | None - images() -> tuple | None - reference_date() -> datetime - time_coverage_start() -> datetime - time_coverage_end() -> datetime - from_hkt(flnames: Path | list[Path], *, instrument: str | None = None, dump: bool = False) -> None - from_lv0(flnames: Path | list[Path], *, file_format: str, tlm_type: str | None = None, debug: bool = False, dump: bool = False) -> None - from_l1a(flname: Path, *, tlm_type: str | None = None) -> None - set_selection(mode: str) -> None - gen_l1a(config: dataclass, mode: str) -> None - convert(key: str, tm_type: str = 'both') -> np.ndarray - units(key: str) -> str
- convert(key: str, tm_type: str = 'both') ndarray[source]
Convert telemetry parameter to physical units.
- Parameters:
key (str) – Name of telemetry parameter
tm_type ({'hk', 'sci', 'both'}, default 'both') – Default is to check if key is present in sci_tlm else hk_tlm
- Return type:
np.ndarray
- from_hkt(flnames: Path | list[Path], *, instrument: str | None = None, dump: bool = False) None[source]
Read telemetry data from a PACE HKT product.
- Parameters:
flnames (Path | list[Path]) – list of PACE_HKT filenames (netCDF4 format)
instrument ({'spx', 'sc', 'oci', 'harp'}, optional) –
dump (bool, default=False) – dump header information of the telemetry packages @1Hz for debugging purposes
- from_l1a(flname: Path, *, tlm_type: str | None = None) None[source]
Read telemetry data from SPEXone Level-1A product.
- Parameters:
flname (Path) – name of one SPEXone Level-1A product
tlm_type ({'hk', 'sci', 'all'}, optional) – select type of telemetry packages
- from_lv0(flnames: Path | list[Path], *, file_format: str, tlm_type: str | None = None, debug: bool = False, dump: bool = False) None[source]
Read telemetry data from SPEXone Level-0 product.
- Parameters:
flnames (Path | list[Path]) – list of CCSDS filenames
file_format ({'raw', 'st3', 'dsb'}) – type of CCSDS data
tlm_type ({'hk', 'sci', 'all'}, optional) – select type of telemetry packages. Note that we allways read the complete Level-0 producs.
debug (bool, default=False) – run in debug mode, read only packages heades
dump (bool, default=False) – dump header information of the telemetry packages @1Hz for debugging purposes
- property hk_hdr: ndarray | None
Return CCSDS header data of telemetry packages @1Hz.
- property hk_tlm: ndarray | None
Return telemetry packages @1Hz.
- property hk_tstamp: ndarray | None
Return timestamps of telemetry packages @1Hz.
- property images: tuple[ndarray, ...] | None
Return image-frames of Science telemetry packages.
- l1a_file(config: dataclass, mode: str) Path[source]
Return filename of Level-1A product.
- Parameters:
config (dataclass) – Settings for the L0->l1A processing.
mode ({'all', 'full', 'binned'}) – Select Science packages with full-frame image or binned images
- Returns:
Filename of Level-1A product.
- Return type:
Path
Notes
=== Inflight === L1A file name format, following the NASA … naming convention:
PACE_SPEXONE[_TTT].YYYYMMDDTHHMMSS.L1A[.Vnn].nc
- where
TTT is an optional data type (e.g., for the calibration data files) YYYYMMDDTHHMMSS is time stamp of the first image in the file Vnn file-version number (omitted when nn=1)
- for example (file-version=1):
[Science Product] PACE_SPEXONE.20230115T123456.L1A.nc [Calibration Product] PACE_SPEXONE_CAL.20230115T123456.L1A.nc [Dark science Product] PACE_SPEXONE_DARK.20230115T123456.L1A.nc
=== OCAL === L1A file name format:
SPX1_OCAL_<msm_id>[_YYYYMMDDTHHMMSS]_L1A_vvvvvvv.nc
- where
msm_id is the measurement identifier YYYYMMDDTHHMMSS is time stamp of the first image in the file vvvvvvv is the git-hash string of the pyspex repository
- property reference_date: datetime
Return date of reference day (tzone aware).
- property sci_hdr: ndarray | None
Return CCSDS header data of Science telemetry packages.
- property sci_tlm: ndarray | None
Return Science telemetry packages.
- property sci_tstamp: ndarray | None
Return timestamps of Science telemetry packages.
- set_coverage(coverage: tuple[datetime, datetime] | None) None[source]
Store or update the class attribute coverage.
- set_selection(mode: str) None[source]
Obtain image and housekeeping dimensions.
- Parameters:
mode ({'full', 'binned', 'all'}) –
- property time_coverage_end: datetime
Return a string for the time_coverage_end.
- property time_coverage_start: datetime
Return a string for the time_coverage_start.
pyspex.version module
Provide access to the software version as obtained from git.
Module contents
This is the SRON Python package pyspex.
It contains software to read PACE HKT products and SPEXone Level-0 products, and read/write SPEXone Level-1A products.