Mars Science Laboratory
(MSL)
Software Interface Specification
DAN Reduced Data Record (RDR)
Version 1.0
JPL D-38122
SIS-SCI018-MSL
DATE: October 11, 2012
Prepared by:
Karl Harshman, Lunar and Planetary Laboratory (University of Arizona)
Copyright 2013. All rights reserved.
Signatures
Custodian:
Karl Harshman, UofA Date
Approval:
PDS Geosciences Node Manager ______
Raymond Arvidson Date
Instrument Scientist:
Alberto Behar, JPL Date
OPGS CDE:
Maher Hanna, JPL Date
DAN Accommodation
Engineer Robert Barry, JPL Date
Concurrence:
Principal Investigator:
Igor Mitrofanov, IKI Date
|
DATE |
SECTIONS CHANGED |
REASON FOR CHANGE |
REVISION |
|
10/19/11 |
All |
Revision by PDS |
Draft |
|
1/20/12 |
6 Labels and format files |
Revision by DAN team |
Draft |
|
2/15/12 |
All |
Remaining questions from PDS |
Draft |
|
3/19/12 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1.3 Applicable Documents and Constraints
1.4 Relationships with Other Interfaces
2. Data Product Characteristics and Environment
2.3.4 Labeling and Identification
2.4 Standards Used in Generating Data Products
2.4.8 Data Storage Conventions
3. Detailed Data Product Specifications
3.2 Label and Header Descriptions
4.2 Applicable PDS Software Tools
6. ExampleS of DAN RDR PDS Labels
|
ASCII |
American Standard Code for Information Interchange |
|
CODMAC CETN CTN |
Committee on Data Management and Computation Counts of Epithermal Neutrons Counts of Thermal Neutrons |
|
DAN DE |
Dynamic Albedo of Neutrons DAN Electronics |
|
EDR FDD |
Experiment Data Record Functional Design Document |
|
FEI |
File Exchange Interface |
|
GDS |
Ground Data System |
|
IOT |
Instrument Operations Team |
|
JPL LPL MER |
Jet Propulsion Laboratory Lunar and Planetary Laboratory Mars Exploration Rovers |
|
MIPL |
Multi-mission image Processing Laboratory |
|
MSL NAIF |
Mars Science Laboratory Navigation and Ancillary Information Facility |
|
NASA |
National Aeronautics and Space Administration |
|
ODL |
Object Description Language |
|
ODS |
MSL’s Operations Data Store |
|
OPGS |
Operations Products Generation Sub-system |
|
PDS PHX PNG |
Planetary Data System Mars Phoenix Lander Pulsing Neutron Generator |
|
RDR |
Reduced Data Record |
|
RMC |
Rover Motion Counter |
|
RTO SCLK |
Real Time Operations (MSL terminology) Spacecraft Clock |
|
SIS |
Software Interface Specification |
|
TBD |
To Be Defined/Determined |
|
|
|
1. INTRODUCTION
1.1 Purpose and Scope
The purpose of this Software Interface Specification (SIS) is to provide suppliers and consumers of the Mars Science Lander’s (MSL) Dynamic Albedo of Neutrons (DAN) instrument with a detailed description of DAN’s Reduced Data Records (RDR). It does not cover any lower-level products, which are collectively known as Experimental Data Records (EDR). They will be covered by a separate SIS, provided by the OPGS (JPL).
This SIS includes descriptions of higher order DAN science and the special engineering data products.
The users for whom this SIS is intended for are the scientists who will analyze the data, including those associated with the MSL Project, DAN instrument engineers, and other users in the general planetary science community.
1.2 Contents
This SIS provides a very high level description of how DAN works/operates. It also describes, at high level, how the DAN data product is acquired by the instrument and how it is processed, formatted, labeled, and uniquely identified on the ground.
The document discusses standards used in generating the product and software tools which may be used to access the information. The data product structure and organization is described in sufficient detail to enable a user to read the product. Finally, an example of the product’s PDS label is provided.
1.3 Applicable Documents and Constraints
This Data Product SIS is responsive to the following MSL documents:
1. Mars Exploration Program Data Management Plan, Arvidson et al., Rev. 4.0, June 15, 2011.
2. Mars Science Laboratory Archive Generation, Validation and Transfer Plan, Joy Crisp, JPL D-35281, May 28, 2010.
3. Planetary Data System Standards Reference, JPL D-7669 part 2, version 3.8, February 27, 2009.
4. Planetary Archive Preparation Guide, JPL D-31224, Version 1.4, April 1, 2010, http://pds.jpl.nasa.gov/tools/archiving.html.
5. DAN FDD, JPL D-34220, MSL 375-1230
6. Mars Science Laboratory Software Interface Specification DAN Experiment Data Record, JPL D-38113, SIS-SCI1014-MSL, to be completed.
7. Mars Exploration Program Data Management Plan, Arvidson et al., Rev. 4.0, June 15, 2011.
8. MSL DAN Science Team and PDS Geosciences Node Interface Control Document (ICD), Susan Slavney and Alberto Behar, Version 3.0, July 26, 2010.
9. Mars Science Laboratory Dynamic Albedo of Neutrons RDR Archive Volume Software Interface Specification, Karl Harshman, September 28, 2011.
10. The Dynamic Albedo of Neutrons (DAN) Experiment for NASA's 2009 Mars Science Laboratory Litvak et al., 2008
This SIS is meant to be consistent with the contract negotiated between the MSL Project and the Instrument Principal Investigator (PI) in which reduced data records (RDR) and documentation (SIS) are explicitly defined as deliverable products.
1.4 Relationships with Other Interfaces
Changes to this DAN SIS document will affect the following products, software, and/or documents.
Table 1: Product and Software Interfaces to this SIS
|
Name
|
Type P-product S-software D-document |
Owner
|
|
DAN RDRs |
P |
U of A |
|
DAN_RDR_gen |
S |
U of A |
|
DAN UA database schema |
P |
U of A |
|
Other DAN Programs/Products/Documents |
P/S/D |
DAN Science Team |
2. Data Product Characteristics and Environment
2.1 Instrument Overview
The
Dynamic Albedo of Neutrons (DAN) is an active/passive neutron spectrometer that
measures the abundance and depth distribution of H- and OH-bearing materials
(e.g., adsorbed water, hydrated minerals) in a shallow layer (~1 m) of Mars'
subsurface along the path of the MSL rover. In active mode, DAN measures the
time decay curve (the "dynamic albedo") of the neutron flux from the
subsurface induced by its pulsing 14 MeV neutron source (Figure
1Figure
1 shows
an example). A detailed description of the DAN instrument and scientific
investigation can be found in Litvak et al. (2008). The experiment is
contributed by the Federal Space Agency of Russia.
Figure 1: Numerical simulations of the neutron count rate versus time (i.e., die-away curve) for the unshielded (left, CTN) and shielded (middle CETN) detectors as a function of water abundance. The right panel shows the difference between the two count rates.
The science objectives of the DAN instrument are as follows: 1) Detect and provide a quantitative estimation of the hydrogen in the subsurface throughout the surface mission; 2) Investigate the upper <0.5 m of the subsurface and determine the possible layering structure of hydrogen-bearing materials in the subsurface; 3) Track the variability of hydrogen content in the upper soil layer (~1 m) during the mission by periodic analysis; and 4) Track the variability of neutron radiation background (neutrons with energy < 100 keV) during the mission by periodic analysis.
The DAN instrument is expected to be used during rover traverses (e.g., during short stops at ~1 m intervals) and while the rover is parked. Short-duration (< 2 min) measurements will provide a rough estimate of the water-equivalent hydrogen distribution with an accuracy of ~1% by weight. Longer-duration (~30 min) measurements are necessary to derive the vertical distribution of water-equivalent hydrogen with an accuracy of 0.1-0.3% by weight.
DAN performs layering structure analyses of the Martian sub-surface, to measure the distribution of H- and OH-bearing materials, with a vertical resolution of < 1 m and horizontal resolutions of 0.5 - 100 m along the path of the rover.
DAN has three different modes of operations: Standby, Passive, and Active.
· STANDBY: Low voltage electronics are on, no science observations.
· PASSIVE: Background observations collected
· ACTIVE: Neutron pulses are produced and science observations collected.
Figure 2: DAN PNG, Pulsing Neutron Generator
Figure 3: DE, DAN Detector
Figure 4: Location of DAN DE and PNG on rover
2.2 Data Product Overview
DAN’s Reduced Data Records (RDR), NASA Level 1-A and 1B products, are generated by the DAN Team from the EDR data products, which are produced by OPGS at JPL.
DAN RDRs consist of clusters of calibrated data records, time-ordered, and grouped together. Each RDR will have a unique PDS label file, also known as a PDS detached label, which provides all the Meta and Ancillary data for each DAN data product. The pair of files, DAN binary data, and its PDS label, is collectively referred to as an RDR. In other words, an RDR is always made up of two files, named identically with only different file extensions, “.DAT” and “.LBL”.
As part of the RDR generation, older RDRs are replaced by newer and better calibrated versions. Since RDRs are generated from EDRs, if new EDRs become available RDRs may need to be regenerated. A regenerated version is identified in the product's PDS label by an incremented product version ID and a later product generation time.
2.3 Data Processing
2.3.1 Data Processing Level
The DAN RDR data set is assigned the PDS Data Set ID "MSL-M-DAN-3/4-RDR-V1.0". The processing level numbers are from the “Committee on Data Management and Computation (CODMAC) data level numbering system (Table 2).
Table 2: Processing Levels for Science Data Sets
|
NASA |
CODMAC |
Description |
|
|
Packet data |
Raw - Level 1 |
Telemetry data stream as received at the ground station, with science and engineering data embedded. |
|
|
Level-0 |
Edited - Level 2 |
Instrument science data (e.g., raw voltages, counts) at full resolution, time ordered, with duplicates and transmission errors removed. |
|
|
Level 1-A |
Calibrated - Level 3 |
Level 0 data that have been located in space and may have been transformed (e.g., calibrated, rearranged) in a reversible manner and packaged with needed ancillary and auxiliary data (e.g., radiances with the calibration equations applied). |
|
|
Level 1-B |
Resampled - Level 4 |
Irreversibly transformed (e.g., resampled, remapped, calibrated) values of the instrument measurements (e.g., radiances, magnetic field strength). |
|
|
Level 1-C |
Derived - Level 5 |
Level 1A or 1B data that have been resampled and mapped onto uniform space-time grids. The data are calibrated (i.e., radiometrically corrected) and may have additional corrections applied (e.g., terrain correction). |
|
|
Level 2 |
Derived - Level 5 |
Geophysical parameters, generally derived from Level 1 data, and located in space and time commensurate with instrument location, pointing, and sampling. |
|
|
Level 3 |
Derived - Level 5 |
Geophysical parameters mapped onto uniform space-time grids. |
|
2.3.2 Data Product Generation
The DAN RDR data products will be generated by the University of Arizona using the data processing software, DAN_RDR_gen. The RDR data products will be calibrated data derived from EDR data products and formatted according to this RDR SIS.
Data derived using SPICE kernels will be included in the RDR data products
2.3.3 Data Flow
The DAN RDR data products generated by the U of A will be created from EDR products provided from OPGS through MIPL’s File Exchange Interface (FEI) system and data derived using SPICE kernels acquired from NAIF. After a data validation period, the DAN RDR data products will be delivered to the Planetary Data System on a pre-determined schedule.
The DAN RDRs will be generated at least one month prior to the scheduled delivery date. The DAN RDR data will be reprocessed if new EDR data are received or if better calibrations are available.
The total estimated volume of the RDRs over the course of the MSL mission is 1 Gigabyte.
2.3.4 Labeling and Identification
There is a file-naming scheme adopted for the MSL image and non-image data products. The scheme applies to the EDR and RDR data products. The file naming schemes adhere to the Level II 36.3 filename convention approved by PDS in 2009.
The primary attributes of the filename nomenclature are:
a) Uniqueness - It must be unique unto itself without the file system’s directory path. This
protects against product overwrite as files are copied/moved within the
file system and external to the file system, if managed correctly.
b) Metadata - It should be comprised of metadata fields that keep file bookkeeping and
sorting intuitive to the human user. Even though autonomous file processing
will be managed via databases, there will always be a human-in-the-loop that
puts a premium on filename intuition. Secondly, the metadata fields should
be smartly selected based on their value to ground processing tools, as it is
less CPU-intensive to extract information from the filename than from the
label.
NOTE: The metadata information in the filename also resides in the product label.
The metadata fields have been selected based on MER and PHX lessons learned. In general, the metadata fields are arranged to achieve:
a) Sortability - At the beginning of the filename resides a primary time oriented field such as
Spacecraft Clock Start Count (SCLK). This allows for sorting of files on the
MSL file system by spacecraft data acquisition time as events occurred on
Mars.
b) Readability - An effort is made to alternate integer fields with ASCII character fields to
optimize differentiation of field boundaries for the human user.
Each DAN product has a detached PDS label associated with the DAN data file. The file-naming scheme for the DAN data products is:
|
where, |
|||||||||||||||
|
Instr |
= |
(2 alpha character) Instrument ID, denoting the source MSL science or engineering instrument that acquired the data. Valid values for Instrument ID’s are: ”DN” - DAN |
|||||||||||||
|
Config |
= |
(1 alphanumeric) Instrument Configuration, an operational attribute of the Instrument that assists in characterizing the data. Valid values: |
|||||||||||||
|
|
Instrument |
Configuration |
|||||||||||||
|
Values |
Description |
||||||||||||||
|
|
DAN (“DN”) |
it is being set to either “A” or “B” |
A-side configuration, B-side configuration |
||||||||||||
|
|
|
||||||||||||||
|
spec
|
= |
(1 character) Special Processing flag, applicable to RDRs on a case-by-case basis. |
|||||||||||||
|
|
Special Processing |
EDR Value |
RDR Value |
|
|||||||||||
|
|
None know |
“_” |
“_” |
|
|||||||||||
|
|
Special method types A - Z |
n/a |
“A” - “Z” |
|
|||||||||||
|
|
|
||||||||||||||
|
sclk |
= |
(9 alphanumeric) Spacecraft Clock Start Count, in units of seconds. Which specific SCLK is used depends on the instrument but is generally expected to be the time the data was acquired. For the DAN EDRs the SCLK provided in the .emd (Data product metadata file) file will be used.
|
|||||||||||||
|
|
The valid values, in their progression, are as follows (non-Hex): Range 000000000 thru 999999999 - “000000000”, “000000001”, … “999999999” Range 1000000000 thru 1099999999 - “A00000000”, “A00000001”, … “A99999999” Range 1100000000 thru 1199999999 - “B00000000”, “B00000001”, … “B99999999” · · · Range 3500000000 thru 3599999999 - “Z00000000”, “Z00000001”, … “Z99999999” |
||||||||||||||
|
|
|
||||||||||||||
|
prod |
= |
(3 char) Product Type identifier. This field has the following rule-of-thumb: Beginning "E" - Type of EDR, which is a raw data product. Beginning “R” - Type of RDR, which is a reduced product.
|
|||||||||||||
|
|
Valid values for DAN Product identifiers are listed below: |
||||||||||||||
|
|
Product Type Description |
Value |
|||||||||||||
|
|
Active EDR |
“EAC” |
|||||||||||||
|
|
Passive EDR |
“EPA” |
|||||||||||||
|
|
Standby EDR |
“EST” |
|||||||||||||
|
|
Derived Engineering RDR |
“REN” |
|||||||||||||
|
|
Derived Passive RDR |
“RPA” |
|||||||||||||
|
|
Derived Active RDR |
“RAC” |
|||||||||||||
|
|
Averaged Passive RDR |
“RAP” |
|||||||||||||
|
|
Averaged Active RDR |
“RAA” |
|||||||||||||
|
sol |
= |
NOTE: If the first character is an underscore (“_”) then the remaining 3 characters denote the day of year (DOY). This format will be used during cruise. |
|
||||||||||||
|
|
The valid values, in their progression, are as follows (non-Hex): Range 0000 thru 9999 - “0000”, “0001”, … “9999” Range 10000 thru 10999 - “A000”, “A001”, … “A999” Range 11000 thru 11999 - “B000”, “B001”, … “B999” · · · Range 35000 thru 35999 - “Z000”, “Z001”, … “Z999” |
|
|||||||||||||
|
site |
= |
(3 alphanumeric) Site location count, from the RMC. This field has the following rules-of-thumb: If value is any 3 alphanumeric characters then content represents Site index extracted from RMC, or 3 underscores (denoting value is out-of-range),.
|
|||||||||||||
|
|
The valid Site values, in their progression, are as follows (non-Hex): Range 000 thru 999 - “000”, “001”, … “999” Range 1000 thru 1099 - “A00”, “A01”, … “A99” Range 1100 thru 1199 - “B00”, “B01”, … “B99” · · · Range 3500 thru 3599 - “Z00”, “Z01”, … “Z99” |
||||||||||||||
|
drive |
= |
(4 alphanumeric) Drive (position-within-Site) location count, from the Rover Motion Counter (RMC). This field has the following rules-of-thumb: If value is any 4 alphanumeric charactersthen content represents Drive index extracted from RMC, or 4 underscores (denoting value is out-of-range),.
|
|||||||||||||
|
|
The valid Drive values, in their progression, are as follows (non-Hex): Range 0000 thru 9999 - “0000”, “0001”, … “9999” Range 10000 thru 10999 - “A000”, “A001”, … “A999” Range 11000 thru 11999 - “B000”, “B001”, … “B999” · · · Range 35000 thru 35999 - “Z000”, “Z001”, … “Z999” Range 36000 thru 36099 - “AA00”, “AA01”, … “AA99” Range 36100 thru 36199 - “AB00”, “AB01”, … “AB99” · · · Range 38500 thru 38599 - “AZ00”, “AZ01”, … “AZ99” Range 38600 thru 38699 - “BA00”, “BA01”, … “BA99” Range 38700 thru 38799 - “BB00”, “BB01”, … “BB99” · · · Range 41100 thru 41199 - “BZ00”, “BZ01”, … “BZ99” Range 41200 thru 41299 - “CA00”, “CA01”, … “CA99” · · · Range 65400 thru 65499 - “LI00”, “LI01”, … “LI99” Range 65500 thru 65535 - “LJ00”, “LJ01”, … “LJ35” |
||||||||||||||
|
req id |
= |
(7 alphanumeric) - Request ID In the case of DAN this files is currently not being used, and therefore it will always have a value of all underscores (“_______”). It is maintained as a placeholder, and to maintain consistency with MSL file-naming nomenclature. Valid values: “_______” – N/A. a) |
|||||||||||||
|
who |
= |
(1 alpha character) Product Producer ID identifies the institution that generated the product. This field has the following rules-of-thumb: Producer - If value is “P” (for Flight) or “Y” (for Engineering), the provider of the product is the Principal Investigator. Except for MIPL as the provider (“M” for Flight or “Z” for Engineering), the remaining characters are assigned to Co-investigator providers at the discretion of the P.I. and will be identified in due time. Within the instrument of the P.I., characters are unique. Across instruments, characters are reusable. See the following table of valid values: |
|||||||||||||
|
|
Venue |
by Producer |
|||||||||||||
|
|
Flight Model |
Eng. Model |
|||||||||||||
|
|
“M” |
“Z” |
MIPL (OPGS at JPL) |
||||||||||||
|
|
“P” |
“Y” |
Principal Investigator of Instrument … |
||||||||||||
|
|
|
|
Instrument Principal Investigator SAM GSFC (Goddard, MD) REMS Ministry of Education & Science (Spain) DAN Federal Space Agency (Russia) RAD SwRI (Boulder, CO) CheMin Ames Research Center (Mountain View, CA) APXS Max-Planck Institute (Germany) SA/SPaH JPL |
||||||||||||
|
|
“A” - “L” |
“Q” - “X” |
Co-Investigators (to be identified by P.I. per instrument) A for UA generated. |
||||||||||||
|
ver |
= |
(1 alphanumeric) Version identifier. |
|||||||||||||
|
|
The valid values, in their progression, are as follows (non-Hex): Range 1 thru 10 - “1”, “2”, … “9”, “0” Range 11 thru 36 - “A”, “B”, … “Z” Range 37 and higher - “_” (underscore) The Version number increments by one whenever an otherwise-identical filename would be produced. Note that not every version need exist, e.g. versions 1, 2 and 4 may exist but not 3. In general, the highest-numbered Version represents the “best” version of that product. NOTE: To be clear, this field increments independently of all fields, including the Special Processing field. |
||||||||||||||
|
= |
(2 to 3 alpha characters) Product type extension. |
||||||||||||||
|
|
Valid values for nominal operations data products: |
||||||||||||||
|
“DAT” “LBL”
|
- -
|
Non-imaging binary instrument data Detached label in PDS format
|
|||||||||||||
|
Example #1: DNA_351797691RPA_0550000000_______P1.DAT |
||||
|
Where, |
||||
|
instr config spec sclk prod sol site drive reqestID who ver ext |
= = = = = = = = = = = = |
“DN” “A” “_” “_” “RPA” “_055” “000” “0000” “_______” “P” “1” “DAT” |
= = = = = = = = = = = = |
DAN TBD (Instrument configuration for now) TBD Spacecraft Clock Start Count of 351797691 sec. “R”-RDR, “PA”-Passive Day-of-Year 55 (Cruise) Site 0 Position (Drive) 0 unused Produced by LPL at the University of Arizona Version 1 Indicates this file contains data, and not label info (LBL). |
2.4 Standards Used in Generating Data Products
2.4.5 PDS Standards
The DAN data product complies with PDS standards for file formats and labels, as specified in the PDS Standards Reference .(Applicable Document 3)
2.4.6 Time Standards
The following time standards and conventions are used throughout this document, as well as the MSL project for planning activities and identification of events.
|
Time Format |
Definition |
|
SCET |
Spacecraft event time. This is the time when an event occurred on-board the spacecraft, in UTC. It is usually derived from SCLK. |
|
SCLK |
Spacecraft Clock. This is an on-board 64-bit counter, in units of nano-seconds and increments once every 100 milliseconds. Time zero corresponds to midnight on 1-Jan-1980. |
|
ERT |
Earth Received Time. This is the time when the first bit of the packet containing the current data was received at the Deep Space (DSN) station. Recorded in UTC format. |
|
Local Solar Time |
Local Solar Time (LST). This is the local solar time defined by the local solar days (sols) from the landing date using a 24 “hour” clock within the current local solar day (HR:MN:SC). Since the Mars day is 24h 37m 22s long, each unit of LST is slightly longer than the corresponding Earth unit. LST is computed using positions of the Sun and the landing site from SPICE kernels. If a landing date is unknown to the program (e.g. for calibration data acquired on Earth) then no sol number will be provided on output LST examples: SOL 12 12:00:01 SOL 132 01:22:32.498 SOL 29 |
|
RCT |
Record Creation Time. This is the time when the first telemetry packet, containing a give data, set was created on the ground. Recorded in UTC format.
|
|
True Local Solar Time |
This is related to LST, which is also known as the mean solar time. It is the time of day based on the position of the Sun, rather than the measure of time based on midnight to midnight “day”. TLST is used in all MIPL/OPGS generated products. |
|
SOL |
Solar Day Number, also known as PLANET DAY NUMBER in PDS label. This is the number of complete solar days on Mars since landing. The landing day therefore is SOL zero. |
2.4.7 Coordinate Systems
The following coordinate systems are used within the project to refer to the position of the Rover and its instruments.
|
Coordinate System |
Origin |
Orientation |
|
Local Level |
Same as payload frame, and it moves with the Rover |
+X North +Z down along gravity vector +Y East |
|
Payload Frame |
At the shoulder of the Robotic Arm. Attached and moves with the Rover |
+X along Rover –X ( point out into the work space) +Z down along Rover (vertical axis) +Y along Rover -Y
|
|
Site Frame |
Same as payload frame when first defined and never moves relative to Mars. Possible to define multiple site frames in case the Rover moves/slips. |
Same as local level |
|
Rover Frame |
Attached to Rover |
Attached to Rover |
2.4.8 Data Storage Conventions
DAN RDR products will be stored as binary files containing IEEE floating-point values, big-endian integers and ASCII character strings. For every DAN RDR there will be a detached PDS label file. The detached PDS labels for DAN RDRs are stored as ASCII text, with each keyword definition terminated by ASCII carriage-return and line-feed characters. The RDR products are defined in section 5. The PDS label keywords are described in Appendix-A.
Each DAN RDRs will contain fixed length records but the length can differ depending on the type of RDR. When necessary, records will be padded with extra bytes of 0x00. Label keywords will provide necessary information to determine the size and organization of the binary records.
2.5 Data Validation
The DAN RDRs, as with all other MSL RDRs, are subject to PDS peer review.
Validation of DAN RDR products during production will be performed according to specifications in the MSL Archive Plan. The DAN Team will validate the science content of the data products, and the PDS Geosciences Node will validate the products for compliance with PDS standards and for conformance with the design specified in this SIS.
Validation of the MSL RDRs will fall into two primary categories: automated and manual. Automated validation will be performed on every RDR product produced for the mission. Manual validation will only be performed on a subset.
Validation of DAN RDRs will be performed by the DAN Team as a part of the archiving process, and will be done simultaneously with the archive volume validation. Validations performed will include such things as verification that the checksum in the label matches a calculated checksum for the data product (i.e., that the data product included in the archive is identical to that produced by the real-time process), a validation of the PDS syntax of the label, a check of the label values against the database and against the index tables included on the archive volume, and checks for internal consistency of the label items. The latter include such things as verifying that the product creation date is later than the earth received time. As problems are discovered and/or new possibilities identified for automated verification, they will be added to the validation procedure.
Manual validation of the data will be performed both as spot-checking of data throughout the life of the mission, and comprehensive validation of a subset of the data (for example, a couple of days' worth of data). A human will view these products. The DAN Team will validate the science content of the data products, and the PDS Geosciences Node will validate the products for compliance with PDS standards and for conformance with the design specified in this SIS.
3. Detailed Data Product Specifications
When powered, the DAN instrument is in one of three states: STANDBY, PASSIVE, or ACTIVE. Onboard, the DAN instruments can generate different data packets depending on the state of the instrument.
Five RDR products will be produced, DERIVED_ENG, DERIVED_PASSIV, DERIVED_ACTIVE, AVERAGED_PASSIV and AVERAGED_ACTIVE Each is defined in the following sections. The DAN RDR file naming convention will uniquely identify each data product. See Section 2.3.4 for detail description of the DAN file naming convention.
The structure of the DAN RDR consists of a detached ASCII PDS label and a binary data file as shown in Figure 3.1.
|
Detached ASCII PDS Label |
|
DAN Binary Data File () |
Figure 3.1: The DAN RDR consists of two files.
3.1 Data Format Descriptions
Appendix A includes the table for binary data column definitions to the byte level.
3.1.1 DERIVED_ENGINEERING
Derived Engineering are the raw DN values from the 6 temperature sensors converted to Celsius. Each EDR record contains these temperatures and therefore a time series dataset can be constructed over the entire time DAN is operating. This data product also contains the states of the two high voltages for the detectors and also the two discriminator values for each of the two detectors (CTN and CETN).
3.1.2 DERIVED_PASSIVE
The Derived Passive data products are in a one to one correspondence with the EDR Passive data products. Each data product has counts for each detector summed over channels 1 to 16 and background counts for the collection period as well as location and time information.
3.1.3 DERIVED ACTIVE
The Derived Active data set is in a one to one correspondence with the EDR Active data set. This data set has the detector counts summed over channels 1 to 16 for each of the 64 spectra in the EDR set. It also contains the background counts for each of the 64 spectra as well as location and time information.
3.1.4 AVERAGED PASSIVE
An Averaged Passive data product contains the average counts over some time period of passive measurements. Derived Passive data products are used to calculate the average of the given time period. The time periods will coincide with passive measurements during a travser from location A to location B. This data product also contains the background counts for the given time period as well as an error and normalization factor. It also contains spatial and temporal information over which the data was taken.
3.1.5 AVERAGED ACTIVE
An Averaged Active data product contains the average counts over some time period of active measurements. The Derived Active data products are used to calculate the average of the given time period. The time periods will coincide with active measurements during a time the rover is stopped. This data product also contains the background counts for the given time period as well as an error and normalization factor. It also contains spatial and temporal information over which the data was taken.
3.2 Label and Header Descriptions
3.2.6 PDS Label
DAN products have detached PDS labels. A PDS label is object-oriented and describes the objects in the data file. The PDS label contains keywords for product identification and for table object definitions. The label also contains descriptive information needed to interpret or process the data objects in the file.
PDS labels are written in Object Description Language (ODL). PDS label statements have the form of "keyword = value". Each label statement is terminated with a carriage return character (ASCII 13) and a line feed character (ASCII 10) sequence to allow the label to be read by many operating systems. Pointer statements with the following format are used to indicate the location of data objects in the file:
^Object = location
where the carat character (^, also called a pointer) is followed by the name of the specific data object. The location is the starting record number for the data object within the file.
Each PDS keyword defined for the DAN label will always be included in the label. If a keyword does not have a value, a value of N/A will be given as the keyword value.
Per PDS rules, “N/A” is used when a keyword exists, but it does not apply to a particular data, and “UNKNOWN” or UNK is used when a value of an applicable keyword cannot be determined at the time the PDS label was generated.
3.2.7 PDS Table Object
The TABLE object is a uniform collection of rows containing ASCII or binary values stored in columns. The INTERCHANGE_FORMAT keyword is used to distinguish between TABLEs containing only ASCII columns and those containing binary data. DAN RDR products are stored as binary tables. Each field is defined as a fixed-width COLUMN object; the value of the COLUMNS keyword is the total number of COLUMN objects defined in the label. All TABLE objects must have fixed-width records.
4. Applicable Software
4.1 Utility Programs
None
4.2 Applicable PDS Software Tools
PDS-labeled images and tables can be viewed, but not manipulated with the program NASAView, developed by the PDS Engineering Node, and are available for a variety of computer platforms. They can be obtained directly from the PDS at http://pds.nasa.gov/tools/nasa-view.shtml.