Lunar Reconnaissance Orbiter

 

LUNAR EXPLORATION NEUTRON DETECTOR

EDR/RDR ARCHIVE VOLUME

SOFTWARE INTERFACE

SPECIFICATION

 

June 1, 2016

 

 

Prepared by:

 

___________________________________

Karl Harshman

LEND SOC Lead

 

 

Approved by:

 

___________________________________

William Boynton

Co-Investigator, Lunar Exploration Neutron Detector

 

 

Approved by:

 

___________________________________

Igor Mitrofanov

Principal Investigator, Lunar Exploration Neutron Detector

 

 

Approved by:

 

___________________________________

Edwin Grayzeck

PDS Program Manager

 

TABLE OF CONTENTS

 

1. Introduction................................................................................................................................... 1

1.1. Purpose and Scope................................................................................................................ 1

1.2. Content Overview.................................................................................................................. 1

1.3. Applicable Documents and Constraints................................................................................... 1

1.4. Relationships with Other Interfaces......................................................................................... 2

2. Archive Volume Contents.............................................................................................................. 2

2.1. Root Directory Contents........................................................................................................ 3

2.2. Data Directory Contents and Naming..................................................................................... 3

2.3. Index Directory Contents....................................................................................................... 4

2.4. Document Directory Contents................................................................................................ 4

2.5. Catalog Directory Contents.................................................................................................... 5

2.6. Label Directory Contents....................................................................................................... 6

2.7. Calibration Directory Contents............................................................................................... 6

3. Archive Volume Format................................................................................................................. 6

3.1. File Formats.......................................................................................................................... 7

3.1.1. Document File Format................................................................................................. 7

3.1.2. Tabular File Format..................................................................................................... 7

3.1.3. PDS Label Format...................................................................................................... 7

3.1.4. Software File Format................................................................................................... 8

3.1.5. Catalog File Format..................................................................................................... 8

3.1.6. Science Data File Formats........................................................................................... 8

4. Archive Volume Generation........................................................................................................... 9

4.1. Data Transfer and Validation Methods................................................................................... 9

4.2. Data Product Sizes and Delivery Rates................................................................................... 9

4.3. Interface Media Characteristics.............................................................................................. 9

4.4. Backup and Duplicates.......................................................................................................... 9

4.5. Labeling and Identification...................................................................................................... 9

5. Support Staff and Cognizant Persons........................................................................................... 10

 

TABLES AND FIGURES

 

Table 1 – Standard Product Sizes and Delivery Rates........................................................................ 9

 

 

DOCUMENT CHANGE LOG

 

Change	Date	Affected Portions
Initial Draft	5/2/07	All
Corrected name of reference document 1	6/25/07	1.3
Corrected reference to correct document	6/25/07	1.4
Updates from end-to-end testing	05/06/08	All
Additional updates from end-to-end testing	07/14/08	All
Added DLX data product	06/01/16	1.2, 2.2, 2.6

 

ACRONYMS AND ABBREVIATIONS

 

GLOSSARY

 

Archive – An archive consists of one or more data sets along with all the documentation and ancillary information needed to understand and use the data. An archive is a logical construct independent of the medium on which it is stored.

Archive Volume, Archive Volume Set – A volume is a unit of media on which data products are stored; for example, one CD-ROM or DVD-ROM. An archive volume is a volume containing all or part of an archive; that is, data products plus documentation and ancillary files. When an archive spans multiple volumes, they are called an archive volume set. Usually the documentation and some ancillary files are repeated on each volume of the set, so that a single volume can be used alone. 

Catalog Information – Descriptive information about a data set (e.g. mission description, spacecraft description, instrument description), expressed in Object Description Language (ODL) which is suitable for loading into a PDS catalog.

Data Product – A labeled grouping of data resulting from a scientific observation, usually stored in one file. A product label identifies, describes, and defines the structure of the data. An example of a data product is a planetary image, a spectrum table, or a time series table.

Data Set – An accumulation of data products. A data set together with supporting documentation and ancillary files is an archive.

Standard Data Product – A data product generated in a predefined way using well-understood procedures, processed in "pipeline" fashion. Data products that are generated in a non­standard way are sometimes called special data products.

 

1. Introduction

1.1. Purpose and Scope

This Software Interface Specification is intended to be used by those who wish to understand the format and content of the Lunar Reconnaissance Orbiter, LRO Lunar Exploration Neutron Detector, LEND Experiment Data Record and LEND Reduced Data Record, EDR/RDR Archive. Typically, these individuals would be software engineers, data analysts, or planetary scientists.      

The specifications in this document apply to all LRO LEND EDR/RDR standard product archive volumes that are generated by the LRO LEND Project.

1.2. Content Overview

The LRO LEND EDR/RDR archive includes the following eight types of products.

LEND_SCI_DATA                                         Neutron Detector science data

LEND_HK_DATA                                          Neutron Detector housekeeping data

LEND_CONVERTED_HK_DATA                Reduced housekeeping data.

LEND_RECTIFIED_SCIENCE_DATA         Temporally and spatially rectified science data.

LEND_DERIVED_DATA                               Background corrected Neutron Data

LEND_EXTENDED_DERIVED_DATA         Background corrected Neutron Data

LEND_AVERAGED_COUNTS_DATA         Normalized, averaged Neutron Data

LEND_AVERAGED_FLUX_DATA   Normalized, averaged Neutron Data

 

The EDR/RDR data products are generated by the LEND SOC team at the University of Arizona and delivered to the PDS Geosciences Node at Washington University, St. Louis.

This Software Interface Specification (SIS) describes the format, content, and generation of the LRO LEND EDR/RDR Archive. Section 2, Archive Volume Generation, describes the procedure for transferring data products to archive media. Section 3, Archive Volume Contents, describes the structure of the archive volumes and the contents of each file. Section 4, Archive Volume Format, describes the file formats used on the archive volumes. Finally, Section 5, Support Staff and Cognizant Persons, lists the individuals responsible for generating the archive volumes.

1.3. Applicable Documents and Constraints

This Data Product SIS is responsive to the following LRO documents:

1. Lunar Reconnaissance Orbiter Project Data Management and Archive Plan, 431-PLAN-00182
2. Lunar Reconnaissance Orbiter Project, LEND Instrument Team Data Management and Archive Plan.
3. Lunar Exploration Neutron Detector EDR (NASA Level 0) Software Interface Specification, July 15, 2008.
4. Lunar Exploration Neutron Detector RDR (NASA Level 1) Software Interface Specification, July 15, 2008.

This SIS is also consistent with the following Planetary Data System documents:

5. Planetary Data System Archive Preparation Guide, August 29, 2006, Version 1.1, JPL D-31224.
6. Planetary Data System Standards Reference, March 20, 2006, Version 3.7. JPL D-7669, Part 2.
7. ISO 9660-1988, Information Processing - Volume and File Structure of CD-ROM for Information Exchange, April 15, 1988.

Finally, this SIS is meant to be consistent with the contract negotiated between the 2008 Lunar Reconnaissance Orbiter and the LEND Principal Investigator (PI) in which reduced data records and documentation are explicitly defined as deliverable products.

1.4. Relationships with Other Interfaces

This Archive Volume SIS could be affected by changes to the design of the LRO LEND standard data products (Applicable Documents 2, 3 and 4).

2. Archive Volume Contents

This section describes the contents of the LRO LEND Archive volumes, including the file names, file contents, file types, and organization responsible for providing the files.

LEND EDR/RDR archive volumes include the following directories.

 

Directory                                              Contents

Volume root directory                         Introduction and errata text files

 

CALIBRATION                                    Calibration document files

 

CATALOG                                            Descriptions of data set, instruments, spacecraft, and mission as

found in the PDS Catalog

 

DOCUMENT                                        Documentation files

 

INDEX                                                   Volume and cumulative index tables

 

LABEL                                                   Descriptions of data file formats, referenced by PDS labels

 

DATA                                                    Data files and their detached PDS labels

 

SOFTWARE                                         Software for viewing the archive and the detached PDS labels

 

The contents of these directories are described in detail in Sections 2.1 to 2.7.

 

2.1. Root Directory Contents

Files in the Root Directory include an overview of the archive, a description of the volume for the PDS Catalog, and a list of errata or comments about the archive. The following files are contained in the Root Directory.

 

File Name	File Contents	File Provided By
AAREADME.TXT	Volume content and format information	LEND SOC
ERRATA.TXT	A cumulative listing of comments and updates concerning all archive volumes published to date	LEND SOC
VOLDESC.CAT	A description of the contents of this volume in a PDS format readable by both humans and computers	LEND SOC

2.2. Data Directory Contents and Naming

LEND EDR/RDR data products are aggregated by flight day; i.e., one product contains one day’s worth of data. Flight day is defined to be midnight to midnight UTC. Data for each flight day will be in a subdirectory named in the format YYYYMMDD, indicating the date at the end of the data acquisition period. Within each subdirectory will be eight PDS labels and associated data files for the eight product types listed in section 1.2.

 

This schematic shows an example of the nesting of directories for data acquired beginning at midnight March 18, 2009 through March 19 2009.

 

<Volume Root>

-------------------

            |

-DATA

            |- 2009

            |           |

            |           |-- 20090319

            |           |           |

            |           |           | -- <LEND_SCI_DATA files acquired March 18-19, 2009>

            |           |           |

            |           |           | -- <LEND_HK_DATA files acquired March 18-19, 2009>

            |           |           |

            |           |           | --<LEND_CONVERTED_HK_DATA files acquired March 18-19,      

                                          2009>

            |           |           |

            |           |           | --<LEND_RECTIFIED_SCIENCE_DATA files acquired March 18-19,                                            2009>

            |           |           |

            |           |           | --<DERIVED_LEND_DATA files acquired March 18-19, 2009>

            |           |           |

            |           |           | --<EXTENDED_DERIVED_LEND_DATA files acquired March 18-19, 2009>

            |           |           |

            |           |           | --<AVERAGED_LEND_DATA files acquired March 18-19, 2009>

            |           |           |

            |           |           | --<AVERAGED_NEUTRON_FLUX_LEND_DATA files acquired March 18-19, 2009>

            |           |           |

 

File Name	File Contents	File Provided By
LEND_ EDR_SCI_ 20090319.DAT,.LBL	Neutron Spectra	LEND SOC
LEND_ EDR_HK_ 20090319.DAT,.LBL	Housekeeping data	LEND SOC
LEND_ RDR_ALD_ 20090319.DAT,.LBL	Averaged Spectra data	LEND SOC
LEND_ RDR_ALF_ 20090319.DAT,.LBL	Averaged Neutron Flux data	LEND SOC
LEND_ RDR_CHK_ 20090319.DAT,.LBL	Rectified Housekeeping data	LEND SOC
LEND_ RDR_DLD_ 20090319.DAT,.LBL	Derived Spectra data	LEND SOC
LEND_ RDR_DLX_ 20090319.DAT,.LBL	Extended Derived Spectra data	LEND SOC
LEND_ RDR_RSCI_ 20090319.DAT,.LBL	Spectra and spatial data	LEND SOC

2.3. Index Directory Contents

Files in the Index Directory are provided to help the user locate products on this archive volume and on previously released volumes in the archive. The following files are contained in the Index Directory.

 

File Name	File Contents	File Provided By
INDXINFO.TXT	A description of the contents of this directory	LEND SOC
INDEX.TAB	A table listing all data products on this volume	LEND SOC
INDEX.LBL	A PDS detached label that describes INDEX.TAB	LEND SOC

2.4. Document Directory Contents

The Document Directory contains documentation to help the user understand and use the archive data. The following files are contained in the Document Directory.

 

File Name	File Contents	File Provided By
DOCINFO.TXT	A description of the contents of this directory	LEND SOC
LEND_EDR_SIS.PDF	The Data Product SIS as a PDF file	LEND SOC
LEND_EDR_SIS.HTM	The Data Product SIS as an HTML file	LEND SOC
LEND_EDR_SIS.LBL	A PDS detached label that describes LEND_EDR_SIS.PDF and HTML	LEND SOC
LEND_RDR_SIS.PDF	The Data Product SIS as a PDF file	LEND SOC
LEND_RDR_SIS.HTM	The Data Product SIS as an HTML file	LEND SOC
LEND_RDR_SIS.LBL	A PDS detached label that describes LEND_RDR_SIS.PDF and HTML	LEND SOC
LEND_ARCHIVE_VOL_SIS.PDF	The Archive Volume SIS (this document) as a PDF file	LEND SOC
LEND_ARCHIVE_VOL_SIS.HTM	The Archive Volume SIS (this document) as an HTML file	LEND SOC
LEND_ARCHIVE_VOL_SIS.LBL	A PDS detached label that describes both LEND_ARCHIVE_VOL_SIS.PDF and HTML	LEND SOC
REFERENCES.TXT	A text file containing references to all relevant LEND publications and conference abstracts	LEND SOC
PDSDD.FUL	The complete PDS Data Dictionary	Geosciences
PDSDD.IDX	PDS Data Dictionary Index file	Geosciences
PDSDD.LBL	A PDS detached label that describes PDSDD.FUL	Geosciences

Additional documents may be added.

2.5. Catalog Directory Contents

The files in the Catalog Directory provide a top-level understanding of the mission, spacecraft, instruments, and data sets. The files in this directory are coordinated with the PDS data engineer, who is responsible for loading them into the PDS catalog. The following files are found in the Catalog Directory.

 

File Name	File Contents	File Provided By
CATINFO.TXT	A description of the contents of this directory	LEND SOC
EDR_DS.CAT	EDR Data set information for the PDS catalog	LEND SOC
RDR_DS.CAT	RDR Data set information for the PDS catalog	LEND SOC
INSTHOST.CAT	Instrument host (i.e., spacecraft) information for the PDS catalog	LRO
INST.CAT	Instrument information for the PDS catalog	LEND SOC
MISSION.CAT	Mission information for the PDS catalog	LRO
PERSON.CAT	Personnel information for the PDS catalog (Team and PDS personnel responsible for generating the archive)	LEND SOC
REF.CAT	References mentioned in other *.CAT files	LEND SOC

2.6. Label Directory Contents

The Label Directory contains files that describe data format and organization. These files are referred to in the PDS labels that accompany the data products. They are "include" files that are intended to be parsed as if they were part of the PDS labels that refer to them.  The following files are contained in the Label Directory.

 

File Name	File Contents	File Provided By
LABINFO.TXT	A description of the contents of this directory	LEND SOC
LEND_EDR_SCI.FMT	Neutron spectra data column descriptions	LEND SOC
LEND_EDR_HK.FMT	Housekeeping and Engineering data column descriptions	LEND SOC
LEND_RDR_RSCI.FMT	Spatial rectified Neutron spectra data column descriptions	LEND SOC
LEND_RDR_CHK.FMT	Converted Engineering data column descriptions	LEND SOC
LEND_RDR_DLD.FMT	Derived Neutron spectra data column descriptions	LEND SOC
LEND_RDR_DLX.FMT	Extended Derived Neutron spectra data column descriptions	LEND SOC
LEND_RDR_ALD.FMT	Averaged Neutron spectra data column descriptions	LEND SOC
LEND_RDR_ALF.FMT	Averaged Neutron flux data column descriptions	LEND SOC

2.7. Calibration Directory Contents

The Calibration Directory contains calibration files used to process the data products, or calibration data needed to use the data products. The following files are contained in the Calibration Directory.

 

File Name	File Contents	File Provided By
CALINFO.TXT	A description of the contents of this directory	LEND SOC
LEND_CALIBRATION_REPORT.PDF	Description of instrument and data calibration performed by LEND team in a PDF document form	LEND SOC
LEND_CALIBRATION_REPORT.HTM	Description of instrument and data calibration performed by LEND team in an HTML document form	LEND SOC

2.8. Software Directory Contents

The Software Directory contains software to view the data archive. It also contain a user guide which explains how to install and use the software.

 

File Name	File Contents	File Provided By
SOFTINFO.TXT	A description of the contents of this directory	LEND SOC

3. Archive Volume Format

This section describes the format of LRO LEND EDR/RDR Archive Volumes. Data that comprise the Archive will be formatted in accordance with Planetary Data System specifications [Applicable Documents 5 and 6].

3.1. File Formats

This section describes file formats for the kinds of files contained on Archive Volumes.

3.1.1. Document File Format

Document files with the .TXT suffix exist in the Root, Index, Catalog, Document, and Label directories. They are ASCII files which may have embedded PDS labels. Lines in a .TXT file end with a carriage return character (ASCII 13) and a line feed character (ASCII 10). This allows the files to be readable under various operating systems.

Documents in the Document directory may contain formatting and figures that cannot be rendered as ASCII text. Therefore each document is given in two formats, hypertext and PDF. The hypertext file contains ASCII text plus hypertext markup language (HTML) commands that enable it to be viewed in a Web browser such as Netscape Navigator or Microsoft Internet Explorer. The hypertext file may be accompanied by ancillary files such as images and style sheets that are incorporated into the document by the Web browser. The second format, PDF (Portable Document Format) is a proprietary format of Adobe Systems Incorporated that is frequently used for distributing documents. Adobe offers free software, Acrobat Reader, for viewing PDF files.

3.1.2. Tabular File Format

Tabular files (.TAB suffix) exist in the Index directory. Tabular files are ASCII files formatted for direct reading into many database management systems on various computers. All fields are separated by commas, and character fields are enclosed in double quotation marks ("). (Character fields are padded with spaces to keep quotation marks in the same columns of successive records.) Character fields are left justified, and numeric fields are right justified. The "start byte" and "bytes" values listed in the labels do not include the commas between fields or the quotation marks surrounding character fields. The records are of fixed length, and the last two bytes of each record contain the ASCII carriage return and line feed characters. This allows a table to be treated as a fixed length record file on computers that support this file type and as a text file with embedded line delimiters on those that don't.

            All tabular files are described by PDS labels, either embedded at the beginning of the file or detached. If detached, the PDS label file has the same name as the data file it describes, with the extension .LBL; for example, the file INDEX.TAB is accompanied by the detached label file INDEX.LBL in the same directory.

3.1.3. PDS Label Format

All data files in the archive have PDS labels, either embedded at the beginning of the file or detached in a separate file. For examples of PDS labels for each type of data product, see the Data Product SISs [Applicable Documents 3 and 4].

A PDS label, whether embedded or detached from its associated file, consists of lines of ASCII text in the form of keyword = value statements that provide descriptive information about the data file. The label is intended to be readable both by humans and by software. Details of the syntax and semantics of PDS labels can be found in the PDS Standards Reference (Applicable Document 6), and definitions of the keywords used in the label can be found by using the PDS Data Dictionary Lookup web service at http://pds.jpl.nasa.gov/tools/data_dictionary_lookup.cfm.

Lines of text in detached labels end with a carriage return character (ASCII 13) and a line feed character (ASCII 10). This allows the files to be read under various operating systems.

3.1.4. Software File Format

None provided at this time.

3.1.5. Catalog File Format

Catalog files (suffix .CAT) exist in the Root and Catalog directories. Like PDS labels, they are text files formatted as keyword = value statements. They contain descriptions of the data set, instrument, spacecraft, and mission, as well as personnel contact information and references to published literature. They are called Catalog Files because they are loaded into the PDS online catalog to make the information available to users searching for data.  

3.1.6. Science Data File Formats

LEND EDR and RDR data files are in the form of binary time-series tables. The binary tables include columns of most-significant-byte-first (“big-endian”) unsigned integers. One file contains one flight day’s worth of data, from midnight to midnight, which covers several orbits.

For more information about the format and content of the data products, see the Data Product SISs [Applicable Documents 3 and 4].

4. Archive Volume Generation

4.1. Data Transfer and Validation Methods

LEND EDR/RDR products are generated by the LEND SOC team at the Lunar and Planetary Laboratory, University of Arizona. The products are delivered to the PDS Geosciences Node at Washington University electronically using ftp according to the schedule in the LRO Data Management and Archive Plan (Applicable Document 1).

4.2. Data Product Sizes and Delivery Rates

Table 1 summarizes expected sizes and production rates for the LRO LEND EDR/RDR Standard Products.

Table 1 – Standard Product Sizes and Delivery Rates

Product	Product Size	Production Rate	Expected Number of Products for Primary Mission (365 days)	Expected Total Data Volume for Primary Mission
LEND_EDR	38 MB	1 per day	365	13870 MB
LEND_RDR	TBD	TBD	TBD	TBD

 

The LEND SOC team releases EDR/RDR data products according to the schedule outlined in the LRO Data Management and Archive Plan [Applicable Document 1].

4.3. Interface Media Characteristics

All volumes in the LRO LEND EDR/RDR Standard Product Archive conform to ISO 9660 standards [ISO 9660, 1988].

4.4. Backup and Duplicates

Backup copies of LEND EDR/RDR products will be stored at the University of Arizona until the final versions of the products have been archived on physical media with the PDS.

Duplicate copies of LEND archive volumes on physical media will be stored at the PDS Geosciences Node, the PDS Central Node and the NSSDC.

4.5. Labeling and Identification

Each LEND EDR/RDR archive volume will be identified by a unique volume ID formed according to the scheme LROLEN_0XXX where:

 

            LRO represents the mission,

LEN represents the instrument, and

XXX is a volume sequence number, starting with 001.

 

If the entire LEND EDR/RDR data set is revised, and the data set ID version number is incremented, the keyword VOLUME_VERSION_ID in the file VOLDESC.CAT will be incremented on volumes containing the revised data set.

5. Support Staff and Cognizant Persons

Igor Mitrofanov, LEND Principal Investigator, Russian Space Institute

William Boynton, LEND Co Investigator, University of Arizona

Richard Star, LEND Co Investigator, Goddard Space Flight Center

Larry Evans, LEND Co Investigator, Goddard Space Flight Center

Karl Harshman, LEND SOC Manager, University of Arizona

Questions and comments regarding the LEND EDR/RDR Archive Volume may be directed to the PDS Geosciences Node, Washington University.