Lunar
Reconnaissance Orbiter
Diviner Lunar
Radiometer Experiment
Archive Volume
Software Interface
Specification
Version 1.15
June 16, 2020
Prepared by:
Mark Sullivan
Approved
by:
________________________________
David Paige
Principal Investigator, DLRE
________________________________ ______________________
Raymond E Arvidson Edwin Grayzeck
Director, PDS Geosciences Node PDS Program Manager
University of California at Los Angeles
TABLE OF CONTENTS
1. Introduction................................................................................................................................. 1
1.1. Purpose and Scope............................................................................................................... 1
1.2. Content Overview................................................................................................................ 1
1.3. Applicable Documents and Constraints............................................................................... 1
1.4. Relationships
with Other Interfaces.................................................................................... 2
2. Archive Volume Contents........................................................................................................... 2
2.1. Online and Physical Storage................................................................................................ 2
2.2. Archive Volume Directory................................................................................................... 2
2.3. Root Directory Contents...................................................................................................... 3
2.4. Data Directory Contents and Naming................................................................................. 3
2.5. Index Directory Contents.................................................................................................... 5
2.6. Document Directory Contents............................................................................................. 6
2.7. Catalog Directory Contents................................................................................................. 6
2.8. Label Directory Contents..................................................................................................... 7
2.9. Calib Directory Contents (RDR only)................................................................................. 7
2.10. Extras Directory Contents (RDR, GDR and FDS only)................................................... 8
2.11. Browse Directory Contents (RDR, GDR and FDS only)................................................. 8
3. Archive Volume Format.............................................................................................................. 8
3.1. File Formats......................................................................................................................... 9
3.1.1. Document File Format............................................................................................... 9
3.1.2. Tabular File Format................................................................................................... 9
3.1.3. PDS Label Format..................................................................................................... 9
3.1.4. Catalog File Format................................................................................................. 10
3.1.5. Science Data File
Formats....................................................................................... 11
4. Archive Volume Generation...................................................................................................... 12
4.1. Data Transfer and Validation Methods............................................................................. 12
4.2. Data Product Sizes and Delivery Rates............................................................................. 12
4.3. Interface Media Characteristics......................................................................................... 13
4.4. Backup and Duplicates...................................................................................................... 13
4.5. Labeling and Identification............................................................................................... 13
5. Support Staff and Cognizant Persons....................................................................................... 13
TABLES AND FIGURES
Table 1- Standard Product Sizes and Delivery Rates................................................................... 11
DOCUMENT
CHANGE LOG
|
Change
|
Date
|
Affected Portions
|
|
Initial
|
3/20/2007
|
all
|
|
Draft 1.1
|
6/1/2007
|
1.3, 1.4, 2.2, 2.3, 2.4, 2.5, 2.6, 2.8, 4.1
|
|
Draft 1.2
|
8/1/2007
|
1.3, 2.1, 2.2, 2.4, 2.5, 2.6, 4.1, 4.2, 4.5
|
|
Draft 1.3
|
8/23/2007
|
1.3, 2.1
|
|
Draft 1.4
|
1/21/2008
|
1.3, 2.4, 4.1, 4.2
|
|
Draft 1.5
|
3/27/2008
|
1.3, 2.2, 2.3, 2.4, 2.6, 2.7, 2.9, 4.2, 4.5
|
|
Draft 1.6
|
4/16/2008
|
2.5, 2.6
|
|
Draft 1.7
|
11/10/2008
|
2.6, 2.9
|
|
Draft 1.8
|
12/01/2009
|
4.2
|
|
Version 1.9
|
2/2/2010
|
2.2, 2.4, 2.9, 2.10
|
|
Version 1.10
|
11/11/2010
|
1.3, 2.4, 2.7, 2.8, 3.1.5, 4.1, 4.2
|
|
Version 1.11
|
3/10/2011
|
2.4, 4.2
|
|
Version 1.12
|
3/13/2013
|
2.4
|
|
Version 1.13
|
3/15/2013
|
1.3, 2.4, 2.6, 2.7, 2.8, 4.2, 4.5
|
|
Version 1.15
|
6/17/2020
|
2.1, 2.4, 2.5, 2.6, 2.7, 2.8, 4.1, 4.2
|
ACRONYMS AND ABBREVIATIONS
|
ASCII
|
American Standard Code for Information Interchange
|
|
CD-ROM
|
Compact Disk - Read-Only Memory
|
|
CD-WO
|
Write-Once Compact Disk
|
|
DLRE
|
Diviner Lunar Radiometer Experiment
|
|
EDR
FDS
|
Experiment Data Record
Foundation Data Set
|
|
GB
|
Gigabytes
|
|
GCP
|
Global Cumulative Product
|
|
GDR
|
Gridded Data Record
|
|
ISO
|
International Standards Organization
|
|
JPL
|
Jet Propulsion Laboratory
|
|
LRO
|
Lunar Reconnaissance Orbiter
|
|
MB
|
Megabytes
|
|
NSSDC
|
National Space Science Data Center
|
|
PCP
|
Polar Cumulative Product
|
|
PDS
|
Planetary Data System
|
|
PRP
|
Polar Resource Product
|
|
PSG
|
Project Science Group
|
|
RDR
|
Reduced Data Record
|
|
SCET
|
Spacecraft Event Time
|
|
SIS
|
Software Interface Specification
|
|
SOC
|
Science Operation Center
|
|
TBD
|
To Be Determined
|
|
UTC
|
Universal Time Coordinated
|
GLOSSARY
Archive
– An archive consists of one or more
data sets along with all the documentation and ancillary information needed to
understand and use the data. An archive is a logical construct independent of
the medium on which it is stored.
Archive
Volume, Archive Volume Set
– A volume is a unit of media on which data products are stored; for example,
one CD-ROM or DVD-ROM. An archive volume
is a volume containing all or part of an archive; that is, data products plus
documentation and ancillary files. When an archive spans multiple volumes, they
are called an archive volume set.
Usually the documentation and some ancillary files are repeated on each volume
of the set, so that a single volume can be used alone.
Catalog
Information –
Descriptive information about a data set (e.g. mission description, spacecraft
description, instrument description), expressed in Object Description Language
(ODL) which is suitable for loading into a PDS catalog.
Data
Product – A
labeled grouping of data resulting from a scientific observation, usually
stored in one file. A product label identifies, describes, and defines the
structure of the data. An example of a data product is a planetary image, a
spectrum table, or a time series table.
Data
Set – An accumulation of data products. A
data set together with supporting documentation and ancillary files is an
archive.
SCET
– Spacecraft Event Time. The time an event occurs in a spacecraft,
usually in UTC.
Standard
Data Product – A
data product generated in a predefined way using well-understood procedures,
processed in "pipeline" fashion.
Data products that are generated in a nonstandard way are sometimes called special data products.
This
Software Interface Specification is intended to be used by those who wish to
understand the format and content of the Diviner Lunar Radiometer Experiment (DLRE
or “Diviner”) Archive. Typically, these individuals would be software
engineers, data analysts, or planetary scientists.
The
specifications in this document apply to all Diviner standard product archive
volumes that are generated by the Lunar Reconnaissance Orbiter Project.
The
Diviner archive is composed of several types of products, including
ASCII-formatted data tables, and images of binned values in raw 32-bit integer
format (IMG), JPEG2000 (JP2), and JPEG (JPG) format. The data products are generated and volumes
assembled by the Diviner teams at JPL and at UCLA.
This Software Interface Specification (SIS) describes the
format, content, and generation of the Diviner Archive. Section 2, Archive
Volume Generation, describes the procedure for transferring data products to
archive media. Section 3, Archive Volume Contents, describes the structure of
the archive volumes and the contents of each file. Section 4, Archive Volume
Format, describes the file formats used on the archive volumes. Finally,
Section 5, Support Staff and Cognizant Persons,
lists the individuals responsible for generating the archive volumes.
This Archive Volume SIS is intended to be consistent with
the following documents:
- Lunar
Reconnaissance Orbiter Project Data Management and Archive Plan, Karen
North, LRO Document 431-PLAN-00182, Rev. 6, November 6, 2006.
- Diviner
Lunar Radiometer Experiment Telemetry Dictionary, S. M. Loring, JPL
D-33198.
- Diviner
Lunar Radiometer Experiment Experiment Data Record Software Interface
Specification, Mark Sullivan, Rev. 1.8, December 10, 2009.
- Diviner
Lunar Radiometer Experiment Reduced Data Record and Derived Products Software
Interface Specification, Mark Sullivan, Rev. 1.13, March 4, 2013.
- Planetary
Data System Archive Preparation Guide, Version 1.4, JPL D-31224, April 1,
2010.
- Planetary
Data System Standards Reference, February 27, 2009, Version 3.8. JPL
D-7669, Part 2.
- ISO
9660-1988, Information Processing - Volume and File Structure of CD-ROM
for Information Exchange, April 15, 1988.
8.
Lunar Reconnaissance Orbiter
Diviner Science Team and PDS Geosciences Node Interface Control Document (ICD),
S. Slavney, Nov. 16, 2006.
9.
Diviner Lunar Radiometer
Experiment Foundation Data Set Software Interface Specification, Mark Sullivan,
Rev. 0.3, March 15, 2013. (forthcoming)
This Archive
Volume SIS could be affected by changes to the design of the Diviner
standard data
products (Applicable Documents #3, #4, & #9).
2. Archive Volume Contents
This
section describes the contents of the Diviner Archive volumes, including the
file names, file contents, file types, and organization responsible for
providing the files.
The
Diviner archive will be made available through the Planetary Data System via
Internet
access. The online Diviner archive is organized by volumes containing EDR, RDR,
and FDS data products, with PDS VOLUME_IDs of LRODLR_0001 and LRODLR_0002 (EDR),
LRODLR_1001 and LRODLR_1002 (RDR), and LRODLR_2001 (FDS, forthcoming). The EDR and RDR repositories were split into
two volumes when their size exceeded the maximum allowed per volume. In this case, the new volume has its
rightmost digit incremented by one.
The Diviner archive volumes will include the following
directories.
|
Directory
Name
|
Contents
|
File
Provided By
|
|
/ -
Volume root directory
|
Introduction
and errata text files
|
Diviner
|
|
/CATALOG
|
Descriptions
of data set, instruments, spacecraft, and mission as found in the PDS Catalog
|
Diviner
and Geosciences
|
|
/DATA
|
Data
product files
|
Diviner
and Geosciences
|
|
/DOCUMENT
|
Documentation
files
|
Diviner
|
|
/INDEX
|
Volume
index tables
|
Diviner
|
|
/LABEL
|
Descriptions
of data file formats, referenced by PDS labels
|
Diviner
|
|
/CALIB
|
RDR volume
only. Calibration data, including channel central wavelengths and spectral
response functions.
|
Diviner
|
|
/EXTRAS
|
RDR
and FDS volumes only. Supplemental Material, including Experimenters Notebook
and XML files useful for supporting proper viewing of JP2 data.
|
Diviner
|
|
/BROWSE
|
RDR
and FDS volumes only. JPEG images of Gridded Data Records (GDRs) and tiled
FDS products for easy viewing of the data.
|
Diviner
|
Files
in the Root Directory include an overview of the archive, a description of the
volume for the PDS Catalog, and a list of errata or comments about the archive.
The following files are contained in the Root Directory.
|
File Name
|
File Contents
|
File Provided By
|
|
AAREADME.TXT
|
Volume content and format information
|
Diviner
|
|
ERRATA.TXT
|
A listing of archive comments and updates
|
Diviner
|
|
VOLDESC.CAT
|
A description of the contents of this volume in a PDS
format readable by both humans and computers
|
Diviner
|
The pipeline data products will be located in the DATA
directory of the archive volume, and will be grouped according to the following
directory structure:
- Immediately
under DATA, a directory representing the year, of the format YYYY.
- Under
the year, a directory representing the month, of the format YYYYMM.
- Under
the month, a directory representing the date, of the format YYYYMMDD.
For EDR, within
each date directory there will be up to 24 hourly data product files and their
labels. The labels will point to the
corresponding data files and contain pointers to a format file detailing the
column layout of the data files. The
data product file names will be in the format YYYYMMDDHH_EDR.TAB, and the label
file names will be in the format YYYYMMDDHH_EDR.LBL. All dates and times are in SCET (UTC). The files and their corresponding labels
contain up to one hour of data and are named for the hour in which the first
observation in the data file was taken.
For RDR, within
each date directory there will be up to 144 ten-minute data product files and
their labels. The labels will point to
the corresponding data files and contain pointers to a format file detailing
the column layout of the data files.
Due to the size of the RDR data table files, they will be compressed
using Info-ZIP (a freely available utility).
The data product file names will be in the format YYYYMMDDHHMM_RDR.ZIP
(when compressed), and the label file names will be in the format YYYYMMDDHHMM_RDR.LBL. All dates and times are in SCET (UTC). The files and their corresponding labels
contain up to ten minutes of data and are named for the ten-minute period in
which the first observation in the data file was taken. The RDR data tables may
be uncompressed using most any generic “unzip” utility, and then the zip file
may be deleted to save space.
The derived (or
non-pipeline) data products will be located in subdirectories of the DATA
directory. Gridded Data Records (GDRs),
which are NASA Level 2 and 3, will be grouped according to the following
directory structure:
- Immediately under DATA, subdirectories
GDR_L2 and GDR_L3.
- Under GDR_L2 and GDR_L3, the
directories CYLINDRICAL and POLAR.
For dated products containing a YYYYMMDD string, a subdirectory for
the year of the form YYYY comes between GDR_L* and the CYLINDRICAL/POLAR
directories.
- Under CYLINDRICAL and POLAR, the
directories IMG and JP2.
Under the IMG
directory, there will be a series of files containing gridded, binned, and
averaged values represented by scaled 16-bit signed integers in least
significant byte (LSB) format. Files
representing counts and error estimates will also be provided. These grids are provided in both cylindrical
and polar stereographic projections. The
master resolution for Level 2 GDRs is 128 pixels per degree and the master
resolution for Level 3 GDRs is 32 pixels per degree, with lower resolutions
available in the same directory. The JP2
directory contains JPEG2000 versions of each GDR product, compatible with many
Geographical Information Systems such as ArcGIS (www.arcgis.com). Additionally the BROWSE/ subtree of the
Archive Volume contains preview images for the JPEG2000 images in JPG format. Detached PDS labels accompany and describe
each product.
A note on the
Foundation Data Set (FDS) products: These data products have not yet been
released. Any references to them in
this document serve only as placeholders.
Foundation Data
Set (FDS) products (forthcoming) are essentially tiled GDRs, global cylindrical
and polar maps split into subdivisions of latitude and longitude. Their structure is similar to that of the
GDR subdirectories, but they are contained in their own volume. They are described in detail in the FDS SIS [9]
(forthcoming).
Some
GDAL-supported applications such as ArcGIS require additional XML files to properly
display JPEG2000 data. These are found
in the archive directory EXTRAS/ (please
see section 2.10 for more).
NASA Level 4 Polar
Resource Products (PRP) are located
under DATA/PRP. The files themselves are
in ASCII format with a .TAB suffix and an accompanying PDS label.
NASA Level 4
Global Cumulative Products (GCP) are
located under DATA/GCP. The files
themselves are in ASCII format with a .TAB suffix and an accompanying PDS
label.
NASA Level 4 Polar
Cumulative Products (PCP) are located
under DATA/PCP. The files themselves are
in ASCII format with a .TAB suffix and an accompanying PDS label.Here is a
brief sketch of the RDR DATA/ directory organization:
DATA - <YYYY> - <YYYYMM> -
<YYYYMMDD> - (RDR zipped tabular data)
DATA - GDR_L<N> - <YYYY> - CYLINDRICAL
- IMG - (GDR IMG data)
JP2
- (GDR JP2 data)
DATA - GDR_L<N> - <YYYY> - POLAR
- IMG - (GDR IMG data)
JP2
- (GDR JP2 data)
DATA - PRP - (PRP tabular data)
DATA - GCP - (GCP tabular data)
DATA - PCP - (PCP tabular data)
For more detailed information, please see the RDR data
product SIS [4]. The FDS data structure
is more complex and is described in the FDS data product SIS [9] (both data and
SIS forthcoming).
Files
in the Index Directory are provided to help the user locate products on the archive
volume. The following files are contained in the Index Directory.
|
File Name
|
File Contents
|
File Provided By
|
|
INDXINFO.TXT
|
A description of the contents of this directory
|
Diviner
|
|
*INDEX.TAB
|
A table listing all data products on the volume, where *
corresponds to the data product type: RDR, GDR, PRP, GCP, PCP, or FDS. The
prefix is omitted for the EDRs.
|
Diviner
|
|
*INDEX.LBL
|
A PDS detached label that describes *INDEX.TAB
|
Diviner
|
The format of the file *INDEX.TAB is described by INDXINFO.TXT
and includes the following fields:
- VOLUME_ID
– A unique identifier that gives the volume.
- PATH_NAME
– The subdirectory path in which the data files are found.
- FILE_NAME
– The name of the PDS label file that describes the data file.
- PRODUCT_ID
– The name of the data table file.
- PRODUCT_CREATION_TIME
– When it was created.
- START_TIME
– The time of the first observation in the data file.
- STOP_TIME
– The time of the last observation in the data file.
- SPACECRAFT_CLOCK_START_COUNT
– The value of the spacecraft clock in the first observation in the data
file.
- SPACECRAFT_CLOCK_STOP_COUNT
– The value of the spacecraft clock in the last observation in the data
file.
- START_ORBIT_NUMBER
– The orbit number of the first observation in the data file (RDR only).
- STOP_ORBIT_NUMBER
– The orbit number of the last observation in the data file (RDR only).
The
Document Directory contains documentation to help the user understand and use
the archive data. The following files are contained in the Document Directory.
|
File Name
|
File Contents
|
File Provided By
|
|
DOCINFO.TXT
|
A description of the contents of this directory
|
Diviner
|
|
DP_SIS.TXT or .HTM
|
The Data Product SIS as text or hypertext. Hypertext
version may have ancillary files in the directory DP_SIS_files. (EDR and RDR
volumes only)
|
Diviner
|
|
DP_SIS.PDF
|
The Data Product SIS as a PDF file ( EDR and RDR volumes
only)
|
Diviner
|
|
DP_SIS.LBL
|
A PDS detached label that describes DP_SIS (EDR and RDR
volumes only)
|
Diviner
|
|
DP_FDS_SIS.HTM
|
The FDS Data Product SIS as hypertext. Hypertext version may have ancillary files
in the directory DP_FDS_SIS_files (FDS volume only, forthcoming)
|
Diviner
|
|
DP_FDS_SIS.PDF
|
The FDS Data Product SIS as a PDF file (FDS volume only,
forthcoming)
|
Diviner
|
|
DP_FDS_SIS.LBL
|
The FDS Data Product SIS as a PDF file (FDS volume only,
forthcoming)
|
Diviner
|
|
DP_ARCHSIS.HTM
|
The Archive Volume SIS (this document) as hypertext.
Hypertext version may have ancillary files in the directory DP_ARCHSIS_files
|
Diviner
|
|
DP_ARCHSIS.PDF
|
The Archive Volume SIS (this document) as a PDF file
|
Diviner
|
|
DP_ARCHSIS.LBL
|
A PDS detached label that describes DP_ARCHSIS
|
Diviner
|
|
PDSDD.FUL
|
Complete PDS Data Dictionary including LRO local keywords
|
Geosciences
|
|
PDSDD.IDX
|
Index to PDSDD.FUL (used by some PDS software)
|
Geosciences
|
|
PDSDD.LBL
|
Label for PDSDD.FUL and PDSDD.IDX
|
Geosciences
|
The
files in the Catalog Directory provide a top-level understanding of the
mission, spacecraft, instruments, and data sets. The files in this directory
are coordinated with the PDS data engineer, who is responsible for loading them
into the PDS catalog. The following files are found in the Catalog Directory.
|
File Name
|
File Contents
|
File Provided By
|
|
CATINFO.TXT
|
A description of the contents of this directory
|
Diviner
|
|
EDRDS.CAT
|
Data set information for EDR products (EDR volume only)
|
Diviner
|
|
RDRDS.CAT
|
Data set information for RDR products (RDR volume only)
|
Diviner
|
|
GDRDS.CAT
|
Data set information for GDR products (RDR volume only)
|
Diviner
|
|
PRPDS.CAT
|
Data set information for PRP products (RDR volume only)
|
Diviner
|
|
GCPDS.CAT
|
Data set information for GCP and PCP products (RDR volume
only)
|
Diviner
|
|
FDSDS.CAT
|
Data set information for FDS products (FDS volume only,
forthcoming)
|
Diviner
|
|
INSTHOST.CAT
|
Instrument host (i.e., spacecraft) information for the PDS
catalog
|
Geosciences
|
|
INST.CAT
|
Instrument information for the PDS catalog
|
Diviner
|
|
MISSION.CAT
|
Mission information for the PDS catalog
|
Geosciences
|
|
PERSON.CAT
|
Personnel information for the PDS catalog (Team and PDS
personnel responsible for generating the archive)
|
Diviner
|
|
REF.CAT
|
References mentioned in other *.CAT files
|
Diviner
|
|
DSMAP.CAT
|
A parameter description for the cylindrically projected
gridded data (RDR and FDS volume only)
|
Diviner
|
|
DSMAP_POLAR.CAT
|
A parameter description for the polar stereographically
projected gridded data (RDR and FDS volume only)
|
Diviner
|
The
Label Directory contains files that describe data format and organization.
These files are referred to in the PDS labels that accompany the data products.
They are "include" files that are intended to be parsed as if they
were part of the PDS labels that refer to them.
The following files are contained in the Label Directory.
|
File Name
|
File Contents
|
File Provided By
|
|
LABINFO.TXT
|
A description of the contents of this directory
|
Diviner
|
|
DLRE_EDR.FMT
|
EDR table data column
descriptions (EDR volume only)
|
Diviner
|
|
DLRE_RDR.FMT
|
RDR table data column
descriptions (RDR volume only)
|
Diviner
|
|
DLRE_GDR.FMT
|
GDR descriptions (RDR
volume only)
|
Diviner
|
|
DLRE_PRP.FMT
|
PRP descriptions (RDR
volume only)
|
Diviner
|
|
DLRE_GCP.FMT
|
GCP descriptions (RDR
volume only)
|
Diviner
|
|
DLRE_PCP.FMT
|
PCP descriptions (RDR
volume only)
|
Diviner
|
|
DF_ORB.FMT
|
Description of FDS
Orbit Tables (FDS volume only, forthcoming)
|
Diviner
|
The Calib Directory holds files that
contain data used in the calibration of the RDR data set. It only contains files if there are
calibration data to report.
|
File Name
|
File Contents
|
File Provided By
|
|
AA_CALIBRATION_OVERVIEW.TXT
|
A top-level summary of the calibration of this dataset
|
Diviner
|
|
CALINFO.TXT
|
A description of the contents of this directory
|
Diviner
|
|
Documentation (*.PDF, *.HTM) and labels (*.LBL)
|
Various documents in
Adobe Acrobat and HTML form and their associated labels.. See CALINFO.TXT for more.
|
Diviner
|
|
Tables (*.CSV) and labels (*.LBL)
|
Various tables in
comma-separated form and their associated labels. See CALINFO.TXT for more.
|
Diviner
|
The Extras directory is for supplementary material that may
be of interest but is not necessary for using or understanding of the data.
|
File Name
|
File Contents
|
File Provided By
|
|
EXTRINFO.TXT
|
A
description of the contents of this directory
|
Diviner
|
|
EXPERIMENTERS_NOTEBOOK.TXT
|
Experimenters
Notebook
|
Diviner
|
|
*.AUX.XML
|
XML
files useful for supporting proper viewing of JP2 data
|
Diviner
|
Some GDAL-supported applications such as ArcGIS require
supplementary XML files in order to properly display JPEG2000 (JP2)
images. These XML files are found in
EXTRAS/, under a similar directory structure as described for DATA/ in section
2.4. Users are advised to download the corresponding XML file for each JP2
image they have and keep it in the same directory as the JP2 image.
The
Browse directory contains JPEG versions of the Gridded Data Records (GDRs) and
tiled FDS products for easy viewing of the data. Users are encouraged to look
here first to get an idea of what the JPEG2000 (JP2) data will look like. JPEG
images load quickly into almost any web browser or image viewing program.
|
File Name
|
File Contents
|
File Provided By
|
|
BROWINFO.TXT
|
A description of the
contents of this directory
|
Diviner
|
|
*.JPG
|
JPG files
representative of JP2 data, for quick and easy viewing of JP2 data
|
Diviner
|
This
section describes the format of Diviner Archive Volumes. Data that comprise the
Archive will be formatted in accordance with Planetary Data System
specifications [Applicable Documents 5 and 6].
This
section describes file formats for the kinds of files contained on Archive
Volumes.
Document
files with the .TXT suffix exist in the Root, Index, Catalog, Browse, Extras, and
Document directories. They are ASCII files which may have embedded PDS labels.
Lines in a .TXT file end with a carriage return character (ASCII 13) and a line
feed character (ASCII 10). This allows the files to be readable under various
operating systems.
Documents
in the Document directory may contain formatting and figures that cannot be
rendered as ASCII text. Therefore each document is given in one of two formats,
hypertext and PDF. The hypertext file contains ASCII text plus hypertext markup
language (HTML) commands that enable it to be viewed in a Web browser such as
Netscape Navigator or Microsoft Internet Explorer. The hypertext file may be
accompanied by ancillary files such as images and style sheets that are
incorporated into the document by the Web browser. The second format, PDF
(Portable Document Format) is a proprietary format of Adobe Systems
Incorporated that is frequently used for distributing documents. Adobe offers
free software, Acrobat Reader, for viewing PDF files.
Tabular
files (.TAB suffix) exist in the Index and Data directories. Tabular files are
ASCII files formatted for direct reading into many database management systems
on various computers. All fields are separated by commas and character fields
are enclosed in double quotation marks ("). Character fields are left justified,
and numeric fields are right justified. The records are of fixed length, and
the last two bytes of each record contain the ASCII carriage return and line
feed characters. This allows a table to be treated as a fixed length record
file on computers that support this file type and as a text file with embedded
line delimiters on those that don't.
All tabular files are described by
PDS labels, either embedded at the beginning of the file or detached. If
detached, the PDS label file has the same name as the data file it describes,
with the extension .LBL; for example, the file INDEX.TAB is accompanied by the
detached label file INDEX.LBL in the same directory.
All data files in the archive have
detached PDS labels. The PDS label provides descriptive information about the
associated file. The PDS label is an
object-oriented structure consisting of sets of 'keyword=value' declarations.
The object to which the label refers (e.g. IMAGE, TABLE, etc.) is denoted by a
statement of the form:
^object
= location
in
which the carat character (^, also called a pointer in this context) indicates
where to find the object. In an embedded
label, the location is an integer representing the starting record number of
the object (the first record in the file is record 1). In a detached label, the location denotes the
name of the file containing the object, along with the starting record or byte
number, if there is more than one object in the file. For example:
^HEADER
= ("F01.IMG",1)
^IMAGE
= ("F01.IMG",1025 <BYTES>)
indicates
that the IMAGE object begins at byte 1025 of the file F01.IMG, in the same
directory as the detached label file.
Below is a list of the possible formats for the ^object definition.
^object = n
^object = n<BYTES>
^object = "filename.ext"
^object = ("filename.ext",n)
^object = ("[dirlist]filename.ext",n)
^object = ("filename.ext",n<BYTES>)
^object =
("[dirlist]filename.ext",n<BYTES>)
where
n is the starting record or byte number of the object,
counting from the beginning of the file (record 1, byte 1),
<BYTES> indicates that the number given is in units of bytes,
filename is the up to 27 character, alphanumeric upper‑case
file name,
ext is the 3 character upper‑case file extension,
dirlist is a period‑delimited path‑list of parent
directories, in upper case, that specifies the object file directory (used only
when the object is not in the same directory as the label file). The list
begins at the directory level below the root directory of the volume.
'[dirlist]' may be omitted when the object being described is located either in
the same directory as the detached label, or in a subdirectory named LABEL that
is located in a higher level of the directory tree, typically the volume root
itself.
Lines of text in detached labels end
with a carriage return character (ASCII 13) and a line feed character (ASCII
10). This allows the files to be readable under various operating systems.
Catalog files (suffix .CAT) exist in
the Root and Catalog directories. They are text files formatted in an
object-oriented structure consisting of sets of 'keyword=value' declarations.
Diviner
data files are all in the form of ASCII tables, binary IMG grids, and JPEG2000
images. For more information about the format and content of the data products,
see the Data Product SIS [Applicable Documents 3, 4, 9].
4. Archive
Volume Generation
Diviner EDR and pipeline RDR data products
and labels are generated by the Diviner Science Team at JPL and then
transferred to the Diviner Team at UCLA.
Derived (non-pipeline) data products such as GDRs, PRPs, GCPs, PCPs, and
FDS products are generated by the Diviner Science Team at UCLA. The Diviner Team at UCLA assembles the
archive volumes and delivers the volumes by Internet transfer to the PDS
Geosciences Node at Washington University in St. Louis, Missouri according to
the schedule in the LRO DM&AP [1]. The Diviner Team validates the products
for scientific integrity.
The Geosciences Node validates the
products and completed archive volumes for compliance with PDS standards and
with the Data Product SIS and Archive Volume SIS documents, and makes the
volumes available to the public via the Geosciences Node web site.
The Geosciences Node will also transfer
Diviner archive volumes to the National Space Science Data Center (NSSDC) for
long term storage according to PDS policy, using a transfer medium agreed upon
by PDS and NSSDC.
Table
1 summarizes expected sizes and production rates for the Diviner Standard
Products.
Table
1-
Standard Product Sizes and Delivery Rates
|
Product
|
Product Size
|
Production Rate
|
Expected Number of Products for Primary
Mission (395 days)
|
Expected Total Data Volume for Primary
Mission (395 days)
|
|
EDR
|
One
hour: 75 MB
|
24 data table files per day
|
9480
|
711
GB
|
|
Pipeline RDR
|
Ten
minutes: 296 MB uncompressed, 40 MB compressed
|
144 data table files per day
|
56,880
|
16,831
GB uncompressed, 2275 GB compressed
|
|
Level 2 GDR
|
Variable
|
Once at the end of first year of mapping.
|
22,050
|
3600
GB
|
|
Level 3 GDR
|
Variable
|
Once at the end of first year of mapping.
|
36
|
1.4
GB
|
|
Level 4 PRP
|
0.6
GB
|
Once at the end of first year of mapping.
|
2
|
1.2
GB
|
|
Level 4 GCP
|
156
MB
|
Once
|
18
|
2.7
GB (Covers 2096 days – 2009-07-05 to 2015-04-01)
|
|
Level 4 PCP
|
Between
70 MB and 290 MB
|
Once
|
768
|
151
GB (Covers 3514 days – 2009-07-05 to 2019-02-17)
|
|
Level 2 FDS (forthcoming)
|
Variable
|
Once
|
3
Million
|
25,000
GB
|
|
Level 3 FDS (forthcoming)
|
Variable
|
Once
|
37,000
|
368
GB
|
After a data validation period of 6
months, the Diviner team will assemble the data products and ancillary files
into archive volumes and transfer the archive to the Geosciences Node. The Diviner archive will be made available
via data releases scheduled at three month intervals as specified in the Lunar
Reconnaissance Orbiter Project Data Management and Archive Plan (see Applicable
document #1).
All
volumes in the Diviner Standard Product Archive conform to ISO 9660 Level II
Standards [ISO 9660, 1988].
Backup copies of
Diviner EDR and RDR products will be stored at UCLA until the final versions of
the products have been archived on physical media with the PDS. Duplicate
copies of Diviner archive volumes on physical media will be stored at the PDS
Geosciences Node and the NSSDC.
Each Diviner archive volume will be
identified by a unique volume ID formed according to the scheme LRODLR_Xnnn,
where
LRO is the Mission ID
DLR is the Instrument ID
X equals 0 denotes EDR, 1 denotes RDR,
2 denotes FDS
nnn
is the volume sequence number starting with 001
David Paige, Diviner Principal Investigator, UCLA
Mark Sullivan, Diviner Archivist, UCLA
Susan Slavney, PDS Geosciences Node, Washington University,
St Louis MO
Edward Guinness, PDS Geosciences Node, Washington
University, St Louis MO