From ViRBO

Looking for the GEM Focus Group pages?: http://virbo.org/GEM

1 Introduction

The Data section below has a complete list of data relevant to radiation belt research. Many previously inaccessible data sets are now available directly from ViRBO and we plan to continuously add new tools, links to new data sets, and services of interest to radiation belt scientists. If you have any suggestions, requests, or questions, email them to virbo@virbo.org.

1.1 Status

This is the version 1.0 alpha release of ViRBO. The "alpha" designation means that the infrastructure of the VxO is nearly complete.

In general, many ViRBO web pages are functional. Some features are not available for all data sets. (And some features will not be available for some data sets.) To see an example of a data set with all of the possible features enabled (except the fully enabled data subsetting and filtering server), please see the SPIDR/AMIE page. A more complete list of pending projects is listed in the Active Projects section.

1.2 About

ViRBO (Virtual Radiation Belt Observatory) is one of the domain-specific virtual observatories that began operations in Fall, 2006 and is funded under the NASA Heliophysics Data Environment program. ViRBO provides access to data products and services related to Earth's radiation belt. As part of this project, we have developed or extended a number of existing software codebases. These codebases have cross-VxO uses, and we are developing them to be freely re-usable by data providers and other virtual observatories.

• Autoplot - Visualize many space science data resources.
• VxOware - The metadata search and edit engine that ViRBO uses.
• TSDS - A fast server of Time Series Data Sets.

2 News

Old (and searchable) news items are available in our metadata catalog [1].

If you have a news item, please send it to [2] or virbo@virbo.org. If you have write permission, you may add it directly to one of the sections at [3].

For more information on keeping up-to-date, see [4].

3 Data

Use the links below to access and view data. The inventory links are given when the data are accessible through ViRBO and can be used to see the time intervals when data are available. Data sets with Inventory links are available as merged files (all data in a single file, see Notes#Merged_files for help with these files); subsetting will eventually be available through the ViRBO API (in alpha testing). Other data sets are available in their original file formats only. The metadata for these data sets are stored at ViRBO's metadata site, http://virbo.org/meta. See the Active Projects section for information about data products and services that we are working on. ViRBO's ftp site is ftp://virbo.org.

4 Resources

• Radiation Belt Science Resources
• Radiation Belt Software Resources
• General Virtual Observatory Resources

5 Projects

These are some of the projects that we are working on and/or are looking for ways to get implemented by building on existing resources and tools or by attaching to existing services. This list is a result of discussions with members of the radiation belt community at the GEM Radiation Belt Focus Group meetings. As we research the best way to solve these problems, we will add more specific details about progress and plans. For updates, see #News.

5.1 Metadata

• Verification and review of all metadata.
• Create structured metadata out of draft and notes pages.
• General clean-up, normalization, and simplification of web pages.

5.2 Services

Near-term:

• A service and data set that provides L and L* for common satellites.
• A service that performs orbit calculations on AE-8 and AP-8 models. Implement a service that does the same for AE-9 and AP-9 models
• L-sort plots for browse products

Long-term:

• A service that simplifies data assimilation for radiation belt models.
• A service that "flies" a real or notional spacecraft through a 4-D environment (e.g., the output of a data assimilation or simulation). In this use case, it's often necessary to time shift the orbit or the simulation to ensure they overlap, which requires additional calculation.
• A principal component (PC) calculator and plotter. Given a 4-D data cube, compute and plot the 3-D PCs, and also plot the time series of PC amplitudes. Similar to computing the boundary normal or minimum variance coordinates. The algorithms are simple, although they do invoke eigenvalue/eigenvector factorization (which is widely available). The results would be stored as a new data product.

5.3 Data

For existing contributed RB data that have been made available in CDF, convert to PRBEM-CDF.

Near-term:

• CRRES – Make available for interactive browse
• HEO 1 and 3 - Make available for interactive browse
• LANL/LANL – Make available for interactive browse
• OV1-19 and OV3-3 - Make available for browse
• Polar/TIMAS - Make available for interactive browse
• SAMPEX - Make available for interactive browse
• S33 - Make available for interactive browse
• SCATHA – Make available for interactive browse
• SYM-H and ASYM-H geomagnetic indices – complete metadata.

Other data we are looking into:

• Full resolution NOAA-14- data – Conversion to CDF and metadata. Now have binary file readers.
• THEMIS SSD – Metadata and implementation and make available as a browse product
• DSX – Metadata and make available as a browse product
• International Space Station dosimeter data
• SAMPEX - post-mission status and document data location/availability
• DEMETER - Metadata
• Orsted - Metadata
• CHAMP - Metadata
• ROSAT - Metadata
• TWINS ES – Metadata
• TOPEX - Metadata
• Any data not on the above list but listed at [5].

5.4 Tools

5.4.1 For metadata services

• Documentation of the metadata API [6]
• Combining metadata management tools (VxOware, SPASE editor, SPASE-QL tools, etc.)

5.4.2 For data services

We are participating in the development of a high-performance data server that allows fast filtering and subsetting - see http://timeseries.org/. This software is expected to simplify implementation of some of the above-listed data services.

5.4.3 For visualization

• Interactive plotting in the browser. See http://autoplot.org/applet_guide for a demo
• Multiple panel plotting of data sets that span different missions

6 FAQ

6.1 What is a Virtual Observatory?

Recent articles related to Virtual Observatories include Weigel et. al, 2009 and Baker et. al, 2008. See also the Senior Review Proposal [7]. From [8]:

6.2 What is ViRBO’s long-term plan?

Most of the NASA Virtual Observatories were funded under 3-year proposals, and this period ended in 2009. The Virtual Observatories underwent a review in Fall 2009, and are expected to receive continued support. A more detailed plan for ViRBO is being prepared. More generally, from the review proposal [9]: "The primary function, then, of the VxOs in the next two to three years will be to complete resource descriptions and easy access to the data within their domains. Some of the latter task will be separated out into activities that clearly cross domains, and thus will be funded separately ..." Check the HPDE page for updates.

6.3 Why does ViRBO host files?

The "small box" Virtual Observatory concept (see next question for definition) says that a virtual observatory should only provide services on data in remote locations.

Our objective is to simplify the process of doing radiation belt research. Sometimes this requires doing things in a "non-VxO" way until a service becomes available that allows us to simultaneously do things the "VxO way" and meet user demands.

ViRBO locally holds two types of data.

1. Data stored in temporary cache. This is data that would have been returned from a web service or a data provider, but the request would have taken too long to complete. For small data sets, it is more efficient and reliable to serve data from ViRBO's server, especially when we want to do transforms or filtering or to combine data from multiple locations. Ideally, there would be a service that provided our required speed and reliability and ViRBO would connect to this service. We are participating in the development of software that will enable such a service [10].
2. The second is data in merged files. These are usually created for (a) users who want to do large-scale statistical studies on a very large data set that is difficult to create or (b) for validation and testing purposes on data analysis programs (e.g., QinDenton). That is, some of the merged files contain data that are used as inputs to data analysis programs. In order to reliably determine how changes to the analysis algorithms affect the output, the input must be the same.

Another reason that we locally hold data files is as a service to the community. Some of the data sets are from users at institutions that do not have easy access to a public ftp site. It was much easier for them to share their (generally small) data files using ViRBO than to set up a ftp site of their own.

In addition, we took on many of these data sets to fulfill obligations from our original proposal to develop ViRBO. Eventually these data sets should be migrated to a more appropriate place.

6.4 What is the “small box” concept?

Figure 1 The “small box” Virtual Observatory concept says that a virtual observatory should only provide services for data in remote locations.

The “small box” Virtual Observatory concept says that a virtual observatory should only provide services for data in remote locations, as shown in Figure 1 from [11].

In the first three years of operations, many virtual observatories have been involved in activities that fall outside of the “small box”. This happened for two reasons. First, some of the virtual observatories included such activities in their original 3-year proposal. Second, this happens when a tool or resource that a VxO needs to enable a service for their community does not exist, so they develop it on their own.

6.5 What is SPASE?

SPASE - (Space Physics Search and Extract) - a metadata model for Heliophysics data and related resource types which can describe each type of resource and their relationships.

SPASE is a metadata model (standardized descriptors) for heliophysics data and related resource types. For any given resource type, the data model specifies a set of "attributes" and their definitions and, for some attributes, specifies enumerated lists of valid values along with the definition of each.

SPASE descriptive standards are expected to facilitate registering, finding, understanding and accessing heliophysics data products, or parts thereof, now distributed across a broad heliophysics data environment. The "understanding" will come partly from SPASE-descriptors and partly from more detailed documentation files pointed to in SPASE descriptors.

SPASE is a description language for describing the holdings of scientific archives and the related entities necessary to find, understand, extract make use of their data holdings.

SPASE descriptions allow interested parties to identify if there are data of interest available, where it is available from, and how to make use of it once the data are acquired.

SPASE allows scientific archives to provide a consistent description for use by access and visualization tools to find and make use of their data holdings.

SPASE isn't about making data systems, but in describing systems that hold data (or possibly, process data, that seemed to be a bit of a debate at one point) for use by other systems.

Accessing individual values in data records will sometimes be enabled by SPASE descriptor content, but in other cases will need supplementary extra-SPASE information.

SPASE is NOT a data system NOR a language. It is at the least a set of standard descriptors and protocols for heliophysics-data metadata. It is designed for use by data systems (e.g., VxOs) to provide interoperable data search, access, retrieval and/or delivery functionalities.

The SPASE Group [12] develops and defines the SPASE metadata model and provides links to the basic tools that aid in the implementation and use of SPASE. The primary goal of SPASE group is to provide a metadata model for heliophysics that will allow VxOs, Resident Archives, and Active Archives to communicate at a high level about people, observatories, repositories, and instruments. Resources include people, observatories, instruments, repositories, and data. The metadata model will include sufficient semantics to enable scientifically relevant searches. The metadata model will include an abstract data model which will support the description of data content at a parameter level. The SPASE Group is to provide specifications to enable the free and open exchange of resources. This includes a query language compatible with the SPASE metadata model and services to support the registration, finding, and accessing and presentation of resources. The SPASE group will provide specifications for the expression of all persistent information in a common, accessible form (currently XML).

SPASE Metadata Working Group (SMWG) - The SPASE Metadata Working Group [13] is a group that manages a distributed and versioned repository of high-level SPASE metadata and encourages common conventions in its implementation.

The SPASE Query Language - [14] is being developed and will provide a semantic model for queries of SPASE repositories.