yourSky: an interface to the National Virtual Observatory's mosaicking code

For the first time, JPL's supercomputing resources are brought to the astronomer's desktop.


By Dr. Joseph Jacob

Jet Propulsion
Laboratory


yourSky is an internet front end to the National Virtual Observatory high performance custom sky mosaicking code. It represents an effort to bring these high performance computing resources to the scientist's desktop. The yourSky system accepts custom mosaic requests from any client that has access to a web browser such as Netscape or Internet Explorer.

The yourSky request screen yourSky input screen

A mosaic request consists of the user's email address, the dataset to use (e.g. 2MASS H band or DPOSS F band), the center longitude and latitude for the mosaic, the radius to be mosaicked, the coordinate system (galactic, ecliptic, j2000 equatorial, or b1950 equatorial), the projection (approximately 30 projections commonly used in the astronomy community are supported), the resolution for the output mosaic, and the output image format (raw data or the FITS image format commonly used in the astronomy community).

The yourSky system includes a daemon that periodically checks for mosaic requests. When one is found, a mySQL database is searched to determine which input images are required to build the mosaic. For example a typical one degree square 2MASS mosaic might require only a few dozen of the 1.8 million images that are contained in the latest 2MASS public release.

yourSky architecture flow diagram The yourSky architecture for desktop access

Once yourSky determines which images are required, it checks to see if all of the images are already located on spinning disk. If not, the missing images are automatically loaded from tape in a background job that permits simultaneous processing of other requests. In other words, a tape request that takes a half hour or more doesn't hold up other jobs that do not require tape access. Once all of the images are located on disk, the parallel mosaicking software rapidly constructs the mosaic to order. When complete, an email is sent back to the user, providing him or her with the URL of the finished mosaic.

yourSky may be used to construct mosaics from nearly 10 terabytes of images in the infrared or visible wavelengths. All of the publicly released 2MASS (the Two Micron All Sky Survey) compressed and raw Atlas images are included, covering more than half the sky in three infrared bands. Also included are all three visible bands of DPOSS (Digitized Palomar Observatory Sky Survey), nearly the entire northern sky. Additional data sets will be added as they become available. The mosaics may be constructed at any resolution down to the full resolution of the data, 1 arc second for both 2MASS and DPOSS. The yourSky system automatically retrieves the raw images necessary to build the mosaic from various tape archive systems. The 2MASS compressed images are retrieved from JPL's tape silo, which has a 128 processor SGI Origin 2000 as the front end. The larger 2MASS Atlas images are retrieved from the San Diego Supercomputing Center's (http://www.sdsc.edu) Storage Resource Broker ( http://www.npaci.edu/DICE/SRB/index.html). The DPOSS plates are retrieved from the HPSS silo at Caltech's Center for Advanced Computing Research (http://www.cacr.caltech.edu/). The yourSky system organizes these high performance computing resources, tracks data locations, manages local disk caches, and performs the parallel computations necessary to construct the requested mosaics.

The National Virtual Observatory (NVO) is being built by the astronomy community, with emphasis on many interoperable components, existing in a distributed high-performance, scalable environment. The clients themselves can be anywhere and do not need specialized or high performance capabilities. The dramatic growth in aperature and focal plane capabilities of institutionally managed observatories has resulted in an avalanche of data, both image and catalog. NVO is conceptualized to deal with these data by developing a layered, interoperable software architecture wedded to the Internet II with its national network of computational resources.

The next steps for the yourSky team will be to develop interoperability protocols that will allow Compute Nodes such as yourSky to interoperate with other tools and services being developed as part of the NVO. The diagram shows how compute nodes are envisioned in the NVO architecture

NVO Architecture Kernel diagram NVO Architectural Kernel

Archive nodes host image and catalog data and provide access services. Catalog federation services relate multiple catalogs to each other. Compute nodes use emerging high performance computing resources (e.g. NASA's computational grid - the Information Power Grid) for tasks such as custom on-the-fly image mosaicking or cluster analysis to isolate and identify interesting objects. Visualization clients access these archives and computational services. The NVO universal standards enable seamless access and interoperability of all services across multiple surveys.

The URL for yourSky is http://yoursky.jpl.nasa.gov/.


See related articles:

"Digital Sky Virtual Observatory demonstrated at Supercomputing 2000," Issue 59, April 2001
"Virtual Observatories of the Future," S.G. Djorgovski, R. Brunner, A. Szalay, Issue 57, 2000
Digital Sky Survey Uncovers Rare Celestial Objects," Issue 52, 1999