Hierarchical Storage Management System For Big Data Astronomical Research.

Abstract

We proposed to purchase and install a new hierarchical storage system to be used by the Astroinformatics Laboratory (AstroLab) at the Center for Mathematical Modeling (CMM) of the University of Chile (UChile). The AstroLab is an intersdisciplinary initiative recently established at CMM-UChile to tackle emerging Big Data problems in astronomy with two main lines of research: the real-time detection of transient objects with large scale cameras (e.g. DECam) and the image synthesis of radio interferometric observations (e.g. ALMA), both of which have large associated storage requirements. The requested storage system would be strongly linked to the National Laboratory for High Performance Computing (NLHPC), housed at the CMM-UChile and would be a natural complement to its expected extension on June 2nd, 2014: from 6 Teraflops to more than 60 Teraflops of computing capacity and from 8 TB to more than 200 TB of fast (Lustre) storage capacity. The proposed system would be a combination of commodity hard drives and robotically fed magnetic tapes, in a hierarchical storage management (HSM) structure from fast (Lustre) to intermediate (SATA hard drives) to slow (magnetic tapes) storage, optimized for an efficient use of the NLHPC resources under the requirements of the AstroLab.