Towards a big data exploration framework for astronomical archives

Eva Sciacca, C. Pistagna, Ugo Becciani, A. Costa, P. Massimino, S. Riggi, F. Vitello, M. Bandieramonte, Mel Krokos

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Exploiting big data astronomical archives is a mandatory and challenging activity due to dramatically increasing sizes and high complexity of datasets coming from radio telescopes or space missions. Visual exploration and discovery can be invaluable tools providing prompt and intuitive insights into the intrinsic data characteristics, enabling scientists to rapidly identify interesting areas within which to apply computationally expensive algorithms or to discover correlations in data patterns. The paper outlines a new approach for creating a user-friendly, integrated and cross-platform framework to facilitate big data access, visualization and exploration, thus empowering astrophysicists to focus on pitching new ideas for scientific advances. We present a flexible distributed architecture striking a balance between local interactive exploration tools and remote services responsible for hiding data complexity. Remote services communicate with advanced distributed computing infrastructures presenting a meaningful lightweight version of the archive dataset obtained by mining or noise filtering methods. They are interfaced with science gateway technologies in order to allow collaborative activity between users and to provide customization and scalability of data analysis/processing workflows hiding underlying technicalities. Local tools enable interactive visualization optimized for ubiquitous computing environments, intuitively controlling the resulting visualisation. The motivations behind such a framework are envisaged to meet the requirements of the exploitation of the Gaia mission outcomes and are shown in the paper by a number of case studies. The presented framework can potentially have a profound impact on astronomical and astrophysical communities in the big data era, allowing to quickly understand datasets, thus aiding in adopting novel ways for scientific discovery.
Original languageEnglish
Title of host publicationProceedings of the 2014 International Conference on High Performance Computing & Simulation (HPCS 2014)
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Print)978-1-4799-5312-7, 9781479953110
Publication statusPublished - Jul 2014
Event2014 International Conference on High Performance Computing & Simulation - Bologna, Italy
Duration: 21 Jul 201425 Jul 2014


Conference2014 International Conference on High Performance Computing & Simulation
Abbreviated titleHPCS 2014


Dive into the research topics of 'Towards a big data exploration framework for astronomical archives'. Together they form a unique fingerprint.

Cite this