Skip to content

Towards a big data exploration framework for astronomical archives

Research output: Chapter in Book/Report/Conference proceedingConference contribution

  • Eva Sciacca
  • C. Pistagna
  • Ugo Becciani
  • A. Costa
  • P. Massimino
  • S. Riggi
  • F. Vitello
  • M. Bandieramonte
  • Dr Mel Krokos
Exploiting big data astronomical archives is a mandatory and challenging activity due to dramatically increasing sizes and high complexity of datasets coming from radio telescopes or space missions. Visual exploration and discovery can be invaluable tools providing prompt and intuitive insights into the intrinsic data characteristics, enabling scientists to rapidly identify interesting areas within which to apply computationally expensive algorithms or to discover correlations in data patterns. The paper outlines a new approach for creating a user-friendly, integrated and cross-platform framework to facilitate big data access, visualization and exploration, thus empowering astrophysicists to focus on pitching new ideas for scientific advances. We present a flexible distributed architecture striking a balance between local interactive exploration tools and remote services responsible for hiding data complexity. Remote services communicate with advanced distributed computing infrastructures presenting a meaningful lightweight version of the archive dataset obtained by mining or noise filtering methods. They are interfaced with science gateway technologies in order to allow collaborative activity between users and to provide customization and scalability of data analysis/processing workflows hiding underlying technicalities. Local tools enable interactive visualization optimized for ubiquitous computing environments, intuitively controlling the resulting visualisation. The motivations behind such a framework are envisaged to meet the requirements of the exploitation of the Gaia mission outcomes and are shown in the paper by a number of case studies. The presented framework can potentially have a profound impact on astronomical and astrophysical communities in the big data era, allowing to quickly understand datasets, thus aiding in adopting novel ways for scientific discovery.
Original languageEnglish
Title of host publicationProceedings of the 2014 International Conference on High Performance Computing & Simulation (HPCS 2014)
ISBN (Print)978-1-4799-5312-7, 9781479953110
Publication statusPublished - Jul 2014
Event2014 International Conference on High Performance Computing & Simulation - Bologna, Italy
Duration: 21 Jul 201425 Jul 2014


Conference2014 International Conference on High Performance Computing & Simulation
Abbreviated titleHPCS 2014

Related information

Relations Get citation (various referencing formats)

ID: 4178485