Towards a big data exploration framework for astronomical archives

Exploiting big data astronomical archives is a mandatory and challenging activity due to dramatically increasing sizes and high complexity of datasets coming from radio telescopes or space missions. Visual exploration and discovery can be invaluable tools providing prompt and intuitive insights into the intrinsic data characteristics, enabling scientists to rapidly identify interesting areas within which to apply computationally expensive algorithms or to discover correlations in data patterns. The paper outlines a new approach for creating a user-friendly, integrated and cross-platform framework to facilitate big data access, visualization and exploration, thus empowering astrophysicists to focus on pitching new ideas for scientific advances. We present a flexible distributed architecture striking a balance between local interactive exploration tools and remote services responsible for hiding data complexity. Remote services communicate with advanced distributed computing infrastructures presenting a meaningful lightweight version of the archive dataset obtained by mining or noise filtering methods. They are interfaced with science gateway technologies in order to allow collaborative activity between users and to provide customization and scalability of data analysis/processing workflows hiding underlying technicalities. Local tools enable interactive visualization optimized for ubiquitous computing environments, intuitively controlling the resulting visualisation. The motivations behind such a framework are envisaged to meet the requirements of the exploitation of the Gaia mission outcomes and are shown in the paper by a number of case studies. The presented framework can potentially have a profound impact on astronomical and astrophysical communities in the big data era, allowing to quickly understand datasets, thus aiding in adopting novel ways for scientific discovery.