Logo du site
  • English
  • Français
  • Se connecter
Logo du site
  • English
  • Français
  • Se connecter
  1. Accueil
  2. Université de Neuchâtel
  3. Notices
  4. Lessons Learned from Applying Big Data Paradigms to a Large Scale Scientific Workflow
 
  • Details
Options
Vignette d'image

Lessons Learned from Applying Big Data Paradigms to a Large Scale Scientific Workflow

Auteur(s)
Kropf, Peter 
Institut d'informatique 
Maison d'édition
: CEUR-WS.org
Date de parution
2016-11-14
De la page
54
A la page
58
Mots-clés
  • Scientific workflows
  • Big Data
  • Cloud Computing
  • Apache Spark
  • Hydrology
  • Scientific workflows

  • Big Data

  • Cloud Computing

  • Apache Spark

  • Hydrology

Résumé
The increasing amount of data related to the execution of scientific workflows has raised awareness of their shift towards parallel data-intensive problems. In this paper, we deliver our experience with combining the traditional high-performance computing and grid-based approaches for scientific workflows, with Big Data analytics paradigms. Our goal was to assess and discuss the suitability of such data-intensive-oriented mechanisms for production-ready workflows, especially in terms of scalability, focusing on a key element in the Big Data ecosystem: the data-centric programming model. Hence, we reproduced the functionality of a MPI-based iterative workflow from the hydrology domain, EnKF-HGS, using the Spark data analysis framework. We conducted experiments on a local cluster, and we relied on our results to discuss promising directions for further research.
Notes
, 2016
Nom de l'événement
11th Workshop on Workflows in Support of Large-Scale Science, Supercomputing
Lieu
Salt Lake City
Identifiants
https://libra.unine.ch/handle/123456789/25176
Autre version
http://ceur-ws.org/Vol-1800/short1.pdf
Type de publication
conference paper
google-scholar
Présentation du portailGuide d'utilisationStratégie Open AccessDirective Open Access La recherche à l'UniNE Open Access ORCIDNouveautés

Service information scientifique & bibliothèques
Rue Emile-Argand 11
2000 Neuchâtel
contact.libra@unine.ch

Propulsé par DSpace, DSpace-CRIS & 4Science | v2022.02.00