Voici les éléments 1 - 6 sur 6
- PublicationAccès libreApproaches for cloudification of complex high performance simulation systemsLe calcul scientifique est souvent associé à un besoin de ressources toujours croissant pour réaliser des expériences, des simulations et obtenir des résultats dans un temps raisonnable. Même si une infrastructure locale peut offrir de bonnes performances, sa limite est souvent atteinte par les chercheurs. Pour subvenir à ces besoins en constante augmentation, une solution consiste à déléguer une partie de ces tâches à un environnement en nuage. Dans cette thèse, nous nous intéresserons au problème de la migration vers des environnements en nuage d’applications scientifiques basées sur le standard MPI. En particulier, nous nous concentrerons sur les simulateurs scientifiques qui implémentent la méthode itérative Monte Carlo. Pour résoudre le problème identifié, nous (a) donnerons un aperçu des domaines du calcul en nuage et du calcul à haute performance, (b) analyserons les types de problèmes actuels liés à la simulation, (c) présenterons un prototype de simulateur Monte Carlo, (d) présenterons deux méthodes de cloudification, (e) appliquerons ces méthodes au simulateur Monte Carlo, et (f) évaluerons l’application de ces méthodes à un exemple d’utilisation réelle., Scientific computing is often associated with ever-increasing need for computer resources to conduct experiments, simulations and gain outcomes in a reasonable time frame. While local infrastructures could hold substantial computing power and capabilities, researchers may still reach the limit of available resources. With continuously increasing need for higher computing power, one of the solutions could be to offload certain resource-intensive applications to a cloud environment with resources available on-demand. In this thesis, we will address the problem of migrating MPI-based scientific applications to clouds. Specifically, we will concentrate on scientific simulators, which implement the iterative Monte Carlo method.
To tackle the identified problem, we will (a) overview high performance and cloud computing domains, (b) analyze existing simulation problem types, (c) introduce an example Monte Carlo simulator, (d) present two cloudification methodologies, (e) apply the methodologies to the example simulator, and (f) evaluate the potential application of methodologies in a real case study.
- PublicationMétadonnées seulementCloudification of a Legacy Hydrological Simulator using Apache Spark(2016-9-14)
; ; ;Carretero, JesusCaíno-Lores, SilvinaThe field of hydrology usually relies on complex multiphysics systems and data collected from geographically distributed sensors in order to obtain good quality predictions and analysis of how wa- ter moves through the environment. Nowadays, the computational resources needed to run such com- plex simulators, and the increasing size of datasets related to the models have arisen an interest to- wards distributed infrastructures like clouds. This paper presents the results of applying a cloudifica- tion methodology to a legacy hydrological simulator (HydroGeoSphere), wrapped with an ensemble Kal- man filter. This work describes how the methodology was applied, the particularities of its implementation and configuration for the Apache Spark iterative map- reduce platform, and the results of an evaluation in a commodity cluster against an MPI implementation of the simulator.
- PublicationAccès libreIntegrating hydrological modelling, data assimilation and cloud computing for real-time management of water resources(2017-7-1)
; ;Kurtz, Wolfgang ; ; ; ; ;Braun, Torsten ; ;Vereecken, Harry ;Sudicky, Edward ;Franssen, Harrie-Jan HendricksOnline data acquisition, data assimilation and integrated hydrological modelling have become more and more important in hydrological science. In this study, we explore cloud computing for integrating field data acquisition and stochastic, physically-based hydrological modelling in a data assimilation and optimisation framework as a service to water resources management. For this purpose, we developed an ensemble Kalman filter-based data assimilation system for the fully-coupled, physically-based hydrological model HydroGeoSphere, which is able to run in a cloud computing environment. A synthetic data assimilation experiment based on the widely used tilted V-catchment problem showed that the computational overhead for the application of the data assimilation platform in a cloud computing environment is minimal, which makes it well-suited for practical water management problems. Advantages of the cloud-based implementation comprise the independence from computational infrastructure and the straightforward integration of cloud-based observation databases with the modelling and data assimilation platform.
- PublicationAccès libreApplying big data paradigms to a large scale scientific workflow: Lessons learned and future directions(2020-6-1)
; ; ;Carretero, JesusCaíno-Lores, SilvinaThe increasing amounts of data related to the execution of scientific workflows has raised awareness of their shift towards parallel data-intensive problems. In this paper, we deliver our experience combining the traditional high-performance computing and grid-based approaches with Big Data analytics paradigms, in the context of scientific ensemble workflows. Our goal was to assess and discuss the suitability of such data-oriented mechanisms for production-ready workflows, especially in terms of scalability. We focused on two key elements in the Big Data ecosystem: the data-centric programming model, and the underlying infrastructure that integrates storage and computation in each node. We experimented with a representative MPI-based iterative workflow from the hydrology domain, EnKFHGS, which we re-implemented using the Spark data analysis framework. We conducted experiments on a local cluster, a private cloud running OpenNebula, and the Amazon Elastic Compute Cloud (AmazonEC2). The results we obtained were analysed to synthesize the lessons we learned from this experience, while discussing promising directions for further research.
- PublicationAccès libreReal-time Environmental Monitoring for Cloud-based Hydrogeological Modeling with HydroGeoSphere(: IEEE Computer Society, 2014)
; ; ; ; ; ;Jamakovic-Kapic, A. ;Braun, T.Maffioletti, S.This paper describes an architecture for real-time environmental modeling. It consists of a wireless mesh network equipped with sensors and a cloud-based infrastructure to perform real-time environmental sim- ulations using a physics-based model combined with an Ensemble Kalman Filter. The purpose of the system is to optimize groundwater abstraction close to a river. These initial studies demonstrate that the cloud infrastructure can simultaneously compute a large number of simula- tions, thus allowing for the implementation of Ensemble Kalman Filters in real-time.
- PublicationAccès libreWireless Mesh Networks and Cloud Computing for Real Time Environmental Simulations(: Springer, 2014-4-22)
; ; ; ; ;Predicting the influence of drinking water pumping on stream and groundwater levels is essential for sustainable water management. Given the highly dynamic nature of such systems any quantitative analysis must be based on robust and reliable modeling and simulation approaches. The paper presents a wireless mesh-network framework for environmental real time monitoring integrated with a cloud computing environment to execute the hydrogeological simulation model. The simulation results can then be used to sustainably control the pumping stations. The use case of the Emmental catchment and pumping location illustrates the feasibility and effectiveness of our approach even in harsh environmental conditions.