Voici les éléments 1 - 10 sur 68
  • Publication
    Accès libre
    Applying big data paradigms to a large scale scientific workflow: Lessons learned and future directions
    (2020-6-1) ; ;
    Carretero, Jesus
    ;
    Caíno-Lores, Silvina
    The increasing amounts of data related to the execution of scientific workflows has raised awareness of their shift towards parallel data-intensive problems. In this paper, we deliver our experience combining the traditional high-performance computing and grid-based approaches with Big Data analytics paradigms, in the context of scientific ensemble workflows. Our goal was to assess and discuss the suitability of such data-oriented mechanisms for production-ready workflows, especially in terms of scalability. We focused on two key elements in the Big Data ecosystem: the data-centric programming model, and the underlying infrastructure that integrates storage and computation in each node. We experimented with a representative MPI-based iterative workflow from the hydrology domain, EnKFHGS, which we re-implemented using the Spark data analysis framework. We conducted experiments on a local cluster, a private cloud running OpenNebula, and the Amazon Elastic Compute Cloud (AmazonEC2). The results we obtained were analysed to synthesize the lessons we learned from this experience, while discussing promising directions for further research.
  • Publication
    Accès libre
    Integrating hydrological modelling, data assimilation and cloud computing for real-time management of water resources
    (2017-7-1) ;
    Kurtz, Wolfgang
    ;
    ; ; ; ;
    Braun, Torsten
    ;
    ;
    Vereecken, Harry
    ;
    Sudicky, Edward
    ;
    Franssen, Harrie-Jan Hendricks
    ;
    Online data acquisition, data assimilation and integrated hydrological modelling have become more and more important in hydrological science. In this study, we explore cloud computing for integrating field data acquisition and stochastic, physically-based hydrological modelling in a data assimilation and optimisation framework as a service to water resources management. For this purpose, we developed an ensemble Kalman filter-based data assimilation system for the fully-coupled, physically-based hydrological model HydroGeoSphere, which is able to run in a cloud computing environment. A synthetic data assimilation experiment based on the widely used tilted V-catchment problem showed that the computational overhead for the application of the data assimilation platform in a cloud computing environment is minimal, which makes it well-suited for practical water management problems. Advantages of the cloud-based implementation comprise the independence from computational infrastructure and the straightforward integration of cloud-based observation databases with the modelling and data assimilation platform.
  • Publication
    Métadonnées seulement
    Applications for ultrascale computing
    Studies of complex physical and engineering systems, represented by multi-scale and multi-physics computer simulations have an increasing demand for computing power, especially when the simulations of realistic problems are considered. This demand is driven by the increasing size and complexity of the studied systems or the time constraints. Ultrascale computing systems offer a possible solution to this problem. Future ultrascale systems will be large-scale complex computing systems combining technologies from high performance computing, distributed systems, big data, and cloud computing. Thus, the challenge of developing and programming complex algorithms on these systems is twofold. Firstly, the complex algorithms have to be either developed from scratch, or redesigned in order to yield high performance, while retaining correct functional behaviour. Secondly, ultrascale computing systems impose a number of non-functional cross-cutting concerns, such as fault tolerance or energy consumption, which can significantly impact the deployment of applications on large complex systems. This article discusses the state-of-the-art of programming for current and future large scale systems with an emphasis on complex applications. We derive a number of programming and execution support requirements by studying several computing applications that the authors are currently developing and discuss their potential and necessary upgrades for ultrascale execution.
  • Publication
    Restriction temporaire
  • Publication
    Restriction temporaire
  • Publication
    Métadonnées seulement
    Collaboration in scientific visualization
    (2010-1-22)
    Casera, Steve
    ;
    Mechanisms and tools able to facilitate collaboration are needed when remotely team members wish to jointly analyze data to create and acquire new knowledge. This paper discusses the main challenges involved in remote collaboration systems used in scientific visualization, and presents our solutions and assessments. To evaluate the usability and effectiveness of the collaborative tools comprising this system, we conduct an experiment on several groups of users. More particularly, we compare various forms of audio and textual communication tools as well as graphical indicators used to exchange information. Our analyses reveal how important it is to choose the means of conversation and, surprisingly, that graphical indicators have little influence. Furthermore, these experiments demonstrate that the system’s usability can be significantly improved through providing appropriate grounding and awareness support. This results in increased satisfaction, and in turn a potential increase the quality of collaborative work.
  • Publication
    Métadonnées seulement
    Application-layer traffic analysis of a peer-to-peer system
    (2008)
    Tutsch, Dietmar
    ;
    Babin, Gilbert
    ;
    Characterizing traffic behavior helps to optimize the network architecture for improved performance. Using a modified LimeWire servent ( for both the server and client) and a variance-time plot for traffic characterization, the authors analyze the Gnutella protocol's traffic shape and find that the messages exhibit a self-similar shape. This result shows network designers that they need to consider the self-similar traffic shape in their set-up - for instance, by introducing appropriate buffer sizes.
  • Publication
    Accès libre
    Application layer traffic analysis of a Peer-to-Peer system
    (2008)
    Tutsch, Dietmar
    ;
    Babin, Gilbert
    ;
    Self-similar or multifractal behavior has been observed for LAN and Internet WAN (backbone) traffic. Investigations about this kind of behavior for application level protocols are rarely found because sessions or even applications are usually too short to be characterized in this direction. Only Telnet and FTP were examined so far. This paper analyzes the traffic shape of Peer-to-Peer (P2P) networks using the Gnutella protocol. Data was collected using a modified LimeWire servent. Self-similarity was estimated using a variance-time plot. The results show that Gnutella messages exhibit a self-similar shape, regardless of the message type.
  • Publication
    Accès libre
    A demo for affinity based content delivery architectures
    (2008)
    Spielvogel, Christian
    ;
    Böszörmenyi, Laszlo
    ;
    Streaming video data over best effort networks is a challenging task concerning the quality of the received content. The quality decreases with the number of frames that are corrupted, lost or received after the playback time. The main reasons for lost, delayed or corrupted frames are overloaded streaming servers and crowded network paths.
    In order to deal with overloaded streaming servers and crowded network paths, we present the prototype of an innovative architecture called Proxy-to-Proxy (X2X). The three main components in the architecture are proxies (provided by end users), videos (replicated from original servers) and end-clients (requesting shared content). The behavior of the three components is defined by the so-called affinity model which consists of three levels. The first level covers the content replication from original to surrogate servers (also called proxies), the second level covers the collaboration between the proxies, and the third level covers the delivery from the proxies to the end-clients.
    Proxies, videos and end-clients have a certain affinity to each other. Only by changing the input parameters to the affinity functions, the overlay network behaves either like a classical 1) Content Delivery Network, 2) a Peer-to-Peer System or 3) a Proxy-to-Proxy overlay network --- which is a combination and generalization of the two former ones.