Rivière, Etienne

2016-6-13, Ivanciu, Iustin-Alexandru, Luchian, Eduard-Florentin, Dobrota, Virgil, Rivière, Etienne

The generalization of distributed systems-of-systems lead to increasing management and operation complexity. A sound approach to deal with this complexity is to leverage overlay networks and the higher level of abstraction they allow for distributed operations. In particular, the virtualization of underlying network resources allows providing a range of reusable network services and compose them with existing systems. Recently, the notion of holons as compositional systems entities was proposed as a general framework for the programming and deployment of complex systems of systems. We investigate if this abstract model can be applied to the complex process of real-time composition of OpenStack-based IaaS clouds. In this context, we also address issues related to the energy-performance tradeoff using techniques similar to those involving dynamic service consolidations. Our solution aims at simplifying the life of infrastructure and service providers, allowing them to face the dynamicity of new demands in Future Internet.

2015-6-2, Sutra, Pierre, Rivière, Etienne, Felber, Pascal

Une construction universelle est un algorithme permettant à un ensemble de processus concurrents d’accéder à un objet partagé en ayant l’illusion que celui-ci est disponible localement. blue Nous présentons un algorithme permettant la mise en œuvre d’une telle construction dans un système à mémoire partagée. Notre construction est sans verrou, et contrairement aux approches proposées précédemment, ne nécessite pas que les processus accédant à l’objet partagé soient connus. De plus, elle est adaptative : en notant n le nombre total de processus dans le système et k < n le nombre de processus qui utilisent l’objet partagé, tout processus effectue Θ(k) pas de calcul en l’absence de contention.

2013-6-30, Maia, Francisco, Matos, Miguel, Vilaça, Ricardo, Pereira, José, Oliveira, Rui, Rivière, Etienne

Recently, tuple-stores have become pivotal structures in many information systems. Their ability to handle large datasets makes them important in an era with unprecedented amounts of data being produced and exchanged. However, these tuple-stores typically rely on structured peer-to-peer protocols which assume moderately stable environments. Such assumption does not always hold for very large scale systems sized in the scale of thousands of machines. In this paper we present a novel approach to the design of a tuple-store. Our approach follows a stratified design based on an unstructured substrate. We focus on this substrate and how the use of epidemic protocols allow reaching high dependability and scalability.

2012-5-21, Rivière, Etienne

Teaching distributed algorithms using a learning-by-doing approach is usually associated with a slow and cumbersome learning process for students. In order to test and evaluate even simple protocols, students need to learn how to set up and operate a test bed, and to write scripts for deploying, running their program and finally retrieving some logs for parsing and observation of the results. Moreover, the amount of code required in languages such as C# or Java for implementing even a simple protocol is often one order of magnitude more than the length of the protocol pseudo-code description as discussed in class or in research papers. Nevertheless, teaching distributed algorithms and protocols using real deployed code on real conditions is highly desirable and can not always be replaced by the use of simulators. We present in this paper our experience of using SPLAY, a distributed systems evaluation framework that greatly simplifies the work of both instructors and students for hands-on learning of distributed systems. SPLAY simplifies the writing, deployment and observation of distributed algorithms and the management of test environments, narrowing the complexity gap between pseudo-code descriptions and executable implementations. In addition, SPLAY's features and the focus kept on the algorithms and their evaluation, allow students to evaluate their protocols in a variety of conditions, by controlling the experiments and their running conditions, or by allowing running them on multiple test beds at no additional costs and with minimal administration complexity.

2016-6-5, Krasikova, Sveta, Carvajal Gómez, Raziel, Borba Ribeiro, Heverson, Rivière, Etienne, Schiavoni, Valerio

Self-organizing construction principles are a natural fit for large-scale distributed system in unpredictable deployment environments. These principles allow a system to systematically converge to a global state by means of simple, uncoordinated actions by individual peers. Indexing services based on the distributed hash table (DHT) abstraction have been established as a solid foundation for large-scale distributed applications. For most DHTs, the creation and maintenance of the overlay structure relies on the exploration and update of an already stabilized structure. We evaluate in this paper the practical interest of self-organizing principles, and in particular gossip-based overlay construction protocols, to bootstrap and maintain various DHT implementations. Based on the seminal work on T-Chord, a self-organizing version of Chord using the T-Man overlay construction service, we contribute three additional self-organizing DHTs: T-Pastry, T-Kademlia and T-Kelips. We conduct an experimental evaluation of the cost and performance of each of these designs using a prototype implementation. Our conclusion is that, while providing equivalent performance in a stabilized system, self-organizing DHTs are able to sustain and recover from higher level of churn than their explicitly-created counterparts, and should therefore be considered as a method of choice for deploying robust indexing layers in adverse environments.

2014-10-6, Maia, Francisco, Matos, Miguel, Vilaça, Ricardo, Pereira, José, Oliveira, Rui, Rivière, Etienne

Very large scale distributed systems provide some of the most interesting research challenges while at the same time being increasingly required by nowadays applications. The escalation in the amount of connected devices and data being produced and exchanged, demands new data management systems. Although new data stores are continuously being proposed, they are not suitable for very large scale environments. The high levels of churn and constant dynamics found in very large scale systems demand robust, proactive and unstructured approaches to data management. In this paper we propose a novel data store solely based on epidemic (or gossip-based) protocols. It leverages the capacity of these protocols to provide data persistence guarantees even in highly dynamic, massive scale systems. We provide an open source prototype of the data store and correspondent evaluation.

2013-4-1, Rivière, Etienne

Large-scale distributed systems appear as the major infrastructures for supporting planet-scale services. These systems call for appropriate management mechanisms and protocols. Slicing is an example of an autonomous, fully decentralized protocol suitable for large-scale environments. It aims at organizing the system into groups of nodes, called slices, according to an application-specific criteria where the size of each slice is relative to the size of the full system. This allows assigning a certain fraction of nodes to different task, according to their capabilities. Although useful, current slicing techniques lack some features of considerable practical importance. This paper proposes a slicing protocol, that builds on existing solutions, and addresses some of their frailties. We present novel solutions to deal with non-uniform slices and to perform online and dynamic slices schema reconfiguration. Moreover, we describe how to provision a slice-local Peer Sampling Service for upper protocol layers and how to enhance slicing protocols with the capability of slicing over more than one attribute. Slicing is presented as a complete, dependable and integrated distributed systems primitive for large-scale systems.

2015-12-7, Blair, Gordon, Bromberg, Yérom-David, Coulson, Geoff, Elkhatib, Yehia, Réveillère, Laurent, Borba Ribeiro, Heverson, Rivière, Etienne, Taïani, François

The world’s computing infrastructure is increasingly differ- entiating into self-contained distributed systems with vari- ous purposes and capabilities (e.g. IoT installations, clouds, VANETs, WSNs, CDNs, . . . ). Furthermore, such systems are increasingly being composed to generate systems of sys- tems that offer value-added functionality. Today, however, system of systems composition is typically ad-hoc and fragile. It requires developers to possess an intimate knowledge of system internals and low-level interactions between their components. In this paper, we outline a vision and set up a research agenda towards the generalised programmatic construction of distributed systems as compositions of other distributed systems. Our vision, in which we refer uniformly to systems and to compositions of systems as holons, employs code generation techniques and uses common abstractions, operations and mechanisms at all system levels to support uniform system of systems composition. We believe our holon approach could facilitate a step change in the convenience and correctness with which systems of systems can be built, and open unprecedented opportunities for the emergence of new and previously-unenvisaged distributed system deploy- ments, analogous perhaps to the impact the mashup culture has had on the way we now build web applications.

2014-9-8, Matos, Miguel, Schiavoni, Valerio, Rivière, Etienne, Felber, Pascal, Oliveira, Rui

Gossip-based live streaming is a popular topic, as attested by the vast literature on the subject. Despite the particular merits of each proposal, all need to implement and deal with common challenges such as membership management, topology construction and video packets dissemination. Well-principled gossip-based protocols have been proposed in the literature for each of these aspects. Our goal is to assess the feasibility of building a live streaming system, LAYSTREAM, as a composition of these existing protocols, to deploy the resulting system on real testbeds, and report on lessons learned in the process. Unlike previous evaluations conducted by simulations and considering each protocol independently, we use real deployments. We evaluate protocols both independently and as a layered composition, and unearth specific problems and challenges associated with deployment and composition. We discuss and present solutions for these, such as a novel topology construction mechanism able to cope with the specificities of a large-scale and delay-sensitive environment, but also with requirements from the upper layer. Our implementation and data are openly available to support experimental reproducibility.

2012-6-1, Maia, Francisco, Matos, Miguel, Rivière, Etienne, Oliveira, Rui

Slicing a large-scale distributed system is the process of autonomously partitioning its nodes into k groups, named slices. Slicing is associated to an order on node-specific criteria, such as available storage, uptime, or bandwidth. Each slice corresponds to the nodes between two quantiles in a virtual ranking according to the criteria. For instance, a system can be split in three groups, one with nodes with the lowest uptimes, one with nodes with the highest uptimes, and one in the middle. Such a partitioning can be used by applications to assign different tasks to different groups of nodes, e.g., assigning critical tasks to the more powerful or stable nodes and less critical tasks to other slices. Assigning a slice to each node in a large-scale distributed system, where no global knowledge of nodes’ criteria exists, is not trivial. Recently, much research effort was dedicated to guaranteeing a fast and correct convergence in comparison to a global sort of the nodes. Unfortunately, state-of-the-art slicing protocols exhibit flaws that preclude their application in real scenarios, in particular with respect to cost and stability. In this paper, we identify steadiness issues where nodes in a slice border constantly exchange slice and large memory requirements for adequate convergence, and provide practical solutions for the two. Our solutions are generic and can be applied to two different state-of-the-art slicing protocols with little effort and while preserving the desirable properties of each. The effectiveness of the proposed solutions is extensively studied in several simulated experiments.