Institut d'informatique

2023, Gkikopoulos, Panagiotis, Schiavoni, Valerio, Kropf, Peter, Josef Spillner

Les systèmes pilotés par les données deviennent rapidement un paradigme de premier plan, l’avènement de l’IA et des systèmes cyber-physiques intelligents devenant une caractéristique déterminante de l’époque moderne. On dit souvent que la qualité des données qui entrent dans ces systèmes est le principal déterminant du comportement et des décisions qu’ils produisent. Il est donc primordial de fournir de meilleures stratégies de gestion et d’amélioration de la qualité des données pour les systèmes autonomes. Un autre attribut de l’infrastructure moderne basée sur les données est sa nature hautement distribuée. En effet, les déploiements de cloud, d’IoT et de continuum/fog sont presque omniprésents dans la pratique actuelle. Dans de nombreux cas, les systèmes pilotés par les données susmentionnés sont déployés sur ce type d’infrastructure en premier lieu. Nous voyons une opportunité de tirer parti de l’ubiquité des ressources de calcul distribuées pour ajouter une couche d’assurance qualité aux données. Notre travail s’inspire de deux sources principales. D’une part, la nécessité de collecter de manière cohérente des données brutes de haute qualité et des informations et mesures dérivées, en présence d’erreurs, de problèmes et d’autres interférences. D’autre part, nous nous inspirons de la fusion de données, une méthodologie couramment utilisée pour combiner des données provenant de différentes sources afin d’obtenir des informations de meilleure qualité que la somme de leurs parties. Nous envisageons une généralisation de la fusion de données à tous les formats d’ensembles de données, en particulier lorsqu’elles sont obtenues en doublons redondants par des observateurs indépendants. Nous appelons ce type d’observation une "observation multiperspective". Notre méthodologie de base consiste à concevoir, mettre en oeuvre et évaluer ce concept d’observation multi-perspective. La première partie est un système d’observateurs indépendants, représentés comme des noeuds dans une architecture distribuée, des collaborateurs dans un projet crowdsourcé ou même simplement des capteurs matériels dans une configuration traditionnelle de fusion de capteurs. Nous commençons par présenter la première partie de cette stratégie d’observation, l’acquisition de données. Nous présentons notre premier scénario motivant, qui consiste à suivre l’évolution de l’écosystème cloud-native dans un observatoire distribué "démocratique". Nous fournissons ensuite notre implémentation de cet observatoire et présentons son utilisation pour comprendre et améliorer le support matériel dans les images Docker. En outre, nous discutons de l’intégration de notre système d’acquisition de données avec des outils de reproductibilité et de preuve des données centrés sur la science des données. Ce travail nous permet également d’étudier les limites et les défis liés à l’obtention de données de la part d’observateurs indépendants. Nous utilisons nos résultats pour développer nos méthodologies dans la partie suivante de ce travail. Nous présentons ensuite la solution que nous proposons pour résoudre les problèmes de qualité des données découverts : Consensus centré sur les données (DCC). En utilisant notre système d’acquisition de données et les données que nous avons obtenues, nous développons une architecture de système pour fusionner les observations en une vue commune de la vérité, sur laquelle tous les observateurs sont d’accord. Nous étudions ensuite les algorithmes que nous pouvons utiliser pour y parvenir, ainsi que les implications de notre système en termes de performances. Enfin, nous nous concentrons sur les algorithmes eux-mêmes et présentons notre propre contribution à l’espace des électeurs définis par logiciel, l’algorithme de vote AVOC, et VDX, une spécification générique pour décrire les électeurs définis par logiciel. Nous évaluons notre contribution à la fois par rapport et en conjonction avec l’état de l’art dans un exemple de fusion de capteurs pour montrer que les algorithmes de vote peuvent en effet augmenter à la fois la qualité des résultats et la performance lorsqu’ils sont correctement combinés avec des approches de fusion de données standard. Dans l’ensemble, cette thèse présente la stratégie de l’observation multi-perspective dans une approche de bout en bout, de l’acquisition des données à la réconciliation des conflits et à la détermination des gains de qualité des données. Nous montrons où cette méthodologie est applicable et fournissons une mise en oeuvre du modèle ainsi qu’une évaluation de ses performances et de ses limites. ABSTRACT Data-driven systems are quickly becoming a prominent paradigm, with the advent of AI and smart cyber-physical systems becoming a defining characteristic of the modern day. It is often stated that the quality of data going into such systems is the primary determinant of the behaviour and decisions they produce. It is thus paramount to provide better strategies for managing and improving data quality for autonomic systems. Another attribute of modern data-driven infrastructure is its highly distributed nature. Indeed cloud, IoT and continuum/fog deployments are almost ubiquitous in current practice. In many cases, the above-mentioned data-driven systems are deployed to such infrastructure in the first place. We see an opportunity to leverage the ubiquity of distributed compute resources to add a layer of quality assurance to data. Our work is inspired by two main sources. On one hand, the need to consistently collect high-quality raw data and derived insights and metrics, in the presence of faults, issues and other interference. On the other hand, we draw inspiration from data fusion, a methodology commonly used to combine data from different sources to obtain insights of higher quality than the sum of its parts. We envision a generalisation of data fusion to all formats of datasets, particularly when obtained in redundant duplicates from independent observers. We call such an observation, a ’multiperspective observation’. Our core methodology is to design, implement and evaluate this concept of multiperspective observation. The first part is a system of independent observers, represented as nodes in a distributed architecture, collaborators in a crowdsourced project or even just hardware sensors in a traditional sensor-fusion setup. We begin by presenting the first part of this observation strategy, data acquisition. We show our first motivating scenario, tracking the evolution of the cloud-native ecosystem in a ’democratic’ distributed observatory. We then provide our implementation of this observatory and present its use in understanding and improving hardware support in Docker images. Further, we discuss the integration of our data acquisition system with data science-centric reproducibility and data provenance tooling. This work also serves to help us study the limitations and challenges of obtaining data from independent observers. We use our findings to develop our methodologies for the next part of this work. Then, we introduce our proposed solution to the discovered issues in data quality: Data-Centric Consensus (DCC). Using our data acquisition system and the data we obtained, we develop a system architecture to merge the observations into a common view of the truth, that all observers agree to. We then investigate the algorithms we can use to achieve this, and the performance implications of our system. Finally, we focus on the algorithms themselves, and present our own contribution to the space of software-defined voters, the AVOC voting algorithm, and VDX, a generic specification for describing software-defined voters. We evaluate our contribution both against and in conjunction with the state-of-the-art in a sensor fusion example to show that voting algorithms can indeed augment both output quality and performance when correctly combined with standard data fusion approaches. All in all, this thesis presents the strategy of multiperspective observation in an end-to-end approach, from acquiring data, to reconciling conflicts to determining the gains in data quality. We show where this methodology is applicable and provide model implementation and an evaluation of both its performance and limitations.

2003, Bassil, Sarita, Bourbeau, Benoît, Keller, Rudolf, Kropf, Peter

This paper introduces a dynamic approach to manage the processing of client requests in a multi-transfer container transportation (MTCT) system. At the operational level, this type of system is faced with a continuously changing environment. In this context, the need for dynamic creation and adaptation of solutions is of utmost importance. The adopted approach is based on a two-layer framework that exploits workflow technology. The latter proposes a formalism to describe sequences of activities to be enacted when processing requests, hence reducing the need for manual, time-consuming management and organization. The proposed two-layer framework has a workflow layer that encapsulates the set of concurrently running workflows associated to client requests. A coordination layer is mainly responsible for the instantiation of new workflows to be inserted in the workflow layer and for modifications of running ones. These modifications are motivated by resource sharing issues or triggered by unanticipated/unexpected events. According to this two-layer framework, an implementation of a prototype for a MTCT system is finally presented.

2006, Savoy, Jacques, Abdou, Samir, Peters, Carol

For our participation in the CLEF 2006 campaign, our first objective was to propose and evaluate a decompounding algorithm and a more aggressive stemmer for the Hungarian language. Our second objective was to obtain a better picture of the relative merit of various search engines for the French, Portuguese/ Brazilian and Bulgarian languages. To achieve this we evaluated the test-collections using the Okapi approach, some of the models derived from the Divergence from Randomness (DFR) family and a language model (LM), as well as two vector-processing approaches. In the bilingual track, we evaluated the effectiveness of various machine translation systems for a query submitted in English and automatically translated into the French and Portuguese languages. After blind query expansion, the MAP achieved by the best single MT system was around 95% for the corresponding monolingual search when French was the target language, or 83% with Portuguese. Finally, in the robust retrieval task we investigated various techniques in order to improve the retrieval performance of difficult topics.

2009-4-1, Patel, Jay A., Rivière, Etienne, Gupta, Indranil, Kermarrec, Anne-Marie

In this paper, we present the design, implementation and evaluation of Rappel, a peer-to-peer feed-based publish-subscribe service. By using a combination of probabilistic and gossip-like techniques and mechanisms, Rappel provides noiselessness, i.e., updates from any feed are received and relayed only by nodes that are subscribers of that feed. This leads to a fair system: the overhead at each subscriber node scales with the number and nature of its subscriptions. Moreover, Rappel incurs small publisher and client overhead, and its clients receive updates quickly and with low IP stretch. To achieve these goals, Rappel exploits “interest locality” characteristics observed amongst real multi-user multi-feed populations. This is combined with systems design decisions that enable nodes to find other subscribers, and maintain efficient network locality-aware dissemination trees. We evaluate Rappel via both trace-driven simulations and a PlanetLab deployment. The experimental results from the PlanetLab deployment show that Rappel subscribers receive updates within hundreds of milliseconds after posting. Further, results from the trace-driven simulator match our PlanetLab deployment, thus allowing us to extrapolate Rappel’s performance at larger scales.

2006, Abdou, Samir, Savoy, Jacques

2012, Savoy, Jacques, Zubaryeva, Olena

2002, Vaucher, Jean, Babin, Gilbert, Kropf, Peter, Jouve, Thierry, Plaice, John, Kropf, Peter, Schulthess, Peter, Slonim, Jacob

Computer networks or distributed systems in general may be regarded as communities where the individual components, be they entire systems, application software or users, interact in a shared environment. Such communities dynamically evolve with components or nodes joining and leaving the system. Their own individual activities affect the community's behavior and vice versa. This paper discusses various practical experiments undertaken to investigate the behavior of a real system, the Gnutella network, which represents such a community. Gnutella is a distributed Peer-to-Peer data-sharing system without any central control. It turns out that most interactions between nodes do not last long and much of their activity is devoted to finding appropriate partners in the network. The experimental results presented have been obtained from a Java implementation of Gnutella running in the open Internet environment, and thus in unknown and quickly changing network structures heavily depending on chance.

1996, Kropf, Peter, Aguilar, Marc, Noel, B.

1996-1-22, McGill, Stuart, Juker, Daniel, Kropf, Peter

This study tested the possibility of obtaining the activity of deeper muscles in the torso-specifically psoas, quadratus lumborum, external oblique, internal oblique and transverse abdominis, using surfce myoelectric electrodes. It was hypothesized that: (1) surface electrodes adequately represent the amplitude of deep muscles (specifically psoas, quadratus lumborum, external oblique, internal oblique, transverse abdominis); (2) a single surface electrode location would best represent the activation profiles of each deep muscle over a broad variety of tasks. We assumed that prediction of activation within 10% of maximum voluntary contraction (RMS difference between the surface and intramuscular channels), over the time history of the signal, was reasonable and acceptable to assist clinical interpretation of muscle activation amplitude, and ultimately for modeled estimates of muscle force. Surface electrodes were applied and intramuscular electrodes were inserted on the left side of the body in five men and three women who then performed a wide variety of flexor tasks (bent knee and straight leg situps and leg raises, curl ups), extensor tasks (including lifting barbells up to 70 kg), lateral bending tasks (standing lateral bend and horizontal lying side support), twisting tasks (standing and sitting), and internal/external hip rotation. Using the criteria of RMS difference and the coefficient of determination (R(2)) to compare surface with intramuscular myoelectric signals, the results indicated that selected surface electrodes adequately represent the amplitude of deep muscles-always within 15% RMS difference, or less with the exception of psoas where differences up to 20% were observed but only in certain maximum voluntary contraction efforts. It appears reasonable for spine modelers, and particularly clinicians, to assume well selected surface electrode locations provide a representation of these deeper muscles - as long as they recognize the magnitude of error for their particular application. Copyright (C) 1996 Elsevier Science Ltd.

2010, Savoy, Jacques