Vignette d'image

Dimitrakakis, Christos

Nom
Dimitrakakis, Christos
Affiliation principale
Fonction
Professor
Email
christos.dimitrakakis@unine.ch
Identifiants
https://libra.unine.ch/handle/123456789/30936
_
0000-0002-5367-5189

Résultat de la recherche

Filtres

5
5
32
Réinitialiser les filtres

Paramètres

Voici les éléments 1 - 5 sur 5
  • Publication
    Accès libre
    Near-optimal Bayesian Solution For Unknown Discrete Markov Decision Process
    (2019-06-20T06:32:36Z)
    Aristide Tossou
    ;
    Dimitrakakis, Christos 
    ;
    Debabrota Basu
    We tackle the problem of acting in an unknown finite and discrete Markov Decision Process (MDP) for which the expected shortest path from any state to any other state is bounded by a finite number $D$. An MDP consists of $S$ states and $A$ possible actions per state. Upon choosing an action $a_t$ at state $s_t$, one receives a real value reward $r_t$, then one transits to a next state $s_{t+1}$. The reward $r_t$ is generated from a fixed reward distribution depending only on $(s_t, a_t)$ and similarly, the next state $s_{t+1}$ is generated from a fixed transition distribution depending only on $(s_t, a_t)$. The objective is to maximize the accumulated rewards after $T$ interactions. In this paper, we consider the case where the reward distributions, the transitions, $T$ and $D$ are all unknown. We derive the first polynomial time Bayesian algorithm, BUCRL{} that achieves up to logarithm factors, a regret (i.e the difference between the accumulated rewards of the optimal policy and our algorithm) of the optimal order $\tilde{\mathcal{O}}(\sqrt{DSAT})$. Importantly, our result holds with high probability for the worst-case (frequentist) regret and not the weaker notion of Bayesian regret. We perform experiments in a variety of environments that demonstrate the superiority of our algorithm over previous techniques. Our work also illustrates several results that will be of independent interest. In particular, we derive a sharper upper bound for the KL-divergence of Bernoulli random variables. We also derive sharper upper and lower bounds for Beta and Binomial quantiles. All the bound are very simple and only use elementary functions.
  • Publication
    Accès libre
    Near-Optimal Online Egalitarian learning in General Sum Repeated Matrix Games
    (2019-06-04T17:43:08Z)
    Aristide Tossou
    ;
    Dimitrakakis, Christos 
    ;
    Jaroslaw Rzepecki
    ;
    Katja Hofmann
    We study two-player general sum repeated finite games where the rewards of each player are generated from an unknown distribution. Our aim is to find the egalitarian bargaining solution (EBS) for the repeated game, which can lead to much higher rewards than the maximin value of both players. Our most important contribution is the derivation of an algorithm that achieves simultaneously, for both players, a high-probability regret bound of order O(lnT−−−√3⋅T2/3) after any T rounds of play. We demonstrate that our upper bound is nearly optimal by proving a lower bound of Ω(T2/3) for any algorithm.
  • Publication
    Accès libre
    Near-optimal Optimistic Reinforcement Learning using Empirical Bernstein Inequalities
    (2019)
    Aristide Tossou
    ;
    Debabrota Basu
    ;
    Dimitrakakis, Christos 
    We study model-based reinforcement learning in an unknown finite communicating Markov decision process. We propose a simple algorithm that leverages a variance based confidence interval. We show that the proposed algorithm, UCRL-V, achieves the optimal regret O~(DSAT−−−−−−√) up to logarithmic factors, and so our work closes a gap with the lower bound without additional assumptions on the MDP. We perform experiments in a variety of environments that validates the theoretical bounds as well as prove UCRL-V to be better than the state-of-the-art algorithms.
  • Publication
    Accès libre
    Differential Privacy for Multi-armed Bandits: What Is It and What Is Its Cost?
    (2019)
    Debabrota Basu
    ;
    Dimitrakakis, Christos 
    ;
    Aristide Tossou
    Based on differential privacy (DP) framework, we introduce and unify privacy definitions for the multi-armed bandit algorithms. We represent the framework with a unified graphical model and use it to connect privacy definitions. We derive and contrast lower bounds on the regret of bandit algorithms satisfying these definitions. We leverage a unified proving technique to achieve all the lower bounds. We show that for all of them, the learner's regret is increased by a multiplicative factor dependent on the privacy level ϵ. We observe that the dependency is weaker when we do not require local differential privacy for the rewards.
  • Publication
    Accès libre
    Algorithms for Differentially Private Multi-Armed Bandits
    (2016)
    Aristide Tossou
    ;
    Dimitrakakis, Christos 
    We present differentially private algorithms for the stochastic Multi-Armed Bandit (MAB) problem. This is a problem for applications such as adaptive clinical trials, experiment design, and user-targeted advertising where private information is connected to individual rewards. Our major contribution is to show that there exist (ϵ,δ) differentially private variants of Upper Confidence Bound algorithms which have optimal regret, O(ϵ−1+logT). This is a significant improvement over previous results, which only achieve poly-log regret O(ϵ−2log2T), because of our use of a novel interval-based mechanism. We also substantially improve the bounds of previous family of algorithms which use a continual release mechanism. Experiments clearly validate our theoretical bounds.