Logo du site
  • English
  • Français
  • Se connecter
Logo du site
  • English
  • Français
  • Se connecter
  1. Accueil
  2. Université de Neuchâtel
  3. Publications
  4. SENTINEL: Taming Uncertainty with Ensemble-based Distributional Reinforcement Learning
 
  • Details
Options
Vignette d'image

SENTINEL: Taming Uncertainty with Ensemble-based Distributional Reinforcement Learning

Auteur(s)
Hannes Eriksson
Debabrota Basu
Mina Alibeigi
Dimitrakakis, Christos 
Institut d'informatique 
Date de parution
2021-02-22T14:45:39Z
In
Computing Research Repository (CoRR)
Vol.
2102.11075
Mots-clés
  • Machine Learning (cs.LG)
  • Machine Learning (cs....

Résumé
In this paper, we consider risk-sensitive sequential decision-making in Reinforcement Learning (RL). Our contributions are two-fold. First, we introduce a novel and coherent quantification of risk, namely composite risk, which quantifies the joint effect of aleatory and epistemic risk during the learning process. Existing works considered either aleatory or epistemic risk individually, or as an additive combination. We prove that the additive formulation is a particular case of the composite risk when the epistemic risk measure is replaced with expectation. Thus, the composite risk is more sensitive to both aleatory and epistemic uncertainty than the individual and additive formulations. We also propose an algorithm, SENTINEL-K, based on ensemble bootstrapping and distributional RL for representing epistemic and aleatory uncertainty respectively. The ensemble of K learners uses Follow The Regularised Leader (FTRL) to aggregate the return distributions and obtain the composite risk. We experimentally verify that SENTINEL-K estimates the return distribution better, and while used with composite risk estimates, demonstrates higher risk-sensitive performance than state-of-the-art risk-sensitive and distributional RL algorithms.
Identifiants
https://libra.unine.ch/handle/123456789/30949
_
10.48550/arXiv.2102.11075
_
2102.11075v3
Type de publication
journal article
Dossier(s) à télécharger
 main article: 2102.11075.pdf (894.17 KB)
google-scholar
Présentation du portailGuide d'utilisationStratégie Open AccessDirective Open Access La recherche à l'UniNE Open Access ORCIDNouveautés

Service information scientifique & bibliothèques
Rue Emile-Argand 11
2000 Neuchâtel
contact.libra@unine.ch

Propulsé par DSpace, DSpace-CRIS & 4Science | v2022.02.00