Reinforcement Learning

Machine Learning RL

Overview

Value-based, policy-based, and actor-critic methods for sequential decision making.

Introduction

• Markov Process

Value-Based RL

• Rescorla-Wagner Learning Rule
• Temporal Difference Learning
• Q-Learning
• Successor Representation

Policy-Based RL

• Policy Gradient Theorem
• Improved Policy Gradient Estimators

Actor-Critic RL

• Introduction

Hierarchical RL (HRL)

Options

• Options (Sutton 1999)
• Options Keyboard (Barreto 2019)

Feudal

• Feudal RL (Dayan 1992)
• Feudal Networks for HRL (Vezhnevets 2017)

Bisimulation

Distributional RL

• Introduction
• Categorical (C51)
• Expectile Regression

• Quantile Regression

• Connection to Neuroscience: Dabney et al. 2020

TODO

• https://proceedings.neurips.cc/paper/2020/file/9dd16e049becf4d5087c90a83fea403b-Paper.pdf

Distributed RL

• Ape-X
• IMPALA
• R2D2 (Kapturowski 2019)