Course Schedule

Weekday Regular Schedule

Group Type Hours Location
01 Lecture Monday 13-16 Orentstein 111

Resources

Introduction to Multi-Armed Bandits, Aleksandrs Slivkins

Bandit Algorithms Tor Lattimore and Csaba Szepesvari

Detailed Schedule

Week Date Lecture topics references Lecture Scribe
1 Oct 11 Introduction and Stochastic Bandits Chapters 1 and Introduction from Slivkins,
Chapter 5.3 from Lattimore and Szepesvari,
Sleeping bandits from Regret bounds for sleeping experts and bandits
by Robert Kleinberg, Alexandru Niculescu-Mizil, Yogeshwer Sharma
Hebrew Notes 1
2 Oct 18 Lower Bounds MAB Chapters 2 from Slivkins,
Sleeping bandits from Regret bounds for sleeping experts and bandits
by Robert Kleinberg, Alexandru Niculescu-Mizil, Yogeshwer Sharma
Hebrew Notes 2
3 Oct 25 Bayesian bandits and Thompson sampling algorithm Chapters 3 from Slivkins,
Near-optimal Regret Bounds for Thompson Sampling
by SHIPRA AGRAWAL and NAVIN GOYAL,
Hebrew Notes 3
4 Nov 1 Lipschitz bandits Chapter 4 from Slivkins,
The Value of Knowing a Demand Curve: Bounds on Regret for On-line Posted-Price Auctions
by Robert Kleinberg and Tom Leighton
Hebrew Notes 4
5 Nov 8 Adversarial costs: Full feedback Chapter 5 from Slivkins,
Learning, Regret minimization, and Equilibria (Section 4.3)
by A. Blum and Y. Mansour
From External to internal regret (Section 7)
by A. Blum and Y. Mansour
Hebrew Notes 5
6 Nov 15 Adversarial costs: MAB Reduction: previous years class notes
THE NONSTOCHASTIC MULTIARMED BANDIT PROBLEM
by P. AUER, N. CESA-BIANCHI, Y. FREUND, and R. SCHAPIRE
Explore no more: Improved high-probability regret bounds for non-stochastic bandits
by G. Neu
Hebrew Notes 6
7 Nov 22 Linear Cost Full information:
Efficient algorithms for online decision problems Kalai and Vempala 2005
Reduction and Brycentic spanner
Online linear optimization and adaptive routing Awerbuch and Kleinberg 2008
Stochastic:
Chapter 19 Bandit Algorithms book (Lattimore and Szepesvari)
Additional resources:
Slivkins book Chapter 7
Stochastic Linear Optimization under Bandit Feedback Dani, Hayes, Kakade 2008
Hebrew Notes 7
8 Nov 29 Contextual Bandits Lecture notes from 2018 course
Efficient Algorithms for Adversarial Contextual Learning
Vasilis Syrgkanis, Akshay Krishnamurthy, Robert E. Schapire, ICML 2016
Hebrew Notes 8
9 Dec 6 Zero-sum games and applications Lecture notes from 2018/19 and 2009/10 course
Slivkins chapter 9
Hebrew Notes 9
10 Dec 13 Swap and and correlated equilibrium Lecture notes from 2018/19 Hebrew Notes 10
11 Dec 20 Bandits with Knapsacks Slivkins, chapter 10
Lower bound:
Bandits with Knapsacks
Ashwinkumar Badanidiyuru, Robert Kleinberg, Aleksandrs Slivkins
UCB like solution:
Bandits with concave rewards and convex knapsacks
Shipra Agrawal, Nikhil R. Devanur
Hebrew Notes 11
12 Dec 27 Bandits with incentives Notes 12
13 Jan 3 Gittins Index Hebrew Notes 13
Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License