# Recommended reading

I recommend the following references to students interested in artificial intelligence research.

My current research is focused on developing principled but scalable Bayesian reinforcement learning methods. A typical Bayesian reinforcement learning method represents its knowledge about an environment by a probability distribution over (one-step dynamics) models and uses this knowledge to seek optimal actions. If you would like to pursue a PhD under my supervision, please skim the highlighted references and my selected publications.

## Graduate level

- Probabilistic graphical models (notes):
- D. Koller, N. Friedman.
*“Probabilistic Graphical Models: Principles and Techniques”*. MIT Press, 2009.

- D. Koller, N. Friedman.
- Supervised learning and unsupervised learning (notes):
- C. Bishop.
*“Pattern Recognition and Machine Learning”*. Springer-Verlag, 2007. - K.P. Murphy.
*“Probabilistic Machine Learning: An Introduction”*. MIT Press, 2022.

- C. Bishop.
- Reinforcement learning and decision making (notes):
- R.S. Sutton, A.G. Barto.
*“Reinforcement Learning: An Introduction”*. MIT Press, 2018. - D.P. Bertsekas.
*“Reinforcement Learning and Optimal Control*”. Athena Scientific, 2019. - M.J. Kochenderfer, T.A. Wheeler, K.H. Wray.
*“Algorithms for Decision Making”*, MIT Press, 2022. - C. Dimitrakakis, R. Ortner.
*“Decision Making Under Uncertainty and Reinforcement Learning”*. Springer, 2022. **Bayesian reinforcement learning (notes, Secs. 14 and 15)**:**M. Ghavamzadeh, S. Mannor, J. Pineau, A. Tamar.***“Bayesian Reinforcement Learning: A Survey”*. Foundations and Trends in Machine Learning, 2015.- A. Guez.
*“Sample-based Search Methods for Bayes-Adaptive Planning”*. PhD thesis, 2015. - I. Osband.
*“Deep Exploration via Randomized Value Functions”*. PhD Thesis, 2017.

- Scalable exploration methods:
- A.P. Badia, B. Piot, S. Kapturowski, P. Sprechmann, A. Vitvitskyi, D. Guo, C. Blundell.
*“Agent57: Outperforming the Atari Human Benchmark”*, International Conference on Machine Learning, 2020.*[Note: follow the references in the introduction.]*

- A.P. Badia, B. Piot, S. Kapturowski, P. Sprechmann, A. Vitvitskyi, D. Guo, C. Blundell.
- Scalable model-based reinforcement learning:
- J. Schrittwieser, I. Antonoglou, T. Hubert, K. Simonyan, L. Sifre, S. Schmitt, A. Guez, E. Lockhart, D. Hassabis, T. Graepel, T. Lillicrap, D. Silver.
*“Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model”*. Nature, 2020. - D. Hafner, T. Lillicrap, M. Norouzi, J. Ba.
*“Mastering Atari with Discrete World Models”*. International Conference on Learning Representations, 2020.

- J. Schrittwieser, I. Antonoglou, T. Hubert, K. Simonyan, L. Sifre, S. Schmitt, A. Guez, E. Lockhart, D. Hassabis, T. Graepel, T. Lillicrap, D. Silver.

- R.S. Sutton, A.G. Barto.
- Artificial intelligence:
- S. Legg.
*“Machine Super Intelligence”*. PhD thesis, 2008.

- S. Legg.

## Undergraduate level

- Calculus (notes):
- T.M. Apostol.
*“Calculus, Volume 1”*. Wiley, 1991. - T.M. Apostol.
*“Calculus, Volume 2”*. Wiley, 1969.

- T.M. Apostol.
- Linear algebra (notes):
- S. Axler.
*“Linear Algebra Done Right”*. Springer, 1997.

- S. Axler.
- Probability theory (notes, Sec. 2):
- D. Bertsekas, J.N. Tsitsiklis.
*“Introduction to Probability”*. Athena Scientific, 2008.

- D. Bertsekas, J.N. Tsitsiklis.
- Algorithms:
- T.H. Cormen, C.E. Leiserson, R. Rivest, C. Stein.
*“Introduction to Algorithms”*. MIT Press, 2022.

- T.H. Cormen, C.E. Leiserson, R. Rivest, C. Stein.
- Theory of computation:
- T. Sipser.
*“Introduction to the Theory of Computation”*. Course Technology, 2012.

- T. Sipser.
- Artificial intelligence:
- S. Russell, P. Norvig.
*“Artificial Intelligence: A Modern Approach”*. Pearson, 2020.

- S. Russell, P. Norvig.