Recommended reading

I recommend the following references to students interested in artificial intelligence research.

My current research is focused on developing principled but scalable Bayesian reinforcement learning methods. A typical Bayesian reinforcement learning method represents its knowledge about an environment by a probability distribution over (one-step dynamics) models and uses this knowledge to seek optimal actions. If you would like to pursue a PhD under my supervision, please skim the highlighted references and my selected publications.

Graduate level

  • Probabilistic graphical models (notes):
    • D. Koller, N. Friedman. “Probabilistic Graphical Models: Principles and Techniques”. MIT Press, 2009.
  • Supervised learning and unsupervised learning (notes):
    • C. Bishop. “Pattern Recognition and Machine Learning”. Springer-Verlag, 2007.
    • K.P. Murphy. “Probabilistic Machine Learning: An Introduction”. MIT Press, 2022.
  • Reinforcement learning and decision making (notes):
    • R.S. Sutton, A.G. Barto. “Reinforcement Learning: An Introduction”. MIT Press, 2018.
    • D.P. Bertsekas. “Reinforcement Learning and Optimal Control”. Athena Scientific, 2019.
    • M.J. Kochenderfer, T.A. Wheeler, K.H. Wray. “Algorithms for Decision Making”, MIT Press, 2022.
    • C. Dimitrakakis, R. Ortner. “Decision Making Under Uncertainty and Reinforcement Learning”. Springer, 2022.
    • Bayesian reinforcement learning (notes, Secs. 14 and 15):
      • M. Ghavamzadeh, S. Mannor, J. Pineau, A. Tamar. “Bayesian Reinforcement Learning: A Survey”. Foundations and Trends in Machine Learning, 2015.
      • A. Guez. “Sample-based Search Methods for Bayes-Adaptive Planning”. PhD thesis, 2015.
      • I. Osband. “Deep Exploration via Randomized Value Functions”. PhD Thesis, 2017.
    • Scalable exploration methods:
      • A.P. Badia, B. Piot, S. Kapturowski, P. Sprechmann, A. Vitvitskyi, D. Guo, C. Blundell. “Agent57: Outperforming the Atari Human Benchmark”, International Conference on Machine Learning, 2020. [Note: follow the references in the introduction.]
    • Scalable model-based reinforcement learning:
      • J. Schrittwieser, I. Antonoglou, T. Hubert, K. Simonyan, L. Sifre, S. Schmitt, A. Guez, E. Lockhart, D. Hassabis, T. Graepel, T. Lillicrap, D. Silver. “Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model”. Nature, 2020.
      • D. Hafner, T. Lillicrap, M. Norouzi, J. Ba. “Mastering Atari with Discrete World Models”. International Conference on Learning Representations, 2020.
  • Artificial intelligence:
    • S. Legg. “Machine Super Intelligence”. PhD thesis, 2008.

Undergraduate level

  • Calculus (notes):
    • T.M. Apostol. “Calculus, Volume 1”. Wiley, 1991.
    • T.M. Apostol. “Calculus, Volume 2”. Wiley, 1969.
  • Linear algebra (notes):
    • S. Axler. “Linear Algebra Done Right”. Springer, 1997.
  • Probability theory (notes, Sec. 2):
    • D. Bertsekas, J.N. Tsitsiklis. “Introduction to Probability”. Athena Scientific, 2008.
  • Algorithms:
    • T.H. Cormen, C.E. Leiserson, R. Rivest, C. Stein. “Introduction to Algorithms”. MIT Press, 2022.
  • Theory of computation:
    • T. Sipser. “Introduction to the Theory of Computation”. Course Technology, 2012.
  • Artificial intelligence:
    • S. Russell, P. Norvig. “Artificial Intelligence: A Modern Approach”. Pearson, 2020.