Prediction and Control with Function Approximation

University of Alberta via Coursera

Overview

Embark on a transformative journey with the "Prediction and Control with Function Approximation" course, offered by the University of Alberta through Coursera. This meticulously designed curriculum is perfect for those looking to navigate the complexities of large, high-dimensional, or potentially infinite state spaces. Discover how to turn the estimation of value functions into a supervised learning challenge, leveraging function approximation to craft agents that strike a perfect balance between generalization and discrimination to optimize rewards.

Start with an exploration of how traditional policy evaluation or prediction methodologies such as Monte Carlo and TD adapt to function approximation. Dive into the intricacies of feature construction for Reinforcement Learning (RL), and master representation learning through neural networks and backpropagation. The course culminates with an in-depth examination of policy gradient methods, offering a direct avenue to learning policies sans value function estimation. Engage in solving two continuous-state control tasks, and unpack the advantages of policy gradient methods within a continuous-action framework.

This course is a continuation of foundational learning, assuming proficiency acquired in the initial courses. Participants should be well-versed in probabilities & expectations, basic linear algebra, basic calculus, and Python 3.0 (with at least a year's experience), including the ability to implement algorithms from pseudocode.

By the conclusion of your studies, you will gain a nuanced understanding of how to employ supervised learning techniques for value function approximation, comprehend objectives for prediction under function approximation, and implement TD with function approximation. Learn the nuances of fixed basis and neural network approaches for feature construction, tackle new exploration challenges introduced by function approximation, and differentiate between discounted and average reward problem formulations for control. Furthermore, you will have the opportunity to apply expected Sarsa and Q-learning with function approximation in continuous state control tasks, understand the foundations of estimating policies directly through policy gradient objectives, and experiment with an Actor-Critic method in a discrete state environment.

Categories include Machine Learning Courses, Reinforcement Learning Courses, and Supervised Learning Courses, making it an essential educational experience for anyone eager to advance their understanding and capabilities in these domains.