7. Semi-gradient Sarsa

  • In this lab you will implement you first Stochastic Gradient Descent algorithm: Semi-gradient Sarsa (Related topic).

  • This time we will use a gym environment 'FrozenLake-v1'.

  • You can create th environment as simply as:

env = gym.make('FrozenLake-v1', map_name="4x4", is_slippery=False, render_mode="ansi")

  • Get familiar with the problem: doc.

7.1. Semi-gradient Sarsa

  • Implement the semi-gradient Sarsa algorithm.

  • Create a class SemiGradientSarsa

  • Create a constructor __init__(env, alpha, eps):

    • env: the gym environment.

    • eps: is the \(\epsilon\) parameter.

    • alpha: is the step size parmeter.

  • Create a function train(self, episodes) that will run the training.

7.2. Experiments

  • For different \(episodes\):

    • Run the algorithm.

    • Draw the evolution of the sum of rewards for both algorithms.

  • Draw the weights vector.