policy function

A policy function $\pi (a|s)=\mathcal{P}[A_{t=a}|S_t=s]$ describes the confidence and preference of an agent when taking an action $a$ given the current state $s$. At any state $s$, an agent following the policy $\pi$ will take the action $a$ with the highest $\pi (a|s)$ value. Note that the policy function is time-independent.

Effects of following a policy

• $⟨\mathcal{S},{\mathcal{P}}^{\pi }⟩$ behaves like a Markov chain.
• $⟨\mathcal{S},{\mathcal{P}}^{\pi },R^{\pi },\gamma ⟩$ is a Markov reward process.