SARSA follows the current policy. Suppose you're minimizing cost. Then, if the value function of the MDP is a lower bound, it will explore interesting states si

SARSA follows the current policy. Suppose you're minimizing cost. Then, if the value function of the MDP is a lower bound, it will explore interesting states simply because SARSA underestimates the cost-to-go. If updates of the value function then tighten this lower bound, SARSA will converge. In this case, SARSA is more efficient than Q-learning. Apart from that using Q-factors does not scale well. If your action space is a game controller, things may still look ok, but not if your action space is multi-dimensional and continuous.