If the actual value-gradients, G, (the cyan lines) match the target value-gradients, G', (the magenta lines) in magnitude and direction at every point along the trajectory then the trajectory will be optimal, as shown in this figure (i.e. the dark blue trajectory curve matches the green theoretical optimal trajectory)

This optimality principle is proven in Appendix A of the paper, and its close relation to Pontryagin's Maximum Principle is described there.