Deep learning runs on cheap gradients supplied by reverse mode automatic differentiation, also known as back propagation. To avoid coding derivatives by hand, we can use automatic differentiation libraries such as PyTorch or TensorFlow.  I'll introduce the algorithm behind reverse-mode automatic differentiation.  I'll then introduce some more advanced uses of this algorithm.  By viewing back propagation through the more general lens of efficient vector-Jacobian products, we can use many useful techniques automatically: differentiating through fixed-point solvers, computing generalized Gauss-Newton products, check pointing, and deriving efficient forward-mode automatic differentiation automatically from reverse-mode.  Time permitting, I'll also discuss more exotic applications, such as differentiating through hyper networks for stochastic hyper parameter optimization, or back propagation through ODE solvers.