We will discuss theory and applications of several non-convex optimization algorithms commonly used in machine learning. These methods include diffusion based methods for finding the global optimum, matrix factorization based methods for computational and memory benefits, and dimension reduction techniques for algorithmic performance.