Many planning problems in supply chain and operations management require decisions to be made sequentially under uncertainty. Examples include sourcing, production and distribution in manufacturing, resource-constrained project scheduling in R&D, and resource planning in professional services. In this talk, I introduce a general and flexible modeling framework based on Markov decision process (MDP) to obtain dynamic and adaptive closed-loop solutions. To tackle the curse-of-dimensionalities of solving large-scale MDPs, an algorithmic approach that integrates simulation, optimization, and reinforcement learning in AI (Sim-Opt-RL) will be elaborated to offer computationally tractable high-quality solutions. Applications of Sim-Opt-RL on several variants of stochastic resource-constrained project scheduling problems (SRCPSP) will be showcased.