Mechanical and Civil Engineering Seminar

A seamless integration of stochastic models and data with computational decision-making, able to directly and autonomously offer optimal actions to decision-makers/agents is discussed in this talk. As shown, challenging sequential decision problems in nonstationary dynamic environments can be efficiently formulated along the premises of stochastic control, through Markov Decision Processes (MDPs), Partially Observable Markov Decision Processes (POMDPs), and mixed approaches thereof. In systems with relatively low dimensional state and action spaces, MDPs and POMDPs can be satisfactorily solved to global optimality through appropriate dynamic programming algorithms, such as value iteration with asynchronous updates and point-based approaches for partial observability cases. However, optimal planning for large systems with multiple components is computationally hard and severely suffers from the curse of dimensionality. New developments on Deep Reinforcement Learning (DRL) methods and their capacity of addressing this problem are discussed, with emphasis on our developed DRL formulations and novel algorithmic schemes, specifically tailored to the needs of large engineering systems, able to solve otherwise intractable problems with immense state and action spaces. DRL relations to Artificial Intelligence and Machine Learning are also explained and techniques are demystified down to their fundamental mathematical attributes, underlying computational aspects and connections to engineering. The talk concludes with ongoing efforts and numerous possible future research directions along these lines, as well as with several pertinent diverse applications, from centralized/decentralized infrastructure management, to automated vehicles and robotics, to wildfire prevention, to autonomous exploration, path planning and navigation.