An Online Adaptive Policy Iteration-Based Reinforcement Learning for a Class of a Nonlinear 3D Overhead Crane

We consider an online policy iteration-based reinforcement learning for a class of a nonlinear three-dimensional overhead crane with bounded uncertainties. Under the assumption that the rope length is fixed with small swing angles, a linearized model is derived. The system has four states; two actuated states: position x and y, and two un-actuated states, which are the rope angles θ_x and θ_y. The adaptive reinforcement learning controller is designed to handle the effects of measurement noises and outliers. We propose a model-free; hence it does not require precise knowledge of the system dynamics. When the state information is not available, a Kalman filter estimator is equipped with a dynamical saturation function to attenuate the effects of measurement noises and to remove outliers. A simulation study is established to illustrate the influence and robustness of the developed controller, and it can enhance the tracking trajectory under different scenarios to test the scheme.