Testing reinforcement learning systems: A comprehensive review

Reinforcement Learning (RL) enables autonomous decision-making in dynamic environments, making it suited for complex, high-stakes domains like healthcare and defense systems. However, RL's high dimensionality and non-deterministic behavior pose testing challenges. This study presents the first literature review on testing RL systems, analyzing 49 studies published between 2013 and May 2025. The review categorizes testing RL techniques based on key workflow components: testing objectives, test generation, test oracles, and test adequacy. It identifies eleven primary gaps, including the lack of validation for testing RL frameworks in real-world applications and the need for specialized testing to verify RL-specific objectives, such as fairness and generalization. Additionally, the review highlights four key challenges: stochasticity leading to inconsistent fault detection, scalability and efficiency constraints in testing adequacy, fault identification complexity due to RL-specific failure definitions, and validation limitations due to reliance on simple tasks and underdeveloped test oracles. Our analysis shows that current research focuses on single-agent RL, robustness, and safety, yet these areas still contain gaps that require further exploration. The findings highlight that testing RL has become an active research area, peaking in 2023 and 2024, with 57% of the reviewed papers published these years. The identified challenges and gaps present opportunities for future research, guiding efforts toward more comprehensive and effective methodologies for testing RL.