Fear Field: Adaptive constraints for safe environment transitions in Shielded Reinforcement Learning

Abstract

Shielding methods for Reinforcement Learning agents show potential for safety-critical industrial applications. However, they still lack robustness on nominal safety, a key property for safety control systems. In the case of a significant change in the environment dynamic, shielding methods cannot guarantee safety until their inherent dynamics model is updated to the new scenario. The agent could reach risky states because the model cannot predict well. These situations could lead to catastrophic outcomes, such as damage to the cyber-physical system or loss of human lives, which are not allowed on safety-critical applications. The novel method presented in this paper, Fear Field, replicates human behaviour in those scenarios, adapting safety constraints whenever a drastic environmental change is introduced. Fear Field reduces safety violations by one order of magnitude compared to an RL agent implementing only a shield.