Shielded Reinforcement Learning: A review of reactive methods for safe learning

Abstract

Reinforcement Learning (RL) algorithms are showing promising results in simulated environments, but their replication in real physical applications, even more so in safety-critical applications, is not yet guaranteed. Ensuring the functional safety of RL algorithms is not a trivial task since the physical integrity of the target system, also called environment, especially when there is interaction with humans, may depend on it. Among the methods recently developed with the objective of guaranteeing safety, Shielded Reinforcement Learning is defined, which defines an interaction mechanism if the action event proposed by the agent causes a non-safe state. This article provides an overview of the different Shielding Reinforcement Learning approaches. In addition to summarising the techniques used by each of them, their advantages and disadvantages are discussed. Finally, the shortcomings associated with Shielded Reinforcement Learning methods that can lead to risk or unsafe situations are discussed.