Published at: IROS 2024 - International Conference on Intelligent Robots and Systems

→ Full paper on   ←

Paper summary

Recognizing failures during task execution and implementing recovery procedures is challenging in robotics. Traditional approaches rely on the availability of extensive data or a tight set of constraints, while more recent approaches leverage large language models (LLMs) to verify task steps and replan accordingly. However, these methods often operate offline, necessitating scene resets and incurring in high costs. To address this issues, we’ve developed RECOVER, a neuro-symbolic method combining ontologies, logical rules, and LLM-based planners for real-time failure recovery. By leveraging symbolic information, RECOVER enhances LLMs’ ability to generate recovery plans and also to decrease the associated costs. We demonstrated the capabilities of our method in a simulated kitchen environment, by developing a new ontology, ONTOTHOR, describing the AI2Thor simulator setting. Our empirical evaluation shows that ONTOTHOR’s logical rules accurately detect all failures in the analyzed tasks, and that RECOVER considerably outperforms, for both failure detection and recovery, a baseline method reliant solely on LLMs.

Summary Video

Recovery videos

Task 1

Task 2

Task 3

Task 4

Task 5

Task 6

Task 7

Task 8

Task 9

Task 10

Task 11

Task 12

The authors

Cristina Cornelio	Mohammed Diab
Samsung AI	University of Plymouth

How to cite

@inproceedings{RECOVER_Cornelio,
  author = {Cornelio, Cristina and Diab, Mohammed},
  title = {RECOVER: A Neuro-Symbolic Framework for Failure Detection and Recovery},
  booktitle = {IROS 2024: IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2024}}