ManiGuard¶
ManiGuard is a benchmark and data suite for specification-grounded safety evaluation and improvement of foundation model–driven robotic manipulation. It is implemented as a Python package on top of BEHAVIOR-1K / OmniGibson that adds LTL safety checking, task-generation pipelines, teleop + scripted data collection, VLA supervised fine-tuning, and policy evaluation in simulated household environments.
Visual update
New simulation runs use 640×640 DLAA rendering, with improved visibility through glass and corrected appearance changes for glass objects. The update covers benchmark previews, task generation, evaluation, scripted data collection, simulated teleop and playback. Published demonstration datasets retain their original 256×256 recordings. See rendering and source recording.
Evaluate your VLA on ManiGuard-Bench → Run the benchmark
Download the benchmark, serve your checkpoint, run a family across ID + OOD, and read the results in the paper's success × safety metrics.

200 base tasks across six families, each with one ID scenario and four OOD variants: target appearance, language, location and environment. Task success and specification-grounded safety are evaluated separately.
Explore the docs¶
-
Install the
behaviorconda env and the BEHAVIOR datasets. -
Architecture, the env layer, the LTL safety system, and OmniGibson patches.
-
The 6 bench families (clutter, cabinet, stack, jar, dusty, lid) — and how to add your own.
-
SO-101 / GELLO teleop, plus the scripted datagen pipeline for SFT demos.
-
The model-agnostic joint dataset + per-model recipes (openpi / GR00T / SmolVLA), and collection↔eval consistency.
-
Run your checkpoint on the benchmark; success + LTL-safety checkers, engagement-gated metrics.
-
The scripted 6-family demo-collection pipeline + RAW → LeRobot conversion.
Citation¶
If you use ManiGuard in your research, please cite the paper:
@misc{peng2026maniguard,
title = {{MANIGUARD}: A Benchmark and Data Suite for Specification-Grounded
Safety Evaluation and Improvement of Robotic Manipulation},
author = {Peng, Yiyan and Wang, Philip and Zhan, Simon Sinong and Lyu, Yiqi
and Ni, Zhenyang and Yan, Jixin and Wong, Fiorelli and Jiao, Ruochen
and Yin, Hang and Cao, Xinyu and Shao, Huajie and Li, Manling
and Zhang, Ruohan and Zhu, Qi},
year = {2026},
eprint = {2608.17386},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2608.17386},
}