Installation

ManiGuard runs in the behavior conda env — OmniGibson simulation, BDDL, teleop, task generation, scripted data generation, and eval. Policy training/serving runs in each model's own environment (openpi / GR00T / SmolVLA — see the Fine-Tuning pages).

System requirements

	Requirement
OS	Linux x86_64 (tested on Ubuntu 22.04 / 24.04); fully headless servers are supported (OMNIGIBSON_HEADLESS=1)
GPU	NVIDIA RTX-class GPU (Isaac Sim requires ray-tracing hardware); tested on RTX 3090 / 4080 / 4090
NVIDIA driver	A tested 5xx-series driver; 580.x is verified — see the setup-specific compatibility note below
CUDA toolkit	Not needed system-wide — the behavior env ships its own runtime (torch 2.6.0 + cu124)
VRAM	16 GB runs the full eval pipeline on one card — simulation + the heaviest policy server (π0 / π0.5) peaks at ~13 GB measured (GR00T ~11 GB, SmolVLA ~6 GB). The simulation client alone needs only ~4 GB, so an 8 GB card works when the policy server sits on another GPU or machine
RAM	≥ 32 GB (64 GB comfortable)
Disk	≥ 80 GB free: BEHAVIOR assets ~36 GB + the behavior conda env ~22 GB + benchmark & headroom

Driver compatibility in the tested ManiGuard stack

ManiGuard uses the OmniGibson-pinned Isaac Sim 4.5 stack and was developed and validated on 5xx-series NVIDIA drivers. In our setup, driver versions newer than 580 produced repeatable startup segfaults across multiple machines and GPU models. This is an observed compatibility issue in the tested stack, not a general driver limit documented by Isaac Sim. We recommend using a tested 5xx-series configuration; driver 580.x with CUDA 12.x userland is verified.

RTX 50-series (Blackwell) GPUs

sm_120 needs a newer torch than the env default: after setup, replace torch with 2.7.0 + cu128 inside the behavior env. The rest of the stack is unchanged.

1. Clone with submodules

git clone --recursive https://github.com/NU-IDEAS-Lab/ManiGuard.git
cd ManiGuard
# or, if already cloned:
git submodule update --init --recursive

2. Install BEHAVIOR-1K + dataset

This runs upstream's setup.sh from inside the submodule. --dataset downloads the encrypted assets into behavior-1k/datasets/, which matches OmniGibson's default resolver — no env var needed afterwards.

cd behavior-1k
./setup.sh --new-env --omnigibson --bddl --joylo --dataset --eval --primitives
cd ..

Available flags: --omnigibson, --bddl, --joylo, --dataset, --eval, --asset-pipeline, --primitives, --dev. Dependencies: --omnigibson requires --bddl; --primitives and --dataset require --omnigibson; --eval requires --omnigibson + --joylo.

3. Install ManiGuard (editable)

conda activate behavior
pip install -e .                 # base
pip install -e ".[serve]"        # with policy-server extras

The pinned OmniGibson installation supplies SciPy, trimesh, OpenCV and huggingface-hub; upstream's --eval step installs PyAV. Command-line video tools also require FFmpeg. Installing ManiGuard alone does not install these simulator dependencies.

For browser-based grasp annotation and mesh preview, install pip install -e '.[annotation]' (viser, SciPy, trimesh and matplotlib). Mesh extraction still requires the simulator and its installed assets.

RAW demonstration conversion uses LeRobot 0.3.3 to write the v2.1 dataset format. These are distinct version numbers. Use a separate Python 3.10 environment so conversion dependencies do not replace the simulator or model training environment:

conda create -n maniguard-convert python=3.10 -y
conda activate maniguard-convert
pip install -e '.[conversion]'
python -m maniguard.data.datagen.to_lerobot --help

Use the model-specific environments for training and policy serving.

4. ManiGuard-Bench + robot asset

To run the benchmark, download two artifacts released with ManiGuard. Both are hosted on Hugging Face and are separate from the Stanford-licensed BEHAVIOR asset bundle.

4a. Robot asset (required). The benchmark uses a Franka Panda with extended fin-ray fingers — not part of the stock OmniGibson robot set. Drop it into the robot-assets tree so the runtime patch (maniguard._omnigibson_patches) finds it. It must land at <data_root>/omnigibson-robot-assets/models/franka/franka_panda_longfinger/, where <data_root> is behavior-1k/datasets/ (or $OMNIGIBSON_DATA_PATH):

hf download IDEAS-Lab-Northwestern/franka-panda-longfinger --repo-type dataset \
  --local-dir behavior-1k/datasets/omnigibson-robot-assets/models/franka/franka_panda_longfinger

On import maniguard, FrankaPanda.usd_path is auto-redirected to this bundle when present (it ships the OmniGibson runtime USD + cuRobo description; no URDF needed at runtime). Set MANIGUARD_SKIP_LONGFINGER=1 to keep the stock Franka instead.

A missing bundle fails silently

If the directory is absent, the stock Franka hand loads with no error — and policies trained on ManiGuard data will approach objects but never quite grasp them. Verify the path above exists before your first eval or datagen run.

4b. Benchmark scenes. The frozen benchmark (per-task scene_ep1.json + diagnostics.jsonl + review videos) lives at IDEAS-Lab-Northwestern/ManiGuard-Bench. maniguard.eval.benchmark accepts the HF repo id directly (snapshot-downloaded into the HF cache) or a local directory:

# A) let eval pull it (run `hf auth login` first)
python -m maniguard.eval.benchmark --benchmark-root IDEAS-Lab-Northwestern/ManiGuard-Bench ...
# B) or download once to the family runner's default location
hf download IDEAS-Lab-Northwestern/ManiGuard-Bench --repo-type dataset \
  --local-dir outputs/lerobot_datasets/maniguard-bench

Option B's target is where scripts/eval_family.sh looks by default (any other path works via BENCH_ROOT=...). Next step: Run the benchmark.

Optional: override the dataset path

Only needed if you keep BEHAVIOR assets outside the repo (e.g. HPC shared storage). Default resolves to behavior-1k/datasets/.

export OMNIGIBSON_DATA_PATH=/abs/path/to/datasets

Other environment variables

For headless deployment:

Variable	Purpose
ISAAC_PATH	Path to Isaac Sim package
OMNIGIBSON_DATA_PATH	Path to BEHAVIOR datasets (override only)
BEHAVIOR_PATH	Path to ManiGuard root
OMNIGIBSON_HEADLESS=1	Required for server / headless rendering
VK_ICD_FILENAMES	Vulkan ICD config for headless GPU rendering
CUDA_VISIBLE_DEVICES	GPU selection (esp. when transitioning between multi-GPU tasks)

Common issues

PhysX CUDA error 700

Set CUDA_VISIBLE_DEVICES=0 to pin a single GPU.

typing_extensions errors with torch 2.6.0

Remove the outdated typing_extensions from Isaac Sim so conda's version is used.

Vulkan ERROR_INCOMPATIBLE_DRIVER

Fix VK_ICD_FILENAMES to point to a valid local ICD JSON.

CUDA OOM

The simulator wants most of a GPU. Put the policy server on a second GPU (CUDA_VISIBLE_DEVICES), or lower camera_resolution in the eval config.