Generative dynamics on manifold.
For robotics: Planning. Control. Learning.
Website · Documentation · Get started · Examples · Learning · Environments · Constraints · Architecture · Roadmap
Full-quality video · Demonstrations and sources · Run the examples
GenerativeDynamics connects generative models, manifold geometry, and robot learning in one modular robotics stack. Learn reusable priors, shape generative trajectory updates with dynamics and constraints, and connect plans to closed-loop control. Shared task, solver and execution interfaces let you swap algorithms, compose controllers and add robots. Use components directly in Python or run complete experiments from a configuration.
- 2026-10-09 — Added GPU and Docker quickstarts, a batched constraint example, and explicit device checks.
- 2026-10-09 — Added a direct Python planning example, a unified paper and hardware gallery, learning workflows, and planner, environment and constraint catalogues.
- 2026-10-05 — Prepared the V1 release candidate with six task families, optional integrations, user documentation and CPU validation evidence.
Follow CHANGELOG.md for new environments, planners and integrations.
| Feature | What you get |
|---|---|
| 🧠 Robot learning | PPO/SAC policy training, reusable checkpoints and PPO horizon priors for MGA. |
| ✨ Generative models | Learned trajectory diffusion and model-based generative inference; DDPM/DDIM and flow-style reverse transports. |
| 🧭 Planning | Interchangeable planners for constrained trajectories, sampling MPC and contact-rich motion–impedance optimization. |
| 🤖 Tasks and robots | Planar navigation, 7-DoF avoidance, quadruped footholds, humanoid corridors, peg insertion and surface scanning. |
| 🛡️ Geometry and constraints | Constraint manifolds, convex primitives, meshes, signed-distance geometry and CSG; collision, state, action and contact constraints. |
| 🌐 Simulation adapters | MuJoCo, MJX, Brax and D3IL integrations, with physics and task adapters separate from solver logic. |
| 🧮 Batched constraints | Evaluate and correct candidate trajectories inside generative inference with batched JAX kernels. |
| ⚡ Compute | JAX planning and batched rollouts; CPU and CUDA installation paths. Dedicated Torch integrations for DPCC and SafeDiffuser. |
| 🎛️ Execution components | Robot I/O, controllers, trajectory followers, governors, safety filters, recovery policies and observers. |
| 📊 Experiment tooling | YAML task matrices, seeds, resumable runs, resolved configurations, metrics, traces, replay and reports. |
| 🔌 Extension points | Registered planners, environments, geometry, metrics and visualizations; optional integrations with explicit dependencies. |
See device and integration support for the qualified paths, and architecture for the interfaces behind these features.
Create a planar environment, give it a 2GO planner, and move from (0.8, 0.8)
to the origin. This example runs on CPU without a simulator or model checkpoint.
git clone https://github.com/hhhhzl/genedynamics.git
cd genedynamics
python -m venv .venv
source .venv/bin/activate
python -m pip install -e ".[optimization]"import jax
import numpy as np
from genedynamics.core.backends.runtime import RuntimeBackendManager
from genedynamics.envs import make_env, make_energy
from genedynamics.solvers import TwoGOSolver
RuntimeBackendManager.set_backend("jax", device="cpu")
env = make_env("single_integrator_box_2d")
planner = TwoGOSolver(
dynamics=env,
energy=make_energy("single_integrator_box_2d"),
backend=RuntimeBackendManager.get_backend(),
dt=env.dt,
horizon=20,
Nsample=128,
Ndiffuse=16,
)
state = np.array([0.8, 0.8], dtype=np.float32)
for step in range(60):
key = jax.random.fold_in(jax.random.PRNGKey(0), step)
plan = planner.solve(state, horizon=20, rng_key=key)
state = env.transition(state, plan.actions[0])
if np.linalg.norm(state) < 0.1:
break
print(f"Goal error: {np.linalg.norm(state):.3f}")The CPU validation reaches the goal in 48 steps, with Goal error: 0.099.
Run the complete example with python examples/plan_to_goal.py, or follow the
Python API walkthrough to adapt it.
On Linux with an NVIDIA GPU, install the CUDA stack and increase the candidate batch without changing the application loop:
python -m pip install -e ".[optimization]" \
-r requirements/gpu-jax.txt "jax[cuda12]==0.6.2"
JAX_PLATFORMS=cuda python examples/plan_to_goal.py --device gpu --samples 1024
JAX_PLATFORMS=cuda python examples/batched_constraints.py --device gpu --samples 4096The examples check array placement and fail when the requested GPU is unavailable. The GPU recipes cover MDOC, MD-COAS, 2GO and batched constraints. MGA CUDA qualification is deferred.
With the existing development images, mount the checkout and run the same loop:
docker run --rm -v "$PWD:/workspace" -w /workspace \
genedynamics/dev-cpu:local python examples/plan_to_goal.py --device cpu
docker run --rm --gpus all -v "$PWD:/workspace" -w /workspace \
-e JAX_PLATFORMS=cuda genedynamics/train-gpu:local \
python examples/plan_to_goal.py --device gpu --samples 1024The NVIDIA container requires Linux and the NVIDIA Container Toolkit. Results remain in the mounted checkout. See the Docker guide for prerequisites, batch examples and headless simulation.
For configured experiments, the first-run guide covers result artifacts and resuming runs. The installation guide covers CUDA, simulator assets, learned planners and deployment extras.
Start with one of our four methods. Each recipe connects an algorithm to a task, its dependencies and an inspectable configuration.
| Method | Explore | Starting configurations |
|---|---|---|
| MDOC | Diffusion planning with safety projections in constrained scenes. | Planar navigation |
| MD-COAS | Constraint-guided diffusion, from planar obstacles to 7-DoF avoidance. | Planar navigation · Arm avoidance |
| 2GO | Geometry-guided trajectory optimization for constrained locomotion. | Quadruped stepping stones · Humanoid corridor |
| MGA | Learned policy priors and generative motion–impedance control for contact-rich tasks. | Peg insertion · Surface scanning |
The recipe catalogue explains setup and assets. The homepage also includes original paper demonstrations: MDOC multi-robot coordination is available in its original project, and MGA humanoid contact footage previews research beyond the qualified V1 recipes. See the demo map for the distinction.
Generative models are part of the architecture: learn a trajectory distribution, reuse a policy as a control-sequence prior, or perform model-based generative inference directly from dynamics and objectives.
| Workflow | How it connects | Start here |
|---|---|---|
| Learn a policy, refine its proposals | Train PPO/SAC policies; reuse PPO checkpoints as MGA horizon priors for constrained, model-based refinement. | MGA learning workflow |
| Learn a trajectory diffusion model | Train on offline sequences, then guide denoising with DPCC projections or SafeDiffuser constraints. | Learned diffusion workflow |
| Plan without a pretrained model | Use physics rollouts, objectives and constraints to construct model-based generative updates. | MDOC, MD-COAS and 2GO recipes |
Learning dependencies and checkpoint setup are documented in the learning guide.
Six task families share the planning and evaluation stack. Each links to a ready-to-inspect configuration.
| Environment | What you can explore | Example |
|---|---|---|
| Planar navigation | Point-robot planning in non-convex scenes, with state and action bounds. | MD-COAS |
| 7-DoF arm avoidance | D3IL obstacle avoidance through Cartesian or joint-velocity interfaces. | MD-COAS |
| Quadruped stepping stones | Body motion, foot placement and gait phase across discrete footholds. | 2GO |
| Humanoid corridor | Body position, posture and arm clearance through narrow passages. | 2GO |
| Surface scanning | Contact tracking, stiffness and force regulation across surfaces. | MGA |
| Peg insertion | Contact-rich insertion under pose, clearance, friction and sensing variations. | MGA |
The environment catalogue lists configuration keys, interfaces, assets and implementation links. MuJoCo, MJX and Brax are simulation adapters; support depends on the selected recipe.
🟢 Ours marks our algorithms. Each entry links its original paper or current implementation; configuration variants are documented in the planner reference.
| Algorithm | What it does | Source |
|---|---|---|
| MDOC 🟢 Ours | Model-based diffusion with control-barrier-function projections inside dynamics rollouts. | Paper · Original multi-robot project |
| MD-COAS 🟢 Ours | Combines an augmented-Lagrangian feasibility prior, CFS projection, and adaptive constraint scheduling. | Paper |
| 2GO 🟢 Ours | Shapes generative trajectory updates and exploration with active constraint geometry for constrained locomotion. | Paper |
| MGA 🟢 Ours | Combines RL sequence proposals, model-based evaluation, and realization-aware geometry for motion–impedance control. | Implementation |
| MBD | Uses known dynamics and Monte Carlo score estimates to optimize trajectories without demonstrations. | Paper · Project |
| EB-MBD | Introduces emerging barriers during model-based diffusion to retain useful samples under constraints. | Paper |
| MPPI | Updates sampled control sequences using exponential rollout-cost weights. | Paper |
| DIAL-MPC | Uses diffusion-inspired annealing to refine sampled controls in receding-horizon MPC. | Paper |
| ATACOM | Learns and executes actions in the tangent space of an augmented constraint manifold. | Paper |
| ISSA | Filters proposed actions through a derivative-free implicit-safe-set search. | Paper |
| PegasusFlow | Uses rolling denoising and weighted spline-basis updates for model-based trajectory optimization. | Paper |
| DPCC | Adds model-based projections and constraint tightening to learned trajectory diffusion. | Paper |
| SafeDiffuser | Applies control-barrier-function corrections within learned diffusion denoising. | Paper |
Naming: CFS-MBD is the historical implementation name for MD-COAS. Learned integrations require their optional dependencies and checkpoints. See planner setup and implementation notes.
Compose CBF action filters, CFS trajectory projections and adaptive augmented-Lagrangian scheduling with a numerical solver below.
| Solver | What it does | Source |
|---|---|---|
| Closed form | Euclidean halfspace projection, box clipping and unconstrained solves for special cases. | Implementation |
| JAXopt OSQP | General convex-QP interface built on jaxopt.OSQP. |
Implementation · Docs |
| OSQP | Sparse CPU convex-QP solves through the OSQP Python interface. | Implementation · Docs |
| CVXOPT | CPU convex-QP solves, including trajectory smoothness and slack formulations. | Implementation · Docs |
Install .[optimization] for the optional QP dependencies. See the
constraint reference for CBF/CFS variants,
scheduling and solver-specific limits.
A planning call maintains a batch of candidate trajectories. Constraints participate in the sampling loop: they correct candidates, change feasibility weights, or shape the geometry of the next generative update.
| Axis | How the work composes |
|---|---|
Candidates N |
JAX vmap applies supported rollout, filter and projection kernels across trajectories. |
Horizon H |
Stateful filters use scan: correct an action, propagate dynamics, then evaluate the next state. Full-horizon CFS couples the sequence. |
Refinement K |
Constraint feedback shapes successive denoising or optimization steps. |
MDOC/MD-COAS compose CBF/CFS corrections; 2GO budgets projection probes; MGA evaluates learned horizon proposals alongside model-based candidates. SafeDiffuser uses batched Torch QPs during denoising; DPCC combines Torch sampling with CPU trajectory projections. Numerical paths are documented per integration.
See batched constraints and generative samples for tensor shapes, each planner's data flow and a runnable JAX batch example.
Define a robot task, then compose planner plugins, constraints and geometry, and learned models or priors. Run through the experiment framework or direct Python API. Environment adapters connect to simulation; explicit trajectory adapters connect selected plans to the separate deployment stack.
Observers, metrics and replay span the workflow. Saved trajectories and outcomes support offline policy training, diffusion-model training and reliability calibration. Each workflow selects the components it needs; model-based planners can also run without learned checkpoints.
Read the architecture guide, configure a task, or add a plugin.
- Policy learning, learned diffusion and reusable priors
- JAX planning and composable constraints
- Six task families and planner baselines
- Configured experiments, metrics and replay
- MGA CUDA qualification and humanoid walking/push
- Torch planner parity; Rust/C++ control and I/O
- Panda/xArm7 controllers and
ros2_control - Async planning/control and fault recovery
- VLA proposals and OpenPI/LeRobot adapters
- Live perception and dynamic scenes
- Reproducible hardware benchmarks
Roadmap details · Current compatibility · Validation evidence
Bring a planner, a robot task, an integration or a documentation improvement. Start with CONTRIBUTING.md. Community guidelines are in CODE_OF_CONDUCT.md; vulnerability reporting is in SECURITY.md.
The solvers build on research algorithms; the framework provides the interfaces and runtime around them. Use CITATION.cff when citing the project, and the papers linked above when using individual methods.
GenerativeDynamics is released under the MIT License. Bundled third-party components retain their own licenses; see third-party notices.

