Gatewind: Vectorized Drone RL Environments

Gatewind is an RL-first 3D quadrotor environment for waypoint tracking, procedural gate courses, wind disturbances, and trained-policy replay. It is built around vectorized NumPy dynamics so training and evaluation can run many environments per process without needing a heavyweight simulator.

The project exists to make fast robotics learning loops inspectable. Dynamics, tasks, controllers, training, evaluation, trajectory recording, and rendering are separate layers with JSON artifacts between them.

Architecture

sim/
  dynamics.py      batched quadrotor integration
  geometry.py      quaternion math

envs/
  waypoint.py      point tracking task
  gate_course.py   procedural gate racing task

controllers.py     deterministic baseline policies
eval.py            batched evaluation metrics
trajectory.py      headless rollout recording
render.py          SVG, 2D HTML, and 3D HTML viewers
training/ppo.py    Torch PPO plus BC and DAgger-style warm starts
wrappers/          optional Gymnasium API

The drone state is batched across environments: position, velocity, orientation quaternion, and angular velocity. The action is stabilized rather than raw motor command: roll-rate, pitch-rate, yaw-rate, and thrust. That keeps the task focused on trajectory control while preserving enough dynamics to make wind, overshoot, and gate alignment matter.

Tasks

Waypoint3D is the learnability check. A policy has to reach a target without building unsafe velocity or leaving bounds.

GateCourse3D creates procedural courses from gate centers and normals. A pass is detected by segment-plane crossing plus radial error inside the gate radius. Named presets cover different difficulty modes:

Wind is part of the environment config. Constant wind and deterministic sinusoidal gusts are supported, with a separate seeded RNG stream so enabling gusts does not change the underlying course seed.

Training path

The training path is explicit:

1. collect rollouts from vectorized NumPy environments,
2. compute GAE,
3. run clipped PPO updates in Torch,
4. save checkpoint, config, and JSONL metrics,
5. evaluate with deterministic policy actions,
6. record a trajectory,
7. render the same trajectory artifact.

Plain waypoint PPO was unstable in early runs. It moved toward the target, built too much velocity, overshot, then left bounds. Observation normalization, zero-output actor initialization, and lower initial action noise helped, but the reliable path was behavior-cloning warm start plus conservative PPO.

For gates, one-step behavior cloning was not enough. The policy matched initial expert actions but failed under rollout because small altitude errors compounded. A DAgger-style loop fixed the state-distribution problem by collecting policy-visited states and relabeling them with the baseline controller.

Current audit numbers

Current local audited checkpoint numbers from the project docs:

The easy gate preset is close to saturated. The next useful experiment is mixed-course training across easy, sweep, vertical, slalom, default, and mixed presets, with per-preset evaluation rather than one aggregate score.

Throughput

Recent local benchmark results from the repo docs:

The point of the vectorized design is not physical realism at all costs. It is fast rollout, controlled randomization, and reproducible artifacts for comparing controllers and policies.

Replay artifacts

The portfolio demo path writes:

• trajectory.json: recorded state, action, reward, and task data,
• trajectory_3d.html: dependency-free interactive 3D viewer,
• trajectory_snapshots.html: static replay frames,
• summary.json: machine-readable episode summary,
• validation.json: demo validation output.

The 3D viewer is generated from trajectory JSON, not live environment state. That makes baseline, trained-policy, and future external-policy replays use the same rendering path.

Related notes

PID Tuning and Step Response, IMU Calibration and Drift, and Design Verification and Test.