Skip to main content

Performance and validation

Paired CPU policy latency

CPU policy-step measurement​

On 2026-11-04, G1 + MimicLite v1.1 + walk1_subject1, CPU ONNX Runtime 1.23.2, Intel Core i9-13900H, Python 3.10.0, GCC 13.3.0 Release (-O3). Both processes were pinned to CPU 0, with one ORT intra/inter-op thread and one OMP/MKL/OpenBLAS thread. The native build was sim-only, without a viewer.

Timed regionPython p50 (ms)Bolt Runtime p50 (ms)Python max (ms)Bolt Runtime max (ms)
Inference adapter0.5157760.4229531.7272890.984647
Full policy step1.4584840.4489373.1909761.029220

The median policy step is 3.25× faster on this workload. Inference uses the same CPU ORT implementation in both runtimes; much of the improvement comes from observation/reference processing and runtime overhead, rather than a new inference engine. This is not a general speedup claim for every policy.

Five alternating Python/C++ trials each used 200 warmup and 2,000 measured steps, yielding 10,000 measured samples per runtime. All measured samples are retained; p50 is pooled across trials and max is the largest observed sample. The shared development host was not an isolated real-time test machine; CPU frequency and scheduling can affect absolute values, especially maxima.

A native headless rollout supplied one fixed robot-feedback stream. Each runtime replayed it outside the timer and consumed the same checkpoint and motion clip. Python called the existing IntegratedPolicyRuntime.step() implementation; Bolt Runtime used its replay backend. This measures policy computation, not separate closed-loop trajectories. The Python checkout was not modified; source hashes identify its tested working-tree implementation.

  • Full policy step: reference sampling, observations/history, inference, action mapping and command construction/submission. Physics stepping, feedback input, rendering, recording, startup and deliberate waits are excluded.
  • Inference adapter: CPU ONNX Run plus each adapter's binding/copy overhead; this is not kernel-only inference latency.

Raw samples, metadata and hashes, and vector chart are included. Native latency collection is available with --timings outputs/latency.csv; see usage.

Onboard CPU (RP1, RK3588)​

On 2026-11-05 the same workload ran on the RP1's onboard computer: RK3588, pinned to one Cortex-A76 core at 2.4 GHz (performance governor), ONNX Runtime CPU with one thread, Python 3.10 with the original sim2real runtime. Five alternating trials of 200 warmup and 2,000 measured steps per runtime.

Timed regionPython p50 (ms)Bolt Runtime p50 (ms)Python max (ms)Bolt Runtime max (ms)
Inference adapter0.8240.7300.9670.806
Full policy step2.0390.7872.3190.867

The median policy step is 2.59× faster on this core. As on the desktop, the gain is in observation, reference and runtime overhead; inference itself uses the same ONNX Runtime. Raw samples: rp1-samples.csv.

Python migration parity​

The frozen 2026-11-02 same-state/reference-window check covered 15 external configurations × 32 ticks. Maximum absolute action difference was 4.035234451293945e-5 (TWIST2). See per-configuration results. This validates policy adapters separately from motion decoding or tracking quality. After the task/namespace restructure, all 15 exports also passed native 32-tick loading/execution checks.

Build and integration coverage​

A clean local clone built sim-only and G1+RP1 configurations; 11/11 CTest checks passed in each. G1 and RP1 passed --validate without opening devices. The clean build completed G1 MimicLite walk1_subject1 headless playback (13,065 ticks, no deadline misses). This is a runtime smoke test, not aligned tracking-quality scoring. PTY terminal controls/restoration and canonical ZMQ continuity/disconnect checks also passed.

CUDA/TensorRT and the complete tracking benchmark remain unsupported.