Cross-modally correlated soft-robot VAE axes are individually manipulable primitives

measured in 1 paper

Han et al. train a multi-modal VAE encoding a soft robot's motion, force, and shape measurements into a shared 32-dimensional latent code [han-etal-2025-anchoring-morphological-representations-prosoro] Six components stand out with high cross-modal correlation, marking coordinate axes shared by all three modalities' encoders rather than modality-specific noise [han-etal-2025-anchoring-morphological-representations-prosoro] Traversing each of these six axes in isolation drives a distinct, geometrically coherent physical deformation mode, a causal intervention on an extracted direction [han-etal-2025-anchoring-morphological-representations-prosoro] Clustering the full latent-code trajectory recovers four separable clusters matching four ground-truth physical interaction phases [han-etal-2025-anchoring-morphological-representations-prosoro]

Structure

Linear Direction

Context

cross-modal correlation as a latent-axis selection criterion (vs. PCA or supervised probing), multi-modal VAE (motion, force, shape encoders fused into a shared 32-dim code), per-axis traversal as a geometry-tied causal intervention ("key morphing primitives"), k-means clustering of latent-code trajectories into physically-meaningful interaction phases, sim-to-real transfer of the learned latent proprioception

Confirmed in models

ProSoRo Multi-modal Variational Autoencoder (motion/force/shape latent proprioception) · ProSoRo (Proprioceptive Soft Robot)

Method

Causal interventions (steering)

Papers

Anchoring Morphological Representations Unlocks Latent Proprioception in Soft Robots — Han, Xudong, Guo, Ning, Xu, Ronghan, Wan, Fang, Song, Chaoyang2025