Supervised Multi-Dimensional Scaling (SMDS)

methods / Dimensionality Reduction / Supervised Multi-Dimensional Scaling (SMDS)

Supervised Multi-Dimensional Scaling (SMDS)

Techniqueadvanced

Fits a linear projection whose Euclidean distances match a user-specified target distance function of a numeric label, turning manifold discovery into a model-selection problem across a menu of candidate geometries.

Used in (3 observations)

Geographic coordinates fit a spherical, not flat, manifoldmeasured

structure: Sphere · models: Qwen2.5-3B-Instruct, Llama-3.2-3B-Instruct, Gemma-2-2B-it · paper: Shape Happens: Automatic Feature Manifold Discovery in LLMs via Supervised Multi-Dimensional Scaling

Numbers lie on a logarithmically-compressed 1D number linemeasured

structure: 1D continuum manifold · models: Pythia-2.8B, Llama-2-7B, Llama-3.1-8B, Llama-3.2-1B, GPT-2-Large, Mistral-7B, Llama-3.1-8B-Instruct, Llama-3.2-1B-Instruct, Qwen2.5-3B-Instruct, Qwen2.5-3B, Llama-3.2-3B-Instruct, Llama-3.2-3B, Gemma-2-2B-it, Gemma-2-2B, Llama-3.1-70B-Instruct, Llama-3-8B-Instruct, Llama-3-8B, Mistral-7B-Instruct-v0.3, Qwen2.5-7B-Instruct · paper: Number Representations in LLMs: A Computational Parallel to Human Perception, Shape Happens: Automatic Feature Manifold Discovery in LLMs via Supervised Multi-Dimensional Scaling, Weber's Law in Transformer Magnitude Representations: Efficient Coding, Representational Geometry, and Psychophysical Laws in Language Models, LLMs Know More About Numbers than They Can Say

Cyclical time is represented as a circle, replicated and causally validatedmeasured

structure: Circle, Cone, Platonic Representation Hypothesis · models: GPT-2-small, Mistral-7B, Llama-3-8B, Gemma-2-2B, EmbeddingGemma, word2vec (trained on Wikipedia), Qwen2.5-3B-Instruct, Qwen2.5-3B, Llama-3.2-3B-Instruct, Llama-3.2-3B, Gemma-2-2B-it, Llama-3.1-8B-Instruct, Llama-3.1-70B-Instruct, Llama-3.1-8B · paper: Symmetry in Language Statistics Shapes the Geometry of Model Representations, Not All Language Model Features Are One-Dimensionally Linear, Shape Happens: Automatic Feature Manifold Discovery in LLMs via Supervised Multi-Dimensional Scaling, Do Sparse Autoencoders Capture Concept Manifolds?