MATH · IN · MODELS

Phi

Microsoft

Structures found in this family (8)

Linear Direction, Attention-graph spectral profile (Fiedler value, HFER, smoothness, spectral entropy), Linear Subspace, Lissajous Curves, Decision boundary (as a codimension-1 hypersurface), Laguerre-Voronoi partition (weighted power diagram of a linear readout layer), Continuous causal attention operator (CCT), Attention reference frame (sink-token anchor configuration)

By model (8)

Phi-2 · 2.7B
Linear Subspace, Laguerre-Voronoi partition (weighted power diagram of a linear readout layer), Attention reference frame (sink-token anchor configuration)
Phi-4 (15B) · 15B
Lissajous Curves, Linear Subspace, Linear Direction
Phi-3.5-mini-instruct · 3.8B
Linear Direction, Decision boundary (as a codimension-1 hypersurface)
Phi-1.5 · 1.3B
Linear Subspace
Phi-3-Medium-4k-Instruct · 14B
Continuous causal attention operator (CCT)
Phi-3-mini · 3.8B
Linear Direction
Phi-3.5-mini · 3.8B
Attention-graph spectral profile (Fiedler value, HFER, smoothness, spectral entropy)
Phi-4-Mini-Instruct · 3.8B
Linear Direction

Observations (14)

A PCA gender direction carries first-name discriminative signal
Attention-graph Laplacian spectra separate valid from invalid proofs
A trained rank-1 steering module rebalances multimodal attention
Sinusoidal number geometry recurs across eight LLMs and is causal
LLM hidden states warp categorically at digit-count boundaries
A linear unembedding layer's weights analytically induce a Laguerre-Voronoi cell partition of representation space, and cell-based concept boundaries outperform point-based baselines
A single SAE latent causally mediates refusal with a dose-response tradeoff
Factual recall is linear superposition with a proven dimension bound
A continuous-time attention operator makes fixed LLMs behave continuously
A sinusoidal probe decodes integers near-perfectly across nine LLMs
Per-task truth probe directions are near-orthogonal with disjoint L1 support
Attention sinks form three position-encoding-linked reference-frame configurations
A diff-of-means shortcut direction tracks and causally mitigates rebounding reward hacking during real GRPO training
Scale determines whether late-layer geometry stays organized by the readout

Papers

On the Mutual Influence of Gender and Occupation in LLM Representations (2025), Geometry of Reason: Spectral Signatures of Valid Mathematical Reasoning (2026), LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steering (2024), Unravelling the Mechanisms of Manipulating Numbers in Language Models (2025), Categorical Perception in Large Language Model Hidden States: Structural Warping at Digit-Count Boundaries (2026), Laguerre Geometry for Interpreting Large Language Models (2026), Steering Language Model Refusal with Sparse Autoencoder Features (2024), Geometric Factual Recall in Transformers (2026), Language Models Are Implicitly Continuous (2025), Pre-trained Language Models Learn Remarkably Accurate Representations of Numbers (2025), The Geometries of Truth Are Orthogonal Across Tasks (2026), What are you sinking? A geometric approach on attention sink (2025), When Reward Hacking Rebounds: Understanding and Mitigating It with Representation-Level Signals (2026), Scale Determines Whether Language Models Organize Representation Geometry for Prediction (2026)