methods / Theoretical / Analytical / RFM/AGOP Subspace Extraction (LLM activations)
RFM/AGOP Subspace Extraction (LLM activations)
Trains a Recursive Feature Machine (RFM) as a target-vs-rest classifier directly on transformer residual-stream activations, then takes the ranked top-k eigenvectors of the resulting Average Gradient Outer Product (AGOP) matrix as a multi-dimensional concept subspace — a cheaper kernel-machine alternative to iterative diff-in-means/PCA subspace search that yields an explicit ranking of how many dimensions a concept needs.