Matrix
The map's mathematical structures against its models — an overview plus four detailed cuts.
- Structures — overview: how many distinct mathematical structures have been observed at each domain × architecture-class (where geometry has been found, and where it's still blank). Hover a cell.
- By architecture / By domain — every structure (row) against model architecture classes / domains (columns); a dot marks where it's been observed, and its style says whether more than one independent group reported it. Click a dot for the observations.
- By family / By model — the same, resolved down to individual model families or models.
| transformer-decoder47 | transformer-encoder19 | cnn19 | diffusion13 | rnn15 | multimodal8 | vision-transformer15 | vae-autoencoder7 | transformer-enc-dec10 | dual-encoder12 | shallow-embedding12 | mlp7 | gnn7 | ssm-linear-attention8 | gan5 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| language49 | 44 | 16 | 8 | 8 | 2 | 3 | 7 | 12 | 11 | 7 | |||||
| vision29 | 1 | 19 | 12 | 8 | 14 | 3 | 12 | 1 | 4 | 5 | |||||
| audio10 | 2 | 8 | 1 | 2 | 1 | 2 | 3 | 4 | 3 | 1 | |||||
| control9 | 4 | 2 | 5 | 1 | 1 | 1 | |||||||||
| algorithmic14 | 7 | 6 | 1 | 3 | |||||||||||
| protein3 | 2 | 3 | 2 | 1 | |||||||||||
| molecular7 | 1 | 2 | 1 | 1 | 2 | 4 | |||||||||
| music3 | 2 | 1 | 2 | ||||||||||||
| board-game5 | 4 | 3 | |||||||||||||
| code3 | 2 | 1 | 1 | ||||||||||||
| graph4 | 1 | 4 | |||||||||||||
| tabular3 | 2 | 1 | 1 | ||||||||||||
| genomics6 | 5 | 1 | 1 | ||||||||||||
| time-series2 | 1 | 1 | 1 | 1 | |||||||||||
| action2 | 2 | 1 | |||||||||||||
| weather4 | 4 | 3 | |||||||||||||
| sign-language2 | 2 | 2 | 1 | ||||||||||||
| remote-sensing3 | 3 |
2+ papers, no shared author2+ papers, overlapping authorsone paperthe dot says how independently the cell is sourced — computed, not hand-assigned
What the axes mean — architecture classes & domains
Architecture classes (15)
- transformer-decoder
- Autoregressive (causal-attention) Transformer — the GPT-style stack behind most large language models.
- transformer-encoder
- Bidirectional (full-attention) Transformer trained with masked objectives; BERT-style representation models.
- transformer-enc-dec
- Encoder–decoder (“seq2seq”) Transformer with cross-attention: T5, the original Transformer, most translation models.
- vision-transformer
- Transformer applied to image patches (ViT and descendants) rather than 1-D token sequences.
- ssm-linear-attention
- State-space and linear-attention sequence models (Mamba, RWKV, S4) — sub-quadratic alternatives to softmax attention.
- rnn
- Recurrent networks (LSTM, GRU, vanilla RNN) that carry a hidden state through time.
- diffusion
- Iterative denoising generative models (DDPM, latent / score-based diffusion).
- gan
- Generative adversarial networks: a generator trained against a discriminator.
- vae-autoencoder
- (Variational) autoencoders and reconstruction models with a learned latent bottleneck.
- cnn
- Convolutional networks (ResNet, U-Net) built on local, weight-shared filters.
- gnn
- Graph neural networks that pass messages over nodes and edges.
- mlp
- Plain feed-forward / multilayer-perceptron networks — no sequence, conv, or attention structure.
- dual-encoder
- Two-tower contrastive models embedding two inputs into one shared space (CLIP, sentence-transformers).
- shallow-embedding
- Non-deep lookup embeddings (word2vec, GloVe, node2vec) — a single learned vector table, not a deep network.
- multimodal
- Models whose core architecture fuses several modalities (e.g. vision-language) and doesn’t reduce to one class above.
Domains (18)
- language
- Natural-language text.
- code
- Source code and programming languages.
- vision
- Images and video — natural-image understanding and generation.
- sign-language
- Signed languages captured as video or pose.
- music
- Musical audio and symbolic scores.
- audio
- Non-music audio: speech, sound, and general audio signals.
- protein
- Protein sequences and structures.
- molecular
- Small molecules and chemistry (SMILES, molecular graphs).
- genomics
- DNA / RNA sequences and single-cell gene expression.
- time-series
- Sequential numeric measurements — sensor, physiological, financial, forecasting.
- graph
- Relational / network-structured data.
- board-game
- Board and strategy games (chess, Go, Othello, poker).
- control
- Continuous control, navigation, and robotics / maze policies.
- algorithmic
- Synthetic reasoning and algorithmic tasks (copy, sort, arithmetic, evidence integration).
- tabular
- Structured rows-and-columns data and recommenders.
- weather
- Atmospheric state and weather forecasting (e.g. ERA5-trained foundation models).
- remote-sensing
- Satellite and Earth-observation imagery.
- action
- Embodied action — vision-language-action policies and agent trajectories.