RoBERTa
Meta AI
Structures found in this family (7)
By model (7)
RoBERTa-base · 125M
RoBERTa-large · 355M
RoBERTa-large-MNLI · 355M
RoBERTa-large-MNLI + double fine-tuning (HELP + NLI_XY) · 355M
RoBERTa-large-MNLI + HELP fine-tuning · 355M
RoBERTa-base (fine-tuned on EmoWOZ, 7-class emotion recognition) · 125M
RoBERTa-base (TripPy-R dialogue-state tracker, fine-tuned on MultiWOZ 2.1) · 125M
Observations (17)
Papers
Interventional Probing in High Dimensions: An NLI Case Study (2023), AST-Probe: Recovering Abstract Syntax Trees from Hidden Representations of Pre-trained Language Models (2022), Intervention Lens: from Representation Surgery to String Counterfactuals (2024), DirectProbe: Studying Representations without Classifiers (2021), Emergence of Separable Manifolds in Deep Language Representations (2020), Discovering Latent Knowledge in Language Models Without Supervision (2023), Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color (2021), Relative Representations Enable Zero-Shot Latent Space Communication (2022), OSCaR: Orthogonal Subspace Correction and Rectification of Biases in Word Embeddings (2020), Outlier Dimensions Encode Task-Specific Knowledge (2023), The Shape of Learning: Anisotropy and Intrinsic Dimensions in Transformer-Based Models (2024), All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Quality (2021), Less is More: Local Intrinsic Dimensions of Contextual Language Models (2025), Anisotropy Decides Cosine vs. Rank Metrics for Text Embeddings (2026), Lost in State Space: Probing Frozen Mamba Representations (2026), WhiteningBERT: An Easy Unsupervised Sentence Embedding Approach (2021)