MATH · IN · MODELS

EHR Foundation Model (Llama-3.2-1B architecture, trained on MIMIC-IV)

University of Chicago / Rush University (Burkhart, Ramadan, Liao, Chhikara, Rojas, Parker & Beaulieu-Jones)

Structures found in this family (1)

1D continuum manifold

By model (1)

Custom Llama-3.2-1B-architecture transformer (trained from scratch on tokenized MIMIC-IV EHR sequences) · 1B
1D continuum manifold

Observations (1)

A real EHR foundation model trained from scratch on MIMIC-IV recovers a correctly-ranked ordinal 1D continuum of decile embeddings via PCA

Papers

Foundation Models for Electronic Health Records: Representation Dynamics and Transferability (2025)