MATH · IN · MODELS

SMI-TED embeddings obey a linear carbon-chain composition law

measured in 1 paper

- SMI-TED289M molecular embeddings satisfy a near-perfect linear-additive composition law: the embedding of a carbon chain grown by k carbons is recovered by a fixed linear combination of base-chain embeddings. [soares-etal-2024-large-encoder-decoder-foundation-models-chemical-language] - Fitting one linear regression per family across six carbon-chain families (CC, CO, CN, CS, CF, CP), chains up to 10 carbon atoms, gives held-out R-squared 0.99 (MSE 0.002) on 114 of 120 composition triples. [soares-etal-2024-large-encoder-decoder-foundation-models-chemical-language] - The same test on MoLFormer embeddings yields only R-squared 0.55, so the linear compositionality is far stronger in SMI-TED. [soares-etal-2024-large-encoder-decoder-foundation-models-chemical-language] - A narrow synthetic probe (one homologous carbon-appending operation); observational, no causal test. [soares-etal-2024-large-encoder-decoder-foundation-models-chemical-language]

Context

carbon-chain-family SMILES triples used to fit a linear composition law, held-out R^2 as a quantified generalization measure of linear compositionality, cross-model comparison (SMI-TED289M vs. MoLFormer) of the same linear test, t-SNE visualization showing hierarchical distance structure across chain lengths

Papers

A Large Encoder-Decoder Family of Foundation Models For Chemical Language — Soares, Eduardo, Shirasuna, Victor, Vital Brazil, Emilio, Cerqueira, Renato, Zubarev, Dmitry, Schmidt, Kristin2024 · arXiv:2407.20267