Leukemia Subtype Classification

The classic Golub et al. (1999) leukemia microarray dataset: 7,129 genes measured across 72 patients (38 training, 34 independent test), each labeled ALL (Acute Lymphoblastic Leukemia) or AML (Acute Myeloid Leukemia). Every computation below (marker-gene discovery, centroids, classification, accuracy scoring) runs as linaldb DSL. Python only reads the raw CSVs; matplotlib only renders results linaldb already computed.

Marker-gene discovery via CORRELATE

For each of the 7,129 genes, correlate its expression profile directly against the ALL/AML label signal, reproducing Golub et al.’s own “neighborhood analysis” marker-selection technique natively:

VECTOR label_signal = [1.0, -1.0, 1.0, ...]  -- one entry per training patient
VECTOR gene_vec = [...]                       -- one gene's expression profile
LET gene_corr = CORRELATE gene_vec WITH label_signal

Run across all 7,129 genes (21,387 DSL calls) in 0.33 seconds. Top markers found:

Gene	Correlation	Description
U50136_rna1_at	+0.828	Leukotriene C4 synthase (AML-associated)
X95735_at	+0.822	Zyxin (AML-associated)
U22376_cds2_s_at	−0.653	C-myb (ALL-associated)
M86406_at	−0.638	Actinin alpha 2 (ALL-associated)

Nearest-centroid classification

Per-class centroids via AVG_VEC + GROUP BY, a cosine-similarity JOIN against the held-out test set, and ROW_NUMBER() to keep each patient’s best match, the same idiom this project’s own digit-classification example uses, applied to a real 2-class biological problem:

WITH centroids AS (
    SELECT label, AVG_VEC(expression) AS centroid FROM train_patients GROUP BY label
)
SELECT t.patient_id, t.label AS true_label, c.label AS predicted_label,
       COSINE_SIM(t.expression, c.centroid) AS similarity,
       ROW_NUMBER() OVER (PARTITION BY t.patient_id ORDER BY similarity DESC) AS rn
FROM test_patients t
JOIN centroids c ON COSINE_SIM(t.expression, c.centroid) > 0.0

A window function’s ORDER BY only accepts a bare column or alias, not a function call directly: ORDER BY similarity (the COSINE_SIM(...) AS similarity alias defined earlier in the same SELECT), not ORDER BY COSINE_SIM(...) again. Filter to WHERE rn = 1 for the final one-row-per-patient prediction.

Real results, both directions honestly reported

Feature set	Accuracy
Full 7,129-dim expression	94.1% (32/34)
50 marker genes only	91.2% (31/34)
Majority-class baseline	58.8%

The reduced 50-gene feature set gets within 3 points of the full 7,129-dim classifier: almost all the signal lives in a small, interpretable marker set.

Two real engine bugs, found here

Building this notebook surfaced two real, reproducible gaps between the documented DSL and the engine’s actual behavior: CORRELATE was computing a raw, unnormalized dot product instead of true Pearson correlation, and SUM/MEAN/STDEV returned the wrong tensor shape internally, silently corrupting vector - MEAN(vector) past the first element. Both fixed and shipped in engine v0.1.79 and PyPI linaldb 0.1.2, the version this notebook verifies against directly with a hand-computed ground-truth sanity check before trusting either at scale.

Try it yourself

• View the full notebook
• Try it in the Playground