Convolution via MATMUL

Real handwritten-digit images from MNIST, the exact .npz file TensorFlow/Keras itself hosts. linaldb has no autodiff or optimizer, so a trained neural network isn’t possible; a convolution + pooling forward pass, built entirely from MATMUL, is a real technique in its own right: the classic im2col trick.

Convolution as one MATMUL

Unroll every convolution window into a row, stack the rows into a matrix, and one matrix multiply against the flattened kernels computes every window’s output at once. Here’s the trick on a tiny 4x4 image with a 3x3 sum kernel (real linaldb output):

MATRIX im2col = [[1,2,3,5,6,7,9,10,11], [2,3,4,6,7,8,10,11,12],
                 [5,6,7,9,10,11,13,14,15], [6,7,8,10,11,12,14,15,16]]
MATRIX kernel = [[1],[1],[1],[1],[1],[1],[1],[1],[1]]
LET conv_out = MATMUL im2col kernel

Shape: [4, 1]
Data: [54.0, 63.0, 90.0, 99.0]

Each output is exactly the sum of its 3×3 window, verified by hand before trusting it at real scale. Pooling is also just a MATMUL: a fixed matrix with 0.25 in the 4 positions each 2×2 block averages, 0 elsewhere.

At real scale: three real CV kernels on real 28×28 digits

Sobel-X, Sobel-Y (standard edge-gradient detectors), and a Laplacian (a real, well-known “blob” detector), not trained weights, since nothing here can train anything:

MATRIX kernel_matrix = [[-1,-1,0], [0,-2,1], [1,-1,0], ...]   -- Sobel-X | Sobel-Y | Laplacian, one column each
MATRIX im2col_m = [...]                                        -- 676 rows (26x26 valid windows) x 9 cols
LET conv_out = MATMUL im2col_m kernel_matrix                   -- all 3 kernels, all 676 positions, one call
LET pooled = MATMUL pool_matrix conv_out                       -- 2x2 average pooling, also a MATMUL
LET feat = FLATTEN pooled

A real connector limitation, found along the way

linaldb’s native NumPy connector (USE DATASET FROM "file.npz" FIELDS (...)) should, in principle, ingest MNIST’s .npz directly; it doesn’t, because the file stores pixels as uint8 and the connector currently only accepts arrays already in f32/f64:

USE DATASET FROM failed: [line 1] Parse error: Connector failed: Parse error:
FIELDS: NPZ array 'x_test': not readable as an f32 or f64 array
(error reading npy file in npz archive: incorrect descriptor ('|u1') for this type)

Worked around with Python’s stdlib only (zipfile + struct + ast.literal_eval to parse the .npy header directly); no numpy needed even for that.

Honest result

Classifying real digits 4 vs. 9 (a classically confusable pair) by nearest centroid, the same idiom every other notebook uses:

Feature set	Accuracy
Raw pixels (784-dim)	62.5% (10/16)
Conv+pool features (507-dim)	62.5% (10/16)
Majority-class baseline	50.0%

Reported exactly as it came out: the fixed, untrained conv+pool features tied raw pixels on this specific hard pair, both beating the baseline. No cherry-picking: a fixed feature extractor isn’t guaranteed to beat raw pixels without the training a real CNN would get, and this is honest about that.

Try it yourself

• View the full notebook
• Try it in the Playground