tinytimemixer

tinytimemixer at 10 layers and hidden size 128. One published checkpoint has this shape.

Implementing it

How to read this sheet: method.

No forward was recorded for this shape, so the block is read from the published configuration, the checkpoint's tensor names, and TinyTimeMixerForDecomposedPrediction in granite-tsfm v0.3.9 — not transcribed from a run.

Generated from granite-timeseries-ttm-r3.

# Two full mixer stacks, not one. The checkpoint is a decomposition: a
# robust LOWESS-like smoother splits the scaled context into a trend and
# a residual, one stack forecasts each, and the two are added back.
# 468 tensors — 217 trend, 251 residual. Not a clean halving, because the
# residual stack is wider, has twice the decoder depth, and carries an
# adapter the trend stack does not need.

scaled = StdScaler(last 512 samples)
trend  = robustLowess(scaled)             # Gaussian + Tukey bisquare,
                                          # 2 passes, reflect padding
resid  = scaled[-90:] - trend[-90:]

# The trend stack is fed the signal, not the trend. `trend_signal` exists
# only to build the residual; the trend forecaster reads the same scaled
# series the smoother read.
yt, qt = stack_trend(scaled)              # patch 64, d 8,  17 tokens
yr, qr = stack_resid(resid)               # patch 9,  d 128, 20 tokens

# Each stack patches at four resolutions. `MultiScaleFromPatchedSequence`
# average-pools by 1, 2, 4, 8 and projects each with its own Linear, then
# concatenates coarsest first (the list is built fine-to-coarse and
# reversed).
# Two learnable register tokens are prepended in front of all of them, so
# token 0 is not a patch of anything. (TSPulse appends its ten.)
# The LayerNorm after patching is over the whole flattened token grid,
# width n_patch * d_model, not per token.

point = (yt + yr) * scale + loc
band  = combineQuantiles(qt, qr) * scale + loc
#       ^ `combine_quantiles_via_variance: true`: medians add, and each
#         level's distance from the median combines in quadrature.

Sources, granite-tsfm v0.3.9: the decomposed forward (the trend forecaster is called on scaled_past_values, with past_values=trend_signal commented out beneath it); robust_lowess_like (Gaussian window of 15% of the context, two Tukey-bisquare passes, reflect padding); the residual context; MultiScaleFromPatchedSequence; the register tokens; the flattened base_norm, applied here; the decoder adapter; combine_quantiles_fast.

This is two models. One mixer stack over the trend and one over the residual, with different patch lengths (64, 9), widths (8, 128) and decoder depths (2, 4). The residual stack's encoder is wider than its 64-wide decoder, so only it carries a decoder.adapter.

The trend stack reads the signal, not the trend. The smoothed series exists only to build the residual.

The register tokens are prepended here and appended in TSPulse. Both load either way.

Each stack patches at four resolutions, concatenated coarsest first, so the obvious fine-to-coarse order is the wrong one.

Geometry

How this was checked

Status and evidence grade in the assessment (see what the grades mean):

• granite-timeseries-ttm-r3: runs, class — the architecture class is implemented; nothing checkpoint-specific was measured

The implementation meant here and below is an unpublished independent inference implementation by the author.

What stands behind the block above, beyond the published configuration it is read from:

• a checkpoint loader, which is where published tensor names are read
• an independent reference implementation of this architecture in Python, driving the published modelling code — an executable statement of what the model should compute, written against the publication rather than against any one implementation of it

Where an independent implementation and the published configuration disagree, the configuration is what this page reports and the disagreement is what it says.

Checkpoints with this architecture