PatchTSTFMForPrediction

PatchTSTFMForPrediction at 20 layers and hidden size 1024. One published checkpoint has this shape.

Implementing it

How to read this sheet: method.

No forward was recorded for this shape, so the block is read from the published configuration and from IBM's modelling code in granite-tsfm v0.3.9, not transcribed from a run. Keys the configuration omits take the code defaults: block_type: "transformer", qkv_bias: true, prehead_norm: false, no patch_stride.

Generated from granite-timeseries-patchtst-fm-r1.

# There is no separate horizon: the forecast occupies the tail of the
# fixed 8,192-point window, marked by a mask the model reads.

fl = max(64, ul, 16 * max(pretrain_mask_cont, 2))   # = 128; ul = -1 in this path,
                                                    #   pretrain_mask_cont: 8
x  = concat(last 8064 observed points,          # 8192 - 128
            [nanmean(series)] * 128)            # placeholder tail
mask = [0]*8064 + [1]*128                       # 128 masked, not 64
# (a shorter history is left-padded with the mean and marked as padding)

# Scaler: statistics over the unmasked points only, divisor = their count.
mean, std = stats(x, where=~mask, divisor=count(~mask))
std = std if std > 1e-5 else 1                  # replaced by 1, not epsilon'd
z = asinh((x - mean) / std)
z = where(mask, 0, z)                           # masked points read zero

p = reshape(z, [512, 16])                       # non-overlapping patches
m = reshape(mask, [512, 16])
h = residual_block(concat(p, ~m))               # 32 -> 1024
h = h + pos_embed[0 .. 511]                     # learned, additive

for i in 0 .. 19:                               # pre-norm transformer
    h = h + attn(layernorm(h))                  # 16 heads, qkv with bias
    h = h + mlp(layernorm(h))                   # 1024 -> 4096 -> 1024,
                                                #   tanh-approximate GELU
# LayerNorm eps 1e-6. No norm before the head (`prehead_norm` unset).

# residual_block(x) = layer2(sigmoid(layer1(x))) + residual(x)
rows = residual_block(h)                        # 1024 -> 16 * 100 per patch
q = rows[0] + cumsum(softplus(rows[1:100]) / 99)    # 99 levels, cannot cross

out = reshape(q, [8192, 99])                    # patches laid end to end
out = sinh(out) * std + mean                    # every level, then ...
out = out[-128:][:64]                           # ... the first 64 of the tail
forecast = trapz(out, levels) + out[0]*0.01 + out[98]*(1 - 0.99)

Sources for the block, all at granite-tsfm v0.3.9: the 128-point window and the appended placeholder tail are PatchTSTFMForPrediction.forward (with ul set at L570) and forecast_single_step_fast; the masked statistics are RevIN._get_statistics; patching, the mask channel and the positional embedding are forward and decode; the block is TransformerBlock.forward with its MLP; the head is decode with ResidualBlock; with no patch_stride the overlap-add is skipped; the point forecast is the trapezoid.

The forecast sits inside the context, not after it. The wrapper keeps the last 8,064 observed points, appends 128 placeholder positions set to the series mean, and marks them masked; the backbone always sees exactly 8,192 points. The window is max(prediction_length, ul, d_patch * max(pretrain_mask_cont, 2)) = 128, where ul is fixed at -1 on this path, and the caller keeps the first 64 of it. Masking only 64 would change both the scaler's statistics and what the model is asked to fill.

The scaler is not affine. asinh on the way in and sinh on the way out; a linear inverse is wrong by a factor that grows with the value. A std at or below 1e-5 is replaced by 1 rather than having an epsilon added, so a flat series gets scale 1.

The masked statistics divide by the unmasked count. With a mask, RevIN computes the mean and the variance over the unmasked points and divides by their number; the no-mask branch calls x.std, which is torch's N-1 default. The prediction path always passes a mask. At 8,064 unmasked points the two divisors differ by about 0.012% in variance and 0.006% in std — small, but systematic.

The head emits 100 rows and 99 levels. Row 0 is a base, not a quantile; each of rows 1..99 adds softplus(...)/99 to a running total. The median is level 0.50, index 49 of the 99 levels; reading the 100 rows as levels puts it one off.

sinh before the integration, not after it. The inverse scaler is applied to every level and the trapezoid integrates the denormalised levels. Because sinh is not linear, sinh(integrate(...)) is a different number, and the gap grows with the value.

No fold in this revision. Without patch_stride the patches do not overlap and are laid end to end; the Hamming overlap-add of r2 is not called, so a port validated only against this revision has not exercised it.

Geometry

Other shapes of PatchTSTFMForPrediction

Identical across all of them: width 1,024, heads 16.

Only the columns that differ are shown; a cell with several values means the checkpoints of that shape disagree.

How this was checked

Status and evidence grade in the assessment (see what the grades mean):

• granite-timeseries-patchtst-fm-r1: runs, class — the architecture class is implemented; nothing checkpoint-specific was measured

The implementation meant here and below is an unpublished independent inference implementation by the author.

What stands behind the block above, beyond the published configuration it is read from:

• a checkpoint loader, which is where published tensor names are read
• an independent reference implementation of this architecture in Python, driving the published modelling code — an executable statement of what the model should compute, written against the publication rather than against any one implementation of it

Where an independent implementation and the published configuration disagree, the configuration is what this page reports and the disagreement is what it says.

Checkpoints with this architecture