Bounding the value a transform produces: a local ceiling, and a refusal

Status: decided and built, by a 3-panelist design panel. The control, its enforcement points, its constants, and the limits it does not close are specified in TRANSFORM_PARAMETER_BOUNDS.md (Unbounded transform-parameter rejection); this note records why the mechanism is the one it is. See docs/notes/README.md.

The linkage terms a partner authors include element transforms that this party runs over its own rows. The bounds on those terms cap what the partner may WRITE -- a param's content length, a pad_left length, a date format's length -- and a bound of that shape says nothing about what a row DERIVES from it: a replace_regex replacement is a substitution template whose match-context sequences re-insert the local cell at every match position, and steps compose, each fed the previous one's output. The spec states the measurements. This note records the two questions the choice turned on and how they were settled.

Which closer

Two were available.

• Neutralize the amplifying $-sequences in a partner-supplied replacement, so $' and $` no longer re-insert the match's context.
• Bound the produced value with a fixed ceiling, refusing an exchange that crosses it.

The second was adopted. The first changes what a replacement MEANS between the two parties: it is a wire-semantics delta, pulling in PROTOCOL_VERSION and the cross-implementation transform vectors, and it makes a legitimate (if unusual) authoring silently derive something else. The ceiling changes no wire semantics at all -- an in-range value is preserved verbatim, every sequence keeps its meaning -- and it is unilateral, which matters for a control whose whole purpose is that this party does not have to trust the terms it agreed to. It also generalizes: neutralizing one pair of sequences bounds one amplifier, where a ceiling on the output bounds every amplifier, including one a later standardizing function introduces.

The same reasoning that made the param content bound uniform across every string param -- a per-amplifier bound is dodged by routing content through a param no measurement covered -- is why the ceiling sits on the OUTPUT rather than on the replacement.

Where it sits, and what happens when it fires

One invariant, three enforcement points, all on the partner-authored path: inside the element transform (the value the element reads, and every step's output), on the row's projected assembled key-string length, and on the candidates as they accumulate toward that same total. Checking the element's incoming value is what bounds an element that declares no transform at all; checking every step's output is what keeps every step's input bounded, so composition cannot compound past the ceiling. The projection limb is the one that bounds the fuzzy expansion's replicated bytes, which is why no fuzzy-specific byte cap sits beside it.

A projection is reachable only once every candidate exists, so on its own it bounds the key strings a row assembles and not the set they are assembled from. A step that runs over an expanded candidate set allocates one output per candidate before any check reads the set, which is the whole per-row total over again with no factor bounding it. Charging each candidate to a running total as it arrives puts the same invariant where the allocation happens. What is left live when a total crosses is three charges rather than one -- the row's retained candidates across the elements already built, the set the crossing step is reading, and the set it is filling -- each held to the cap by a check of its own, plus the value in flight. The spec states the figure.

Two properties of that total are what keep it a bound on the row's live bytes rather than a second, tighter cap on the work reaching it. It charges what the candidate set RETAINS, because a duplicate the set collapses reaches no key string: a transform that maps every value of a multi-value cell onto one string is a legitimate authoring, and charging it per realized value would end an exchange over a row whose assembled key is a couple of thousand characters. And it is ONE total for the row rather than one per element, because the elements' candidate sets are all live together when the cross-product is projected; a total restarted at each element bounds the row at the cap times the element count instead, which is the same unbounded-until-assembly shape the projection has. The spec states both measurements.

The fate on exceedance is a refusal, never a clamp. Both parties must derive byte-identical keys, so a value shortened to fit would match nothing and would appear as a mismatch against the canonically hashed agreement -- the same reason every transform-parameter control refuses rather than trims.

Where a declared fan-out producer accounts for the row's multiplicity, though, the fate is the drop that producer's own width bound specifies: the transforms are the partner's while the values expanded are this party's rows, so a refusal would let a partner end an exchange by authoring a delimiter that shatters one local value.

The accumulating bound decides that per (row, key) BEFORE any of the key's elements runs, from the compiled steps of the elements and of the fields feeding them, and applies the one decision at both of its charges. What it reads is which producers CAN expand the key rather than which have been observed to: a raw cell is one string and a compiled step is fixed for the run, so every producer that could expand a row is knowable there, and the drop is bound to whether the key declares one rather than to the moment the crossing occurs. That buys a property the runtime alternative could not give: for one row and one key the fate is the same wherever the crossing falls and however the partner ordered the elements. A charge on the first element, before anything has fanned out, and one on the last resolve identically; so do a step's own expansion charge and the row's aggregate across the elements. The unit tests drive both fates through permuted orders and through crossings forced early and late.

A key any producer OUTSIDE the declared list can expand stays fail-closed on the refusal, whatever position that producer occupies and whether or not the row's crossing ever reaches it -- the pre-run reading is what makes that a property rather than a race with the element order. So does a key no producer can expand at all: there is no producer to bind a drop to, so the classification defaults to the refusal. That arm is the fail-closed default rather than a case rows arrive at -- with nothing expanding a value, what holds such a key is the per-value ceiling on each element it contains. Beside the classification, and only as a safety check, the run still records multiplicity realized by an unlisted step: a step that expands a value while unlisted under a drop classification is a producer the classification did not see as one, which is a fault in the classification rather than a fate the row earned, and it refuses. The assembled limbs, which run once every element exists, keep reading what the row actually realized.

The costs accepted

• The ceiling fires on the operator's own data, which the parameter controls never do. A legitimate long free-text cell ends the exchange, so the refusal has to be a good message: it names the element, the step position, the row, and the remedy, and it names none of the value.
• A partner can still abort a run deterministically on the paths that refuse -- the per-value ceiling, a crossing on a key no declared producer can expand, and a projection no declared producer expanded -- by tuning an amplifier to cross one of them, and learns one coarse bit about the local data's magnitude per consented exchange from whether it does. That trade was taken by design: an abort is visible and attributable to whoever authored the step, where an unannounced narrowing -- a value clamped to fit, a candidate set silently shrunk -- is neither. Where a declared fan-out producer accounts for the width the same tuning buys the warned drop instead, which is the width bound's own trade rather than a second one.
• Nothing here bounds the WORK a transform spends, only the bytes a row retains. The two are the same for an amplifier that keeps what it built, and far apart for one that expands toward the cap and then collapses its candidates onto one string: that row is accepted, charged almost nothing, and has cost the cap's work, per step and per element. Charging realized candidates instead is the alternative this note rejects above, so closing it means a budget on work rather than a different total; the spec has the measurement and states it as a limit.
• A single candidate's amplification is still allocated once before the output check refuses it. What bounds that transient is the ceiling on the step's input; the spec states the measurement and the point where the engine's own string limit takes over. What bounds the row is the accumulating total beside it, so the transient is one candidate's rather than one per candidate the row realized.

Considered and left alone

• A tighter ceiling. The constant is generous by design -- far above any legitimate key element -- because the cost of binding an honest exchange is paid by the operator, and the aggregate a partner can still drive is a bounded constant factor rather than an unbounded one.
• Bounding the operator's own standardization pipeline. Its config and its data both belong to the operator; nothing partner-influenced sizes it, so it stays out of scope.