Variance Compound Viz

A bar chart of activation variance per layer for a deep feed-forward network. Each bar is Var(h_l) = inputVariance · varianceFactor^l — the textbook geometric-series approximation of how the second moment of activations evolves with depth under a constant per-layer multiplier. Drag the varianceFactor slider and watch the bars collapse into the vanishing regime or blow past the top into the exploding one. The component exists to make the case for Xavier and He initialisation visceral: when the per-layer factor isn't exactly 1, depth amplifies any mismatch exponentially.

Installation

npx shadcn@latest add https://craftbits.dev/r/variance-compound-viz.json

Usage

import { VarianceCompoundViz } from "@craft-bits/core";
 
<VarianceCompoundViz />

Pick a different depth and the input variance the initialiser is preserving:

<VarianceCompoundViz numLayers={32} inputVariance={1.0} />

Drive varianceFactor from outside (controlled mode):

const [factor, setFactor] = useState(1.0);
 
<VarianceCompoundViz
  varianceFactor={factor}
  onVarianceFactorChange={setFactor}
/>

Switch to a linear y-axis to dramatise the collapse or the blow-up:

<VarianceCompoundViz showLogScale={false} defaultVarianceFactor={0.7} />

Understanding the component

1. The formula on the bars. Each bar shows Var(h_l) = inputVariance · varianceFactor^l. Layer 0 (the input) is on the left; layer numLayers − 1 (the deepest layer) is on the right. The forward direction matches the way signal flows through the network, so the bars on the right tell you what arrives at the final layer.
2. Log scale is the default for a reason. At depth 16 with varianceFactor = 1.2, deep-layer variance reaches ~19; at 0.8 it falls to ~0.03. Linear axes clip one or the other; the log axis centres on inputVariance and walks 6 decades above and 6 below, so the dramatic case is always visible without losing the stable one.
3. Baseline is the input variance. The midline marks Var(h_0) — the variance the initialiser is trying to preserve. Bars above the midline mean variance has grown; bars below mean it has shrunk. At varianceFactor = 1.0 every bar sits exactly on the midline — the "stable" pose Xavier and He target.
4. Regime colors are derived, not state. When the per-layer factor exceeds 1.05 and the deepest bar is >10× the input variance, bars paint --cb-error (exploding). When the factor is below 0.95 and the deepest bar is <0.1× the input, bars paint --cb-warning (vanishing). Otherwise everything is --cb-accent (stable). The regime label under the slider gives the textual diagnosis — colour is never the only signal.
5. SPRINGS.smooth on the bar heights. Dragging the slider springs each bar's height and y independently — the animation reads as a wave that ripples across the chart. prefers-reduced-motion: reduce collapses every spring to instant.
6. The Xavier / He connection. For identity / tanh, Xavier sets weight variance 1 / fan_in so the per-layer variance multiplier is exactly 1. For ReLU, He picks 2 / fan_in to cancel the activation's 0.5 second-moment factor and again land at 1. Get the constant wrong and the chart says so in one glance — that's the entire teaching purpose.

Props

Accessibility

• The figure is role="figure" with an aria-labelledby that names the chart "Variance per layer."
• An aria-live="polite" summary announces the input variance, the per-layer factor, the final deep-layer variance, and the diagnosed regime (stable / vanishing / exploding) whenever any input changes.
• The slider is a LabeledSlider, itself a native <input type="range"> — keyboard arrows nudge the factor, Home / End jump to extremes, and screen readers narrate the value.
• Color is never the only signal — the regime label and the final-variance readout are both textual.
• prefers-reduced-motion: reduce snaps every bar transition; no per-step easing is shown.

Credits

• Extracted from: craftingattention (app/src/lessons/primitives/nn/VarianceCompoundViz.tsx). The source was a 6-layer fixed-depth widget driven by a σ (weight standard deviation) slider with D = 5 baked in, a four-phase narration state machine (observe / exploding / vanishing / balanced), a 1/√D critical-sigma marker on the slider track, imperative motion.animate() on each bar's height and label content, a breathing-pulse hint on the rightmost bar, a var=1 baseline reference line, and SvgLabel chrome. Reframed as a depth-agnostic primitive — varianceFactor directly is the per-layer multiplier (so consumers can express Xavier as 1.0, He as the activation-cancelling value, or any custom scheme) and inputVariance is the baseline Var(h_0). Stripped the phase / narration system, the 1/√D track marker, the SVG-glow filter, the imperative animation loop, and the breathing pulse. Replaced the raw <input type="range"> with LabeledSlider, switched bar colour and y-axis scaling to derived regime / log-around-baseline so the chart works at any inputVariance, and added Radix controlled / uncontrolled pairs for varianceFactor. Sits in ML Viz → Regularization alongside DropoutEnsembleViz, OverfittingGapViz, and RunningStatsViz.