SwiGLU Activation Viz

The gated activation modern transformer FFN blocks use end-to-end: SwiGLU(x, g) = Swish(x) · g. The gate g is a learned scalar that scales the bare Swish curve; this view pins g to a single slider so the gating intuition is mechanical and direct. Drag the gate up and the SwiGLU curve grows; push it through zero and the curve flips across the x-axis.

Installation

npx shadcn@latest add https://craftbits.dev/r/swi-glu-activation-viz.json

Usage

import { SwiGLUActivationViz } from "@craft-bits/core";
 
<SwiGLUActivationViz defaultGate={1} />

Drive the gate from outside (parent scrubber, animation, or another control):

const [g, setG] = useState(1);
 
<SwiGLUActivationViz
  gate={g}
  onGateChange={setG}
  currentX={0.75}
/>

Add the sigmoid reference and hide the bare Swish overlay:

<SwiGLUActivationViz
  defaultGate={1.2}
  showSwish={false}
  showSigmoid
/>

Understanding the component

1. Three curves on one axis. Accent curve: SwiGLU(x, g) = Swish(x) · g. Muted dashed curve: the bare Swish(x) = x · σ(x) reference. Warning-toned horizontal line at y = g — the gate value made directly visible. The headline of the chart is the relationship between the three.
2. Gate range [-2, 2]. Production LLMs see gate values that cluster around 1 but routinely cross zero — that's where SwiGLU "decides not to fire". The slider exposes the full bipolar range so the user can watch the curve invert when g < 0.
3. Stable y-axis. The plot pre-samples SwiGLU at the worst-case gate (|g| = 2) so dragging the slider never re-pans the y extents. The Swish and σ overlays only widen the axis when enabled.
4. Spring transitions. Both the SwiGLU curve and the gate line animate with SPRINGS.smooth. Drags feel continuous; reduced-motion users snap.
5. Optional σ overlay. The sigmoid σ(x) = 1 / (1 + e^-x) is the smooth-gate primitive Swish wraps. Off by default — toggle it on to remind the learner where the s-shape originates.
6. Optional currentX highlight. Pass a number and the chart draws a dashed vertical guide plus a marker on the SwiGLU curve at that x. The live readout reports SwiGLU(currentX, g).
7. Controlled + uncontrolled. Pass gate + onGateChange to drive g from outside, or omit both and let the component own its state via defaultGate.

Props

Accessibility

• The chart is role="img" with an aria-labelledby heading that names the current gate value.
• The live readout uses aria-live="polite" so screen-reader users get the same value updates as sighted ones whenever the gate or currentX changes.
• The gate slider is a LabeledSlider (native <input type="range">) — full keyboard support out of the box (← / → step by 0.01, Page Up / Page Down step by 10%, Home / End jump to the extents).
• Animation is fully disabled when prefers-reduced-motion: reduce is set — both the curve morph and gate-line spring snap.

Credits

• Extracted from: craftingattention (app/src/lessons/primitives/nn/SwiGLUActivationViz.tsx). Stripped the lesson-specific phase / narration system, the parallel gate-lane / value-lane data-flow diagram, the SiLU inset plot, and the ReLU-vs-GELU comparison strip; generalized to a single function-plot primitive with a scrubbable gate, an optional currentX highlight, and Swish + σ reference overlays. Inline animate(..., SPRINGS.snappy) calls were replaced with declarative motion.path / motion.line transitions, and the project's --color-warn-400 / --color-accent-400 palette with cb-* semantic tokens.