LoRA Factorization Viz

A teaching visualisation of Low-Rank Adaptation. Three matrix blocks sit side-by-side: the full d × d weight-update ΔW on the left, then =, then the factorization B (d × r) · A (r × d) on the right. Drag the rank slider and watch B and A shrink — the parameter-count readout shows the same story numerically: d² parameters collapse to 2 d r.

Installation

npx shadcn@latest add https://craftbits.dev/r/lora-factorization-viz.json

Usage

import { LoRaFactorizationViz } from "@craft-bits/core";
 
<LoRaFactorizationViz d={64} defaultR={8} />

Drive the rank from outside the component:

const [r, setR] = useState(8);
 
<LoRaFactorizationViz d={64} r={r} onRChange={setR} />

Hide the full ΔW block to focus on the factorization alone:

<LoRaFactorizationViz d={64} defaultR={4} showFullMatrix={false} />

Drop the parameter-count readout for a leaner figure:

<LoRaFactorizationViz d={64} defaultR={8} showParameterCount={false} />

Understanding the component

1. Three blocks, one equation. Left is ΔW (d × d) — the full weight update, drawn as a speckled muted matrix so it reads as "every one of d² entries is in play." Right is the factorization: B (d × r) accent-tinted in the top half, then a multiplication dot, then A (r × d) accent-muted underneath. An = between them makes the identity explicit.
2. Thin dimension animates with r. The width of B and the height of A are both bound to the rank — slide r from 1 up to ⌊d / 2⌋ and the two thin matrices fill out smoothly. The full ΔW block stays the same size, so the gap between "how much we'd train without LoRA" and "how much we actually train" is visible at a glance.
3. d² → 2 d r parameter math. Below the diagram, three stats render side-by-side: the full count d², the LoRA count 2 d r, and the savings as a percentage. With d = 64 and r = 8 that's 4,096 → 1,024 — a 75% reduction. With d = 4,096 and r = 8 it's 16.8M → 65K — a 99.6% reduction.
4. Slider clamps to ⌊d / 2⌋. LoRA is only interesting when r ≪ d. Past d / 2 the factorization stops being a compression — the slider stops there so the visualisation never lies.
5. Controlled + uncontrolled. Pass r + onRChange to drive the rank from a parent; omit them and the component owns its own state via defaultR.
6. Reduced motion. When prefers-reduced-motion: reduce is set, the spring on B and A's thin dimension collapses to duration: 0 — the matrices snap to their new size instead of springing.

Props

Accessibility

• The root is role="figure" with aria-label describing the current dimensions and parameter counts.
• A visually-hidden aria-live="polite" summary mirrors the same line so screen readers re-announce as the rank slider moves.
• The rank slider is a native <input type="range"> via LabeledSlider — keyboard arrows, screen-reader value announcements, and :focus-visible rings come for free.
• Color is never the only signal — every matrix carries a text label (ΔW, B, A) plus a dimension annotation (d × d, d × r, r × d), and the parameter-count panel renders the numbers in font-variant-numeric: tabular-nums.
• prefers-reduced-motion: reduce collapses every spring to an instant swap.

Credits

• Extracted from: craftingattention (app/src/lessons/primitives/nn/LoRaFactorizationViz.tsx). The source was a lesson-mode widget with a phase system (observe → compressing → insight), narration strings, a breathing pulse, and a hard-coded D = 4096 model dimension. The library extract is the pure factorization primitive — three matrix blocks, a rank slider, and a parameter-count readout. The phase / narration scaffolding lives in the consuming lesson, not the primitive.