craftbits*

useShikiTokens

A React hook that tokenizes a code string with shiki and returns structured per-line tokens (ThemedToken[][]) with resolved colors. Reach for it when you need full control over how each token renders — wrap each <span> in a motion component, animate token entry, attach hover handlers, drive a typewriter-style reveal.

CodeBlock produces HTML strings (faster, opaque). useShikiTokens returns structured data (slower, transparent). Use the hook only when you actually need the raw tokens.

function add(a, b) {
  return a + b;
}(loading…)
Customize
Snippet
1 / 3
Language

Installation

npx shadcn@latest add https://craftbits.dev/r/use-shiki-tokens.json

shiki is an optional peer dependency. Install it alongside the hook only if you intend to use it:

npm install shiki

Usage

import { useShikiTokens } from "@craft-bits/core";
 
function MyHighlightedSnippet({ code, lang }: { code: string; lang: string }) {
  const { tokens, loading, error } = useShikiTokens(code, lang);
 
  if (error || !tokens) {
    return <pre>{code}{loading ? " (loading…)" : ""}</pre>;
  }
 
  return (
    <pre>
      {tokens.map((line, lineIdx) => (
        <div key={lineIdx}>
          {line.map((token, tokenIdx) => (
            <span key={tokenIdx} style={{ color: token.color }}>
              {token.content}
            </span>
          ))}
        </div>
      ))}
    </pre>
  );
}

API

Parameters

ParameterTypeDefaultDescription
codestringrequiredThe source code to tokenize. Empty strings short-circuit to tokens: null, loading: false.
langstringrequiredShiki language id ("ts", "tsx", "py", etc.). Cast at the shiki boundary so unknown ids surface as error instead of a type error.
options.themeBundledTheme | ThemeRegistration | stringbundled neutral themeShiki theme. Defaults to the same neutral theme CodeBlock and CodeTrace ship with so consumers blend visually with those components.

Return value

UseShikiTokensResult:

FieldTypeDescription
tokensThemedToken[][] | nullPer-line arrays of shiki tokens with resolved colors. null while loading, on error, or when code is empty.
loadingbooleantrue while shiki is loading or re-tokenizing.
errorError | nullNon-null when shiki failed to load (peer dep missing) or tokenize. Fall back to a plain-text render in this case.

Behaviour

  1. Dynamic import. Shiki is loaded with await import("shiki") inside the effect — the hook only pays the cost when actually mounted, and the dependency stays tree-shakeable for consumers that don't use it.
  2. Re-tokenizes on code / lang / theme change. A cancellation flag inside the effect guards against late resolves after unmount or input change.
  3. Graceful failure. If shiki isn't installed, the dynamic import rejects and error surfaces the failure. The hook never throws into render — callers handle the plain-text fallback.

Examples

Animate each token on mount

import { motion } from "motion/react";
import { useShikiTokens } from "@craft-bits/core";
 
function AnimatedSnippet({ code, lang }: { code: string; lang: string }) {
  const { tokens } = useShikiTokens(code, lang);
  if (!tokens) return <pre>{code}</pre>;
 
  return (
    <pre>
      {tokens.map((line, li) => (
        <div key={li}>
          {line.map((token, ti) => (
            <motion.span
              key={ti}
              initial={{ opacity: 0, y: 2 }}
              animate={{ opacity: 1, y: 0 }}
              transition={{ delay: (li * line.length + ti) * 0.01 }}
              style={{ color: token.color }}
            >
              {token.content}
            </motion.span>
          ))}
        </div>
      ))}
    </pre>
  );
}

Custom theme

import { useShikiTokens } from "@craft-bits/core";
 
const { tokens } = useShikiTokens(code, "ts", { theme: "github-dark" });

Credits

  • Extracted from: terminal-dreams (src/components/recipe-lab/code-primitive-1/use-shiki-tokens.tsx).
  • Pairs with CodeBlock (HTML output) and CodeTrace (per-line animation) — all three share the same neutral theme JSON.