Build with Chimera.

Get started

1. Add the package

npm install chimera-kit

Use a browser app with ES modules, such as Vite or Next.js. In Next.js, keep voice creation and playback in client code. Load the voice once, then start playback from a click or tap.

2. Speak on a click

<button id="speak">Speak</button>

import { createVoice, play } from "chimera-kit";

const voice = await createVoice({ body: "female" });

document.querySelector("#speak")?.addEventListener("click", () => {
  const playback = play(voice.stream("Hello there."), voice.sampleRate);
  // playback.stop() interrupts immediately.
  playback.done.then(() => console.log("Finished"));
});

play() returns a playback handle with stop() and done. Reuse the voice for later sentences. Call voice.free() when your app no longer needs it, after playback finishes.

3. Choose how much control you need

Use voice.stream(text) for audio chunks, voice.render(text) for a complete { audio, sampleRate } result, or voice.plan(text) for an editable score. The engine sounds synthetic and currently speaks English.

Voices & direction

Start with male, female or child. A body can also be a blend with physical and source settings. A seed gives each character the same voice every time.

import { bodyFromSeed, createVoice, deliver, play, take } from "chimera-kit";

const voice = await createVoice({
  body: bodyFromSeed("guide"),
  style: deliver("calm"),
});

const reading = take(voice.plan("I never said she stole my money."));
reading.word("never").emphasize();
reading.word("money").pitch("+2st").stretch(1.3);

play(voice.stream(reading.score), voice.sampleRate);

Text accepts rate, baseline pitch f0 in Hz, and stress excursion accent. Scores and songs already contain timing and pitch, so those three options are refused. Use score transforms such as stretch, shift and deliver instead.

Streaming answers

Pass an async iterable of text tokens to speech(). It speaks clause by clause, skips Markdown code blocks and reads links as their text. The following example reuses a loaded voice; tokens is your application's token stream.

import { play, speech } from "chimera-kit";

const playback = play(speech(voice, tokens), voice.sampleRate);
// When the user interrupts:
playback.stop();
await playback.done;

For a busy interface, use a worker voice and call voice.say(tokens). It sends samples directly from the worker to the audio thread. Call playback from a user gesture, or unlock your audio context before awaiting a network response.

Singing

A song pairs one note with each syllable. Hyphens split syllables inside a word. A standalone ~ or - holds the previous vowel through another note.

import { createVoice, play } from "chimera-kit";
import { harmony, melody } from "chimera-kit/singing";

const voice = await createVoice();
const song = {
  lyrics: "Hel-lo there",
  notes: melody("C4 D4 E4:2"),
  bpm: 110,
  vibrato: 30, // cents on held vowels
};
const playback = play(voice.stream(song), voice.sampleRate);
const upperPart = harmony(song, "third-above");
// Give upperPart its own voice for simultaneous harmony.
// Call playback.stop() to end the song.

Notes use pitch and optional duration in beats: C4:1 D#4:0.5 Bb3:2. MIDI numbers are accepted; | is ignored. Silent rests are currently refused. scoop and fall add pitch slides to a score.

Agent tools

Expose say, sing and set_voice to a language model. The runner validates arguments, speaks concurrent calls in order, and returns readable success or error data.

import { createWorkerVoice } from "chimera-kit";
import { createToolRunner, forAnthropic, forOpenAI } from "chimera-kit/tools";

const voice = await createWorkerVoice();
const runner = createToolRunner(voice, { body: "female" });

const result = await runner.run("say", { text: "Hello there." });
// { ok: true }, or { ok: false, error: "..." }

const anthropicTools = forAnthropic();
const chatCompletionsTools = forOpenAI();
// Pass the appropriate definitions to your model provider.
// Send each returned tool name and arguments to runner.run().

runner.stop(); // Interrupt audio and discard queued calls.
voice.free();

The adapters provide tool definitions, not model API clients. forOpenAI() returns the Chat Completions tool format. TOOLS contains provider-neutral JSON Schema definitions. Calls run against the Chimera voice and return when playback finishes.

Worker playback

A worker keeps Chimera synthesis off the main thread. Its render and stream methods are asynchronous; say returns a playback handle immediately.

import { createWorkerVoice } from "chimera-kit";

const voice = await createWorkerVoice({ body: "female", delivery: "calm" });
const playback = voice.say("Take a breath.");
await playback.done;
voice.free();

Workers take delivery and intensity as data instead of a style function. They accept text, songs and edited scores, with samples sent directly to the audio thread. Call playback.stop() to interrupt and voice.free() on teardown.

Live instrument & beatbox

The instrument runs in an AudioWorklet. Load it once, then connect these controls to your app's note, pointer or MIDI events. Trigger audio from a user gesture.

import { createInstrument } from "chimera-kit/live";

const live = await createInstrument({ body: "female" });

live.vowel("a");
live.noteOn(60); // Your app's note-on event.
live.shape({ jaw: 0.4, lipAp: 0.2 });
live.source({ breathiness: 0.3, tension: 0.6 });
live.bend(62);
live.noteOff(); // Your app's note-off event.

live.beat.steps("kick . snare .");
live.beat.start({ bpm: 96 });
live.beat.hit("kick");
live.beat.stop();
live.free(); // Release the worklet when the instrument closes.

Pass onError to receive controls rejected by the engine. live.telemetry() returns the worklet's most recent snapshot. Play the instrument →

Creature calls

Chimera also models a songbird, seal, alligator and elephant. Creatures load their own small WASM artifact, independently of the human voice.

import { loadCreatures } from "chimera-kit/creatures";
import { play } from "chimera-kit";

const zoo = await loadCreatures();
const bird = zoo.create("songbird");
// Run playback from a click or an already-unlocked audio context.
await play([bird.call("flute")], zoo.sampleRate).done;
bird.free();

zoo.list describes the available species and calls. Use call for a complete clip, stream for chunks, or set, gate and render for live control. Hear the creatures →

Anatomy & animation

The main-thread voice exposes the actual airway while it renders. Read cross-sectional areas, pressures and a versioned suffix containing retained articulation and geometry.

const chunks = voice.stream("Hello there.", {
  airway(state) {
    const sections = state.subarray(3, 6).reduce((sum, n) => sum + n, 0);
    const areas = state.subarray(6, 6 + sections);
    const pressures = state.subarray(6 + sections, 6 + 2 * sections);
    console.log({ areas, pressures }); // Feed your own visualizer.
  },
});
play(chunks, voice.sampleRate);

The callback follows generation, which can run ahead of playback. For lip sync, timestamp and buffer frames by sample position, then select them with the audio clock. Keep retention bounded. Watch the real-time Chimera profile →

Voice profiles

chimera-kit/imitate measures pitch, rate and a coarse body profile from mono audio: profile(samples, sampleRate, { speakerAgreed: true }). It requires the speaker's agreement and produces a caricature, with no neural voice cloning.

API reference

Each entry point ships TypeScript declarations. Import the Chimera features your app needs; creatures load their own small WASM artifact.

Voice & playback

createVoice(options?) → Promise<Voice>

Loads the engine once per voice. Options include body, rate, f0, accent, style, wasm and dictionary.

voice.plan(text | song) → Score

Returns sounds, timing, pitch curves and word spans for inspection or editing.

voice.render(text | song | score, options?)

Returns { audio: Float32Array, sampleRate: number }.

voice.stream(text | song | score, options?)

Yields audio chunks. The main-thread voice also accepts an airway(state) callback for physical telemetry.

play(chunks, sampleRate, options?) → Playback

Accepts synchronous or asynchronous chunks. Supports an AudioContext and AbortSignal. Use stop() to interrupt and done to await completion.

voice.free()

Releases the engine. Stop and finish playback before freeing a voice that is still streaming.

Bundled resources

The package exports chimera-kit/worker.js, chimera-kit/worklet-live.js, chimera-kit/chimera.wasm, chimera-kit/creatures.wasm and chimera-kit/cmudict.txt.gz. They ship with the library; there is no synthesis service or API key.

Troubleshooting

No sound after an await

Browsers require a user gesture. Call play() directly inside a click or tap, or call unlockAudio(context) at the start of that handler before awaiting work, then pass that context to playback.

WASM, dictionary or worker returns 404

Assets resolve next to the library with new URL("./file", import.meta.url). Vite and Next.js copy them into the build. Bun's bundler does not: serve the files yourself and pass explicit wasm and dictionary URLs, plus a module Worker instance as worker. Set dictionary: false to use spelling rules alone, with reduced pronunciation accuracy. The live instrument additionally accepts a worklet URL.

Webpack tries to bundle Node's file loader

The file: resource loader is used only by Node. In a Webpack browser bundle, ignore that exact builtin import. For Next.js, add this to your configuration; server-side loading stays unchanged.

export default {
  webpack(config, { isServer, webpack }) {
    if (!isServer) {
      config.plugins.push(new webpack.IgnorePlugin({
        resourceRegExp: new RegExp("^node:fs/promises$"),
      }));
    }
    return config;
  },
};

Next.js says window or AudioContext is undefined

Create voices in a client-side effect or event handler. The documentation's module example assumes a browser entry point. Release voices in your component's cleanup.

A score rejects timing options

Songs and scores already specify pitch and timing. Use score transforms to edit them instead of passing rate, f0 or accent. Chimera currently speaks English, sounds synthetic, and phone latency has not yet been measured.