Skip to content

Sound and music ​

Sound in Molen is data. A sound bank names the clips, three components say when and where they play, and scripts can fire one-shots with molen.audio.play. The client's audio director reads all of it every frame and plays the result through Web Audio. Nothing about playback touches the simulation, so sound never changes a state hash, and molen audio plan shows what a scene would play without opening a browser.

The pieces ​

PieceWhere it livesWhat it does
molen/soundbank@1a content pack (provides.soundbank) or a file beside the sceneSound ids → clip files, loop points, gain, pitch range, bus, spatial defaults and provenance
audioEnvironmentone entity in the sceneAmbience layers, music playlist, event → sound mappings, footsteps, bus gains
audioSourceany entityAn emitter: loops follow the entity, pitch and gain follow signals, one-shots fire on triggers
audioZoneany entityArea ambience that fades in as the listener approaches a sphere or box
molen.audio.*scene scriptsplay, stop and music, emitted as audio.* events
@bendyline/molen-client/audiothe pagecreateAudioLayer loads banks and plays what the rules ask for

The molen.sounds content pack carries 170 CC0 sounds (ambience, vehicles, footsteps, impacts, foley, UI, stings and six music tracks). Fetch it into a project with:

sh
npx molen pack fetch https://molen.dev/packs/index.json molen.sounds --out-dir public/packs

That writes public/packs/index.json and pins the pack in project.json, so CLI ops such as molen audio plan find the bank without flags.

Annotate a scene ​

Add one entity with an audioEnvironment. This one plays rain while it rains, birds on sunny days, music throughout, and a crash sound on the crash event your script emits:

json
{
  "id": "audio",
  "components": {
    "audioEnvironment": {
      "buses": { "music": 0.5 },
      "ambience": [
        {
          "sound": "ambience.rain",
          "when": { "weather.precipitation.kind": "rain" },
          "gainFrom": { "signal": "weather.precipitation.intensity", "curve": [[0, 0], [1, 1]] }
        },
        { "sound": "ambience.birds", "when": { "sky.daylight": { "min": 0.35 } }, "fadeS": 4 }
      ],
      "music": { "playlist": ["music.ambient.first-light", "music.ambient.lifewave"], "mode": "shuffle" },
      "events": { "crash": { "sound": "impact.metal.heavy" } }
    }
  }
}

Give a moving thing its own voice with audioSource. The engine below follows the car, and its pitch and loudness follow the car's state:

json
{
  "id": "player",
  "components": {
    "transform": { "pos": [0, 0.5, 0] },
    "audioSource": {
      "sound": "vehicle.engine.car",
      "pitchFrom": { "signal": "vehicleState.speed", "curve": [[0, 0.8], [30, 1.7]] },
      "gainFrom": { "signal": "vehicleInput.throttle", "curve": [[0, 0.5], [1, 1]] }
    }
  }
}

Put the audioSource on an entity type and every instance gets it. The molen.entities vehicles and aircraft carry their engines this way: cars run vehicle.engine.car or vehicle.engine.diesel only while someone drives them ("when": { "self.occupied": true }), with pitch from vehicleState.speed and gain from vehicleInput.throttle. The P-51 (a Merlin) and the OH-6 take pitch and gain from aircraftState.rpm, so they are silent until the engine starts and spool up and down with it. A loop whose gain is zero holds no voice.

Fire one-shots from a script when something happens that no component describes:

ts
molen.onCommand('attack', () => {
  molen.audio.play('weapon.slash');
});
if (delivered === total) molen.audio.play('sting.win');
const alarm = molen.audio.play('ui.error', { entity: 'door', loop: true });
molen.audio.stop(alarm, 0.5);
molen.audio.music('music.dungeon');

molen.emit works too: any event type listed under audioEnvironment.events plays its sound, at the entity or position the payload names (payload.entity, payload.position, or payload.a for collisions), or at the listener.

Rules ​

Every rule is a small piece of data the director re-evaluates each frame.

FieldMeaning
whenAll conditions must hold. A value means equality, a list means any of, { "min", "max" } is an inclusive range, { "exists": true } tests presence
gainFrom, pitchFrom{ signal, curve, smoothS? }: a piecewise-linear curve of [input, output] points, clamped at both ends. A list of them multiplies, so a sound can fade with height and with distance at once
triggerFrom{ signal, every?, onChange?, sound? }: a one-shot each time the signal advances by every (footsteps by distance) or changes
startTickPlay once at that tick; changing it plays again
fadeSFade time for ambience layers (default 1.5 s)

Sounds that come from the ground, such as birds in the trees or traffic on the roads, should get quieter as the listener rises. List one curve for height and another for distance to the source:

json
{
  "sound": "ambience.city.traffic",
  "gainFrom": [
    { "signal": "listener.heightAboveGround", "curve": [[3, 1], [30, 0.7], [100, 0.3], [300, 0.08], [500, 0]] },
    { "signal": "host.streetDistance", "curve": [[0, 1], [40, 0.55], [120, 0.2], [250, 0]] }
  ]
}

A layer turns on after its condition has held for 0.25 s and off after it has failed for 1 s, and numeric ranges widen by 5% while a layer plays. A signal hovering at a threshold therefore never makes a sound flicker.

Signals ​

SignalValue
<component>.<field>A field of a component on the source's entity, e.g. vehicleState.speed, aircraftState.rpm, transform.pos.1. In audioEnvironment rules it reads the listenerEntity
self.speed, self.distance, self.x/y/zThe source entity's measured speed (m/s), distance travelled (m) and position
self.occupiedtrue while someone rides the source entity (an entity's mounted component names it), in any seat but a passenger one: a passenger alone does not start an engine
listener.speed, listener.distance, listener.x/y/z, listener.altitudeThe listener's speed, odometer and position
listener.heightAboveGroundWhat the host reports as the listener's height above the terrain (m); 0 when it reports nothing
listener.mode, listener.grounded, listener.surface, listener.indoorsWhat the host reports: walk, drive, fly, pilot or orbit; on the ground; the surface underfoot; inside a building
weather.precipitation.kind, weather.precipitation.intensity, weather.windSpeed, weather.visibility, weather.clouds.coverageThe scene's weather; a scene without weather reads as clear and calm
sky.daylight, sky.starVisibility, sky.sunElevationThe sky; a scene without a sky reads as day
time.secondsSeconds since the audio layer started
host.<name>Anything the page passes in, e.g. host.urban

An unknown built-in signal is reported with a did-you-mean suggestion by molen audio plan and in the browser console.

Check it headlessly ​

molen audio plan runs the scene for N ticks, feeds the director, and prints what would play:

sh
npx molen audio plan scene.json --ticks 300 --commands commands.json
text
300 ticks at 30 Hz, banks: molen.sounds@1.0.0:sounds.soundbank.json
loops:
  [0 → end]      music.racing  bus music  peak 0.80  pitch 1.00
  [0 → end]      vehicle.engine.car @player  bus sfx  peak 0.61  pitch 0.75–1.36
one-shots (1):
  t=115    impact.metal.heavy  (sfx)

It reports unknown sound ids with a suggestion, and exits non-zero when it finds one. Pass --weather rain, --daylight 0.1, --mode walk or --listener <entity> to hear other conditions. --json prints the full report, and the plan_audio MCP tool returns the same. Script sounds are ordinary events, so assertions work on them too:

json
{ "format": "molen/assert@1", "assertions": [{ "event": "audio.play", "op": "occurred" }] }

Play it in the browser ​

A page with a Worker kernel creates one layer, attaches it to the client, and updates it each frame. Sound starts on the first click or key press, as browsers require.

ts
import { mountExperience } from '@bendyline/molen-client';
import {
  attachAutoplayUnlock,
  createAudioLayer,
  loadPackSoundbanks,
} from '@bendyline/molen-client/audio';
import { createPackSet, openPack } from '@bendyline/molen-pack';

const { client } = await mountExperience({ link: worker, scene, canvas });
const index = await (await fetch('packs/index.json')).json();
const entry = index.packs['molen.sounds'];
const packs = createPackSet([await openPack(new URL(entry.file, new URL('packs/', location.href)).href)]);
const audio = createAudioLayer({
  banks: await loadPackSoundbanks(packs),
  provider: packs.assetProvider(),
  renderer: client.renderer, // the camera is the listener; weather and sky come from here too
});
audio.attachClient(client);
attachAutoplayUnlock(audio);
requestAnimationFrame(function frame(now) {
  audio.update(now);
  requestAnimationFrame(frame);
});

Every game sample does exactly this in its src/audio.ts. Hosts without a Worker kernel use audio.attachWorld(world) for a main-thread World, or no source at all and pass the listener to update(now, { position, forward, mode, grounded, heightAboveGround }). setVolume, setMuted and setBusGain('music', 0) drive a settings menu.

mountEarthView plays the Earth soundscape on its own when its content includes the molen.sounds pack: 88 recorded ambience beds and 12 walking surfaces, plus engines on the cars and aircraft you board. Birds favor the morning; warm nights bring crickets/frogs and warm days bring cicadas. Broad geographic palettes select local recordings. Wind and rain follow weather, surf follows mapped ocean shores, rivers follow flowing water, and traffic follows streets. Ground ambience requires a known terrain height and fades with altitude; enclosed cockpits attenuate the outdoors to 18%. Open boats and exterior cameras retain outside ambience. Only one bed per category plays, with staggered rotation and five-second crossfades. See Earth view for host inputs. Pass audio: false to turn it off, and use view.audio for volume.

Vehicle details use the same simulation snapshots as the visuals:

  • Pedestrian-signal recordings belong to the earth:category:crossing pool. They require a nearby mapped or rendered traffic signal, fade to silence 55 m from the junction edge and 25 m above ground, and stay silent when the map context is unknown. Ordinary traffic keeps its separate road-distance mix. The recording contains its own signal cycle; it is not a promise that a simulated crossing is safe to use.
  • Paved airplane touchdowns play a short tire spin-up chirp, scaled by ground speed and descent rate. Retracted gear, crashes, grass/unknown surfaces, parked spawns and rapid landing bounces stay quiet. Composing hosts supply EarthAudioFrame.surfaceAt(x, z) so contact uses the aircraft's position; the standard Earth view and World Explorer do this automatically.
  • Door, hood and trunk actuator movement triggers opening/sliding foley; a closing latch sounds only when travel reaches the shut position. Van rear doors slide. Wipers sound at each half-cycle of their actual actuator phase, including the return stroke when switched off. Legacy vehicles retain boarding cues. Passenger boarding never starts or stops an engine.
  • Flap and retractable landing-gear motor loops track changes in flapPosition and gearPosition. They run during extension/retraction, preserve one voice through reversals, and fade out when the mechanism stops, including partial flap settings. Fixed or instant gear stays silent; retractable helicopter gear is supported. Pauses, teleports, removed aircraft and world switches clean up the loops. The two generic servo recordings are CC0: vacuumfan7072 and Burningmonkey.
  • Helicopter engine loops tagged rotor use estimated collective and cyclic feathering angles, bank/rate, descending inflow and listener direction to change gain, low-pass cutoff and a once-per-revolution loading envelope. Web Audio generates that envelope at audio rate; the recorded blade-passage chop remains intact. RPM still controls playback speed. This is an authored acoustic approximation, not a calibrated aerodynamic noise solver, and it does not change flight physics or replay hashes. The dependency choices are informed by NASA's maneuvering flight noise study.

Interactive emitters take priority over background beds when the voice budget is full. Missing new foley in older/custom packs is silent. The added recordings are CC0, with source pages, authors, verified license dates and source hashes in the sound pack's sources.json and credits in NOTICE.md: tire foley, wipers and sliding door foley.

Sound packs can also provide a birdsong document (molen/earth-birdsong@1). The default pack contains eight CC0 regional recordings, selected by geography, observed habitat, calendar month and local sunlight. Dawn choruses peak near sunrise, breeding songs quieten outside their authored season, and uncovered places stay quiet. A catalog replaces broad bird palettes as well as the legacy park chorus. Curated birds fade out inside enclosed vehicles. The catalog's bounds are conservative playback areas, not surveyed species ranges. Composing hosts pass EarthAudioFrame.location (latitude, longitude, sky utcMs and sunAltitudeDeg) and regionalAmbience.habitat; the standard Earth view and World Explorer supply these automatically, including changes to the sky date/time controls.

Make your own sounds ​

A sound bank is a JSON file beside its clips. Add clips with molen audio import, which copies the file, hashes it, reads its duration, and records where it came from:

sh
npx molen audio import rain.mp3 --id ambience.rain --bank sounds/sounds.soundbank.json \
  --license CC0-1.0 --source https://freesound.org/s/000000/ --loop \
  --description "Steady rain on leaves"
npx molen audio import step-2.mp3 --id footstep.wood --bank sounds/sounds.soundbank.json \
  --license CC0-1.0 --append
npx molen audio check sounds/sounds.soundbank.json

A bank with a later position in banks overrides sounds with the same id, so a project can re-skin molen.sounds by shipping only the sounds it changes. Every entry records its license. A bank with "license": "CC0-1.0" rejects anything else, which is how molen.sounds stays public domain.

TIP

MP3 plays in every browser. Ogg Vorbis does not decode in older Safari. Keep one-shots mono so they pan cleanly, and set loopStart/loopEnd a few milliseconds inside a loop so encoder padding never plays.

Good places to find sounds:

  • Freesound — the largest library; filter to Creative Commons 0.
  • Kenney — CC0 packs of UI, impacts, footsteps and jingles.
  • OpenGameArt — filter by license, then check each page, since many items are CC-BY or GPL.
  • Generated clips (ElevenLabs sound effects, Stable Audio Open) — record the prompt and the generator in the entry's source, and check the service's terms for your use.

CC-BY sounds and music (Kevin MacLeod's, for example) are fine in your own bank if you show the credit they require. Avoid the BBC Sound Effects archive (non-commercial licence) and libraries that forbid redistribution.

50 guides · 44 schema formats · 51 API entry points · 11 samples. API, CLI and schema pages are generated from the shipped build.