Voice Message Audio Decoding#

Element records and sends voice messages as Opus/Ogg audio, at 48 kHz mono with a 24 kbps bitrate for voice and 96 kbps for high-quality audio 1. On the receiving side, the Playback class in src/audio/Playback.ts owns the full decode-and-playback pipeline. The event content schema is defined in createVoiceMessageContent, which follows MSC3245/MSC1767 and names the file Voice message.ogg 2.

Decode Pipeline#

Playback.prepare() is the entry point for decoding. It handles two cases based on file size:

• Files > 5 MB — decoded through an <audio> element to avoid holding a large decoded buffer in memory 3. This path does not produce a waveform.
• Files ≤ 5 MB — decoded via AudioContext.decodeAudioData(), which is the path for all normal voice messages 4.

Safari / Opus Fallback#

Safari does not natively support Opus/Ogg decoding. When decodeAudioData fails, the error handler in Playback.prepare() calls decodeOgg() from src/audio/compat.ts, which converts the Ogg/Opus buffer to WAV using two Web Workers from the opus-recorder library:

1. decoderWorker — decodes the Opus/Ogg pages to raw PCM Float32 buffers 5.
2. wavWorker — re-encodes those buffers to a 24-bit WAV at 48 kHz 6.

The resulting WAV ArrayBuffer is passed back to decodeAudioData for a second decode attempt 7.

ArrayBuffer Detachment#

AudioContext.decodeAudioData() transfers (detaches) the ArrayBuffer it receives — after the call, buf.byteLength is 0 and the buffer cannot be re-read 8. The Playback constructor captures this.fileSize = this.buf.byteLength before any decode attempt to preserve this metadata 9.

The same issue applies when passing data to Web Workers via postMessage with a transferable list. In decodeOgg(), decoded PCM pages are transferred to wavWorker as transferable objects 10, which is correct and intentional for performance. However, when computing the playback waveform, PlaybackEncoder.getPlaybackWaveform() deliberately avoids transferring the Float32Array by copying it with Array.from(input) before sending to the worker — this prevents the detachment from corrupting the AudioBuffer's channel data in the main thread 11.

Rule of thumb: Only pass ArrayBuffer as a transferable when the main thread no longer needs it. Use Array.from() or .slice() to copy when the data must remain accessible.

Waveform Generation#

After a successful decode, Playback.prepare() calls PlaybackEncoder.instance.getPlaybackWaveform() with the first channel of the decoded AudioBuffer. This is a singleton that dispatches work to src/workers/playback.worker.ts via WorkerManager.

The worker:

1. Converts all amplitudes to absolute values 12.
2. Resamples to PLAYBACK_WAVEFORM_SAMPLES (39) using a smoothing algorithm 13.
3. Rescales to the [0, 1] range and posts the result back 14.

Key Files#