Technology

Under the hood

For the technically curious and the press: how SayToEdit Audio is built, how your own AI connects, and how we measure it.

Your own AI via MCP

SayToEdit runs an MCP server on your Mac. Connect Claude Desktop, Claude Code or another MCP app, and it edits with the same tools as the buttons in the app. Included in every plan, also Basic.

Your AI account

Your AI does the thinking on your own account. SayToEdit uses no credits for it.

Safe to try

Every change is a step in the history, marked as made by MCP, and can be undone.

Stays local

The server only listens on your Mac and needs a key. Your AI gets text and measurements, not the audio.

Connect Claude Code in one line
claude mcp add --scope user --transport http \
  saytoedit-audio http://127.0.0.1:47811/mcp \
  --header "Authorization: Bearer <your key>"

The app shows the command with your own key under Settings > AI connection. Claude Desktop: copy the setup from the same place; it starts SayToEdit by itself.

The tools 78

Transcript

transcribe get_transcript correct_word rename_speaker compare_to_script

Cleaning

clean_audio compare_cleanings rate_quality analyze_room analyze_tape

Cutting

suggest_edits delete_words delete_range find_retakes keep_takes

Level and loudness

level_to_target measure_loudness duck_under_speech set_track_level

Plugins

list_plugins add_plugin set_plugin_parameter load_plugin_preset

Templates and formats

prepare_podcast run_format save_voice_profile place_jingle

Export and meetings

export_audio export_transcript export_summary set_summary

Project and history

get_project get_history undo redo add_marker

See all 78 tools

Read from the app’s source code when this page is built. Descriptions are in English, the way your AI sees them.

get_project
Get the project structure: tracks, clips (ids, positions in seconds), levels, markers for earlier automatic edits, and whether a transcript exists. Call this first.
get_transcript
Read the transcript as it sounds on the timeline now (after edits). Times are timeline seconds. Use format 'words' to get word indices for delete_words. Minutes of a meeting or interview: read the default text format (each turn starts with [minutes:seconds] and the speaker), then call set_summary with a short summary (3-6 sentences, same language as the meeting), decisions, tasks (who, what, deadline) and open questions, each with the time it is said; only what is actually said. Long recordings: read it in parts with from_seconds/to_seconds.
transcribe
Transcribe all audio that is not transcribed yet, with word timestamps and speaker detection. Runs locally; can take a few minutes for long recordings.
transcribe_range
Transcribe only a time range again (e.g. where words are missing or wrong) and replace the words in that range. Corrections of later words follow along. Use verbatim=true to write repetitions and broken-off words as said (e.g. 'var ikke var ikke').
find_untranscribed_speech
Find speech the transcript may have missed: stretches of at least 0.4 s of voiced speech (not breath or hiss) inside a sentence but with no words, and words that last far longer than their letters suggest (the model often swallows a repeated word). Marks them on the timeline by default. Try transcribe_range with verbatim=true there.
suggest_edits
Preview automatic edits (fillers, repetitions, long pauses) without changing anything. Returns each proposed cut with time and reason.
apply_auto_edit
Apply automatic edits: remove fillers, repetitions and/or tighten long pauses. Everything becomes one undo step and is marked on the timeline.
prepare_podcast
Prepare the whole project for a podcast in one step: transcribe what is missing and clean the sound at the same time (standard chain), then remove fillers, repetitions and long pauses, and even out the level. Each part is one undo step. Use this first for requests like 'make it ready for podcast'; then use find_retakes/review_retakes, get_transcript and delete_range for editorial cuts.
review_retakes
Review all retakes (sentences the speaker started over) at once: every group with the text and times of each take and a suggested take (the last one, usually the complete one). With apply_suggested=true the suggested takes are kept right away (one undo step). Otherwise judge them and call keep_takes.
find_retakes
Find sentences the speaker started over (retakes). Returns groups of takes with their text and times, so you can judge which take is best. Then call keep_takes.
keep_takes
Keep one take in each retake group from find_retakes and remove the others. One undo step, marked on the timeline.
delete_range
Cut out a time range on all tracks and close the gap.
delete_words
Delete words from the transcript (and the audio) by index, including the pause after them. Get indices from get_transcript with format 'words'.
split
Split all clips at a time.
move_clip
Move a clip in time and optionally to another track (e.g. to place music). Overlapping clips on the same track crossfade automatically.
trim_clip
Change a clip's in and out points (timeline seconds).
set_fades
Set fade in and fade out length on a clip (seconds).
delete_clip
Remove a whole clip (without closing the gap).
set_track_level
Set a track's level in dB.
set_clip_level
Set a clip's overall level in dB.
set_level_points
Draw a level curve on a clip with points, e.g. to fade music up and down at specific times.
duck_under_speech
Lower a track (usually music) automatically whenever someone speaks on the other tracks, with smooth fades. Undoable.
set_track_solo
Solo a track (only solo tracks are heard) or turn solo off.
list_plugins
List the audio plugins (AU/VST3) installed on this computer.
add_plugin
Put a plugin at the end of a track's plugin chain. The user can open its window in the editor to adjust it.
remove_plugin
Remove a plugin from a track.
set_plugin_bypass
Turn a plugin off (bypass) or on.
get_plugin_parameters
List a plugin's parameters: id, name, unit, min, max, current value, and value names for choices. Use set_plugin_parameter to change them (e.g. turn down the strength of a noise reduction plugin).
set_plugin_parameter
Set one or more plugin parameters at once. The plugin's new settings are saved in the project as one undoable step.
list_plugin_presets
List a plugin's presets: factory presets (number 0 and up), user presets (negative numbers) and the current one.
load_plugin_preset
Load a plugin preset. The plugin's new settings are saved in the project as one undoable step.
set_track_muted
Mute or unmute a track.
import_audio
Import an audio file (e.g. music or a jingle) onto a new track starting at 0 s. Use move_clip, set_track_level and set_fades afterwards to place it.
clean_audio
Clean audio locally: noise reduction (light = at most 6 dB, medium = 12 dB, strong = 24 dB; optionally extra noise reduction and keep_character), calm background between words, room reverb removal (dereverb), broadcast voice (EQ, de-esser, compressor), level evening, and for old tapes: hum removal, click and dropout repair, careful treble lift and wow/flutter correction. Use analyze_tape first for tapes. Replaces the tracks' audio with a cleaned version (undoable). All options off restores the original.
list_recipes
List saved recipes (reusable workflows such as 'Klar til podcast').
run_recipe
Run a saved recipe on the whole project.
export_transcript
Export the transcript (what is on the timeline now) as text: Word (.docx), PDF, plain text, SRT or VTT subtitles, or JSON. With title, date, duration and participants at the top, speaker names, timestamps and a note that the text is machine-made. Rename speakers first with rename_speaker so the names are right.
set_summary
Save a summary (minutes) of a meeting or interview in the project, where the user sees it and can export it. Make it from get_transcript (format "text" has [mm:ss] timestamps): a short summary, decisions, tasks (who, what, deadline) and open questions, each with the time it is said. Only include what is actually said. Replaces an earlier summary (undoable).
list_voice_profiles
List the voice profiles: saved cleanings for perfect speech (built-in and the user's own), with their choices, plugins, loudness target and an explanation.
save_voice_profile
Save a voice profile: from a track's current cleaning and plugins (from_track_id), or from an explicit cleaning. Explain the choices in 'description'. To find good settings first, use compare_cleanings and rate_quality on a typical excerpt, or create_voice_profile_from_audio.
apply_voice_profile
Apply a voice profile: clean with its choices and put its plugins (with their settings) on the track. One undo step per part.
create_voice_profile_from_audio
Make a voice profile from a typical excerpt: tries the built-in profiles (and Mac voice isolation if available) with compare_cleanings, rates them, saves the best as a new profile with an explanation, and answers with all scores. The excerpt is not changed. Apply it afterwards with apply_voice_profile.
list_library
List the sound library: intros, outros, jingles, music and effects with id, name, kind, length (s), loudness (LUFS), tags and default fades, level and ducking.
add_to_library
Add an audio file to the sound library (it is copied in, measured and kept). Kind: intro, outro, jingle, music or effect.
place_jingle
Put a sound from the library on the "Jingler" track: at the start (the rest moves so speech starts as it fades), at the end, after a sentence (makes room) or on top at a time. Uses its default fades and level relative to the speech; music is ducked under the speech. Undoable.
list_formats
List the episode formats: intro, outro, jingle (every N minutes), music, voice profile, cut rules, loudness, export (mp3/wav, bitrate, artist, album, title with {nr}) and the next episode number.
save_format
Save an episode format (a recipe for a whole episode), from the open project (from_project) or from the fields. Library sounds and profiles by id or name.
run_format
Make a whole episode from a format in one go: (new project with the raw files,) clean with the voice profile, transcribe, cut by the rules, even level, intro, jingles, outro and music, then export with title and episode number. Takes a few minutes for long recordings. The episode number counts up afterwards.
merge_speakers
Merge two speakers when speaker detection split one person in two: every word from from_speaker gets into_speaker. Speaker ids start at 0 ("Taler 1" is 0).
export_summary
Export the project's summary (minutes) as Word (.docx), PDF or plain text.
save_project
Save the project. Give 'path' to save under a new name (required if the project has never been saved).
undo
Undo the last change (any change, also the user's).
get_history
List the undo history, newest first: step number, what was done, who did it (user, agent = the editor's assistant, mcp = you or another connected AI) and whether it is undone.
measure_loudness
Measure how the timeline sounds (after plugins, exactly as exported) in a time range: integrated loudness (LUFS), loudness range (LRA), highest true peak (dBTP), noise in the pauses (pauseNoiseDbfs: median in quiet parts of at least 150 ms; quietest10pDbfs is the 10th percentile of 100 ms RMS, which hits breaths when pauses are tight), number of 100 ms places with peaks over -1 dBTP, and short-term loudness (3 s) over time (at most 600 points). With per_sentence: LUFS for each sentence. Use it to check levels before and after edits.
set_script
Set the text that is read aloud (reading with a manuscript). Afterwards find_retakes, suggest_edits and apply_auto_edit use it: repetitions that are in the text are kept, words that are not in the text and start over are restarts. Undoable.
compare_to_script
Compare the transcript with the reading text: words said that are not in the text (with times; errors, restarts or fillers), words that are spelled differently than in the text (e.g. names), and words from the text that were not said.
models_status
List the local AI models: whether each is installed, its size and license, the installed and the newest version, and whether a newer version can be downloaded (the user downloads models in Settings).
analyze_room
Measure the room: reverb tail 50-200 ms after words (tailDb, dB relative to the end of the word; higher = more reverb; a dry room is about -30), measured on the sound before cleaning (first 10 minutes). 'reverberant' says whether clean_audio with dereverb is worth trying.
rate_quality
Listening test: compare a time range as it sounds now with the sound before cleaning and plugins. Gives a score from 1 to 5, voice preservation per band (bass 100-300 Hz, mid 300-2000, presence 2-5 kHz, treble 5-10 kHz; loss relative to the mid band), noise in pauses before/after, 'musical noise' (short tonal remnants in pauses, per mille), pumping and the reverb tail after words before/after, with an explanation in Danish.
compare_cleanings
Compare up to 6 cleanings on an excerpt (at most 60 s) without changing the project: built-in cleaning with different options and/or plugin chains with presets or parameters. Gives a table: loudness, noise in pauses, quality score and voice preservation (listening test), and how long it took. Each result is also saved as a WAV file to listen to.
analyze_tape
Analyze old tape recordings: per channel noise floor, signal-to-noise and bandwidth, correlation and time offset between channels with a recommended channel choice; mains hum (exact frequency); tape speed from the hum (when the hum is clear); clicks per minute; bandwidth; noise floor; voice pitch; and whether wow/flutter can be corrected. Use the result to choose set_channel_mode, set_clip_speed and clean_audio options.
analyze_channels
Analyze the two channels of stereo recordings (noise, bandwidth, correlation, time offset) and recommend left, right, both or both aligned. Same channel part as analyze_tape.
set_channel_mode
Choose which channels of a stereo recording are heard (e.g. only the better channel of an old tape, or both mixed after correcting a head misalignment). Applies to all clips from the recording. Undoable.
set_clip_speed
Set the playback speed of a recording (varispeed), e.g. to correct a tape that ran too fast or slow. analyze_tape gives the speed from the mains hum when possible. All clips from the recording follow and keep their order. Undoable.
measure_pitch
Measure the voice pitch (fundamental frequency) in a time range as it sounds now (with any speed change): median and spread in Hz.
level_to_target
Even out the level: measure every sentence (after plugins; without a transcript, speech parts found by loudness) and set level points on the clips so each sentence reaches the target loudness, with 50 ms smooth transitions in the pauses. One undo step.
get_waveform
Get the energy curve of a time range: RMS level in dBFS per point (default 200 points, at most 2000), and quiet places (start, end, quietest point, level), e.g. pauses between words where a cut is clean.
add_marker
Put a note marker on the timeline (shown in amber in the ruler; the note appears when the user points at it). Undoable.
remove_marker
Remove a marker (a note or a mark from an earlier automatic edit). Undoable.
list_markers
List all markers in time order: id, seconds, kind (note, or filler/pause/repetition/cut/cleaning/click/dropout for automatic edits), the note or label, and who made it.
redo
Redo the last undone change.
correct_word
Correct a misheard word in the transcript (e.g. a name). Only the text changes, not the audio. Undoable.
rename_speaker
Rename a speaker in the transcript (e.g. 'Taler 1' becomes the host's name).
export_audio
Export the timeline (or a time range) to a WAV file (24 bit, 48 kHz) or an MP3 file (44.1 kHz CBR, 128 kbit/s stereo / 96 mono by default, with ID3 tags) with podcast loudness (-16 LUFS stereo / -19 LUFS mono, peaks below -1 dBTP). Waits at most 20 seconds; long exports continue in the background and answer with a job id for export_status.
start_export
Start exporting to a WAV or MP3 file in the background and answer at once with a job id. Same options as export_audio. Follow it with export_status.
export_status
Progress of an export started with start_export or export_audio: progress 0-1, done, error and the loudness report.
play
Play the timeline so the user can listen, optionally from a time.
pause
Pause playback.

Architecture

Rust from end to end

The audio engine, cleaning, transcription and the editing model are written in Rust, in separate crates. The app is built with Tauri 2 and a React interface.

Local models

Speech to text, speaker recognition, noise removal and the optional language model all run on the Mac. Large language models run in their own process.

One shared layer

Buttons, the built-in assistant and MCP call the same functions. What your AI does is exactly what a button does.

MCP server

Streamable HTTP on 127.0.0.1 with a key, plus a small stdio bridge for Claude Desktop that starts the app when needed. The built-in assistant connects to the same server in memory.

Plugin host

Audio Units are hosted by a separate Swift program that the editor talks to over a socket, apart from the editor itself.

Nothing is lost

Every edit is an action in a history that records who made it: you, the assistant or MCP. Cleaning writes new files and keeps the original.

The cleaning chain

The order matters. Speech is found before the noise removal, and the calm background is applied last, so the levelling cannot lift the pauses unevenly.

  1. Room echoWPE: STFT 2048/512, delay 3 frames, filter 32 frames, 3 passes, one Cholesky solve per frequency band, on all cores.
  2. NoiseDeepFilterNet3. The speech is lifted to a fixed level first and lowered again afterwards, so quiet voices are not removed. Limits: 6, 12 or 24 dB.
  3. VoiceHigh-pass, de-esser, automatic EQ measured against a standard speech spectrum (bass, presence, treble, at most ±4 dB), compressor.
  4. LevelEach sentence is measured and moved towards the target, at most ±12 dB, with 50 ms transitions in the pauses.
  5. Calm backgroundSpeech found in 10 ms steps before the noise removal; pauses lowered 6, 10 or 14 dB and filled with comfort noise in the same spectrum.
  6. LoudnessITU-R BS.1770-4 measurement and a true-peak limiter on export: -16 LUFS stereo, -19 LUFS mono, peaks below -1 dBTP.

Old tape adds, before the noise removal: wow and flutter (experimental), clicks and dropouts (LPC order 32), hum and its harmonics, and afterwards a gentle treble lift.

Measurements

All numbers are our own, measured on a Mac with Apple silicon with the tests in the source code.

Standard chain against leading commercial AI plugins

Three 30 s excerpts (start, middle, end) of a real recording. Commercial chain: noise removal, echo removal and voice enhancement plugins from one vendor, without levelling (so we compare the background relative to the speech, not in dBFS). Character: SayToEdit’s own automatic listening test (1-5). Pumping: how much the level in the pauses varies (dB). Time: per 30 s. The result applies to this recording; other recordings can give other results.

ExcerptChainBackground below speechMusical noiseCharacterPumpingTime
StartSayToEdit-42.0 dB10.5 ‰5.01.8 dB4.5 s
Commercial-39.7 dB52.5 ‰5.02.7 dB3.2 s
MiddleSayToEdit-44.5 dB2.2 ‰5.00.9 dB5.4 s
Commercial-39.3 dB53.3 ‰5.01.9 dB3.4 s
EndSayToEdit-44.2 dB17.6 ‰5.02.4 dB5.8 s
Commercial-40.1 dB56.8 ‰4.01.3 dB3.3 s

SayToEdit is 2-5 dB quieter in the pauses relative to the speech, has a fraction of the musical noise, and keeps character 5.0 in all three excerpts. The commercial chain is faster and pumps less in the last excerpt.

Other measurements

Room echo removal (WPE, medium)
Reverb tail lowered 3.5 dB (RT60 0.4 s) and 3.9 dB (RT60 0.8 s) on artificial reverb, character 5.0, tone changed at most 0.1 dB. About 1.2 s per 30 s of mono.
Format: raw recording to MP3
5:22 in 32-36 s; 30:00 in 3:45 with intro, outro and 11 jingles. -16.0 LUFS, peak -1.8 to -1.9 dBTP.
Meeting transcription
2.5 hours with three speakers: 3.0-6.1 minutes per hour of audio, speakers consistent in 99.8-100% of the words, memory at most 2.5-3.0 GB.
Spoken commands
From releasing “Talk” to text: 0.3-0.6 s with Whisper small on the Mac.
Words kept
A full 5:22 recording before and after cleaning: 692 and 694 words, first and last word the same; -16.0 LUFS, -1.2 dBTP.

Models and licences

The models are downloaded from their official sources on first start and checked. All are under open licences: MIT, Apache 2.0 and, for the MP3 library, LGPL.

ModelUsed forLicence
Whisper large-v3 turbo (whisper.cpp)Speech to text with word timingsMIT
3D-Speaker CAM++ (sherpa-onnx)Speaker recognitionApache-2.0
DeepFilterNet3Noise removalMIT / Apache-2.0
Qwen3 4B (llama.cpp, optional)Local language model: best take, summary offlineApache-2.0
Whisper small (optional)Spoken commandsMIT
LAME 3.100MP3 export, separate dynamic libraryLGPL 2

The app includes open-source software and models under their own licences. See the licence page.