Under the hood
For the technically curious and the press: how SayToEdit Audio is built, how your own AI connects, and how we measure it.
Your own AI via MCP
SayToEdit runs an MCP server on your Mac. Connect Claude Desktop, Claude Code or another MCP app, and it edits with the same tools as the buttons in the app. Included in every plan, also Basic.
Your AI account
Your AI does the thinking on your own account. SayToEdit uses no credits for it.
Safe to try
Every change is a step in the history, marked as made by MCP, and can be undone.
Stays local
The server only listens on your Mac and needs a key. Your AI gets text and measurements, not the audio.
claude mcp add --scope user --transport http \
saytoedit-audio http://127.0.0.1:47811/mcp \
--header "Authorization: Bearer <your key>"The app shows the command with your own key under Settings > AI connection. Claude Desktop: copy the setup from the same place; it starts SayToEdit by itself.
The tools 78
Transcript
transcribe get_transcript correct_word rename_speaker compare_to_script
Cleaning
clean_audio compare_cleanings rate_quality analyze_room analyze_tape
Cutting
suggest_edits delete_words delete_range find_retakes keep_takes
Level and loudness
level_to_target measure_loudness duck_under_speech set_track_level
Plugins
list_plugins add_plugin set_plugin_parameter load_plugin_preset
Templates and formats
prepare_podcast run_format save_voice_profile place_jingle
Export and meetings
export_audio export_transcript export_summary set_summary
Project and history
get_project get_history undo redo add_marker
See all 78 tools
Read from the app’s source code when this page is built. Descriptions are in English, the way your AI sees them.
get_project- Get the project structure: tracks, clips (ids, positions in seconds), levels, markers for earlier automatic edits, and whether a transcript exists. Call this first.
get_transcript- Read the transcript as it sounds on the timeline now (after edits). Times are timeline seconds. Use format 'words' to get word indices for delete_words. Minutes of a meeting or interview: read the default text format (each turn starts with [minutes:seconds] and the speaker), then call set_summary with a short summary (3-6 sentences, same language as the meeting), decisions, tasks (who, what, deadline) and open questions, each with the time it is said; only what is actually said. Long recordings: read it in parts with from_seconds/to_seconds.
transcribe- Transcribe all audio that is not transcribed yet, with word timestamps and speaker detection. Runs locally; can take a few minutes for long recordings.
transcribe_range- Transcribe only a time range again (e.g. where words are missing or wrong) and replace the words in that range. Corrections of later words follow along. Use verbatim=true to write repetitions and broken-off words as said (e.g. 'var ikke var ikke').
find_untranscribed_speech- Find speech the transcript may have missed: stretches of at least 0.4 s of voiced speech (not breath or hiss) inside a sentence but with no words, and words that last far longer than their letters suggest (the model often swallows a repeated word). Marks them on the timeline by default. Try transcribe_range with verbatim=true there.
suggest_edits- Preview automatic edits (fillers, repetitions, long pauses) without changing anything. Returns each proposed cut with time and reason.
apply_auto_edit- Apply automatic edits: remove fillers, repetitions and/or tighten long pauses. Everything becomes one undo step and is marked on the timeline.
prepare_podcast- Prepare the whole project for a podcast in one step: transcribe what is missing and clean the sound at the same time (standard chain), then remove fillers, repetitions and long pauses, and even out the level. Each part is one undo step. Use this first for requests like 'make it ready for podcast'; then use find_retakes/review_retakes, get_transcript and delete_range for editorial cuts.
review_retakes- Review all retakes (sentences the speaker started over) at once: every group with the text and times of each take and a suggested take (the last one, usually the complete one). With apply_suggested=true the suggested takes are kept right away (one undo step). Otherwise judge them and call keep_takes.
find_retakes- Find sentences the speaker started over (retakes). Returns groups of takes with their text and times, so you can judge which take is best. Then call keep_takes.
keep_takes- Keep one take in each retake group from find_retakes and remove the others. One undo step, marked on the timeline.
delete_range- Cut out a time range on all tracks and close the gap.
delete_words- Delete words from the transcript (and the audio) by index, including the pause after them. Get indices from get_transcript with format 'words'.
split- Split all clips at a time.
move_clip- Move a clip in time and optionally to another track (e.g. to place music). Overlapping clips on the same track crossfade automatically.
trim_clip- Change a clip's in and out points (timeline seconds).
set_fades- Set fade in and fade out length on a clip (seconds).
delete_clip- Remove a whole clip (without closing the gap).
set_track_level- Set a track's level in dB.
set_clip_level- Set a clip's overall level in dB.
set_level_points- Draw a level curve on a clip with points, e.g. to fade music up and down at specific times.
duck_under_speech- Lower a track (usually music) automatically whenever someone speaks on the other tracks, with smooth fades. Undoable.
set_track_solo- Solo a track (only solo tracks are heard) or turn solo off.
list_plugins- List the audio plugins (AU/VST3) installed on this computer.
add_plugin- Put a plugin at the end of a track's plugin chain. The user can open its window in the editor to adjust it.
remove_plugin- Remove a plugin from a track.
set_plugin_bypass- Turn a plugin off (bypass) or on.
get_plugin_parameters- List a plugin's parameters: id, name, unit, min, max, current value, and value names for choices. Use set_plugin_parameter to change them (e.g. turn down the strength of a noise reduction plugin).
set_plugin_parameter- Set one or more plugin parameters at once. The plugin's new settings are saved in the project as one undoable step.
list_plugin_presets- List a plugin's presets: factory presets (number 0 and up), user presets (negative numbers) and the current one.
load_plugin_preset- Load a plugin preset. The plugin's new settings are saved in the project as one undoable step.
set_track_muted- Mute or unmute a track.
import_audio- Import an audio file (e.g. music or a jingle) onto a new track starting at 0 s. Use move_clip, set_track_level and set_fades afterwards to place it.
clean_audio- Clean audio locally: noise reduction (light = at most 6 dB, medium = 12 dB, strong = 24 dB; optionally extra noise reduction and keep_character), calm background between words, room reverb removal (dereverb), broadcast voice (EQ, de-esser, compressor), level evening, and for old tapes: hum removal, click and dropout repair, careful treble lift and wow/flutter correction. Use analyze_tape first for tapes. Replaces the tracks' audio with a cleaned version (undoable). All options off restores the original.
list_recipes- List saved recipes (reusable workflows such as 'Klar til podcast').
run_recipe- Run a saved recipe on the whole project.
export_transcript- Export the transcript (what is on the timeline now) as text: Word (.docx), PDF, plain text, SRT or VTT subtitles, or JSON. With title, date, duration and participants at the top, speaker names, timestamps and a note that the text is machine-made. Rename speakers first with rename_speaker so the names are right.
set_summary- Save a summary (minutes) of a meeting or interview in the project, where the user sees it and can export it. Make it from get_transcript (format "text" has [mm:ss] timestamps): a short summary, decisions, tasks (who, what, deadline) and open questions, each with the time it is said. Only include what is actually said. Replaces an earlier summary (undoable).
list_voice_profiles- List the voice profiles: saved cleanings for perfect speech (built-in and the user's own), with their choices, plugins, loudness target and an explanation.
save_voice_profile- Save a voice profile: from a track's current cleaning and plugins (from_track_id), or from an explicit cleaning. Explain the choices in 'description'. To find good settings first, use compare_cleanings and rate_quality on a typical excerpt, or create_voice_profile_from_audio.
apply_voice_profile- Apply a voice profile: clean with its choices and put its plugins (with their settings) on the track. One undo step per part.
create_voice_profile_from_audio- Make a voice profile from a typical excerpt: tries the built-in profiles (and Mac voice isolation if available) with compare_cleanings, rates them, saves the best as a new profile with an explanation, and answers with all scores. The excerpt is not changed. Apply it afterwards with apply_voice_profile.
list_library- List the sound library: intros, outros, jingles, music and effects with id, name, kind, length (s), loudness (LUFS), tags and default fades, level and ducking.
add_to_library- Add an audio file to the sound library (it is copied in, measured and kept). Kind: intro, outro, jingle, music or effect.
place_jingle- Put a sound from the library on the "Jingler" track: at the start (the rest moves so speech starts as it fades), at the end, after a sentence (makes room) or on top at a time. Uses its default fades and level relative to the speech; music is ducked under the speech. Undoable.
list_formats- List the episode formats: intro, outro, jingle (every N minutes), music, voice profile, cut rules, loudness, export (mp3/wav, bitrate, artist, album, title with {nr}) and the next episode number.
save_format- Save an episode format (a recipe for a whole episode), from the open project (from_project) or from the fields. Library sounds and profiles by id or name.
run_format- Make a whole episode from a format in one go: (new project with the raw files,) clean with the voice profile, transcribe, cut by the rules, even level, intro, jingles, outro and music, then export with title and episode number. Takes a few minutes for long recordings. The episode number counts up afterwards.
merge_speakers- Merge two speakers when speaker detection split one person in two: every word from from_speaker gets into_speaker. Speaker ids start at 0 ("Taler 1" is 0).
export_summary- Export the project's summary (minutes) as Word (.docx), PDF or plain text.
save_project- Save the project. Give 'path' to save under a new name (required if the project has never been saved).
undo- Undo the last change (any change, also the user's).
get_history- List the undo history, newest first: step number, what was done, who did it (user, agent = the editor's assistant, mcp = you or another connected AI) and whether it is undone.
measure_loudness- Measure how the timeline sounds (after plugins, exactly as exported) in a time range: integrated loudness (LUFS), loudness range (LRA), highest true peak (dBTP), noise in the pauses (pauseNoiseDbfs: median in quiet parts of at least 150 ms; quietest10pDbfs is the 10th percentile of 100 ms RMS, which hits breaths when pauses are tight), number of 100 ms places with peaks over -1 dBTP, and short-term loudness (3 s) over time (at most 600 points). With per_sentence: LUFS for each sentence. Use it to check levels before and after edits.
set_script- Set the text that is read aloud (reading with a manuscript). Afterwards find_retakes, suggest_edits and apply_auto_edit use it: repetitions that are in the text are kept, words that are not in the text and start over are restarts. Undoable.
compare_to_script- Compare the transcript with the reading text: words said that are not in the text (with times; errors, restarts or fillers), words that are spelled differently than in the text (e.g. names), and words from the text that were not said.
models_status- List the local AI models: whether each is installed, its size and license, the installed and the newest version, and whether a newer version can be downloaded (the user downloads models in Settings).
analyze_room- Measure the room: reverb tail 50-200 ms after words (tailDb, dB relative to the end of the word; higher = more reverb; a dry room is about -30), measured on the sound before cleaning (first 10 minutes). 'reverberant' says whether clean_audio with dereverb is worth trying.
rate_quality- Listening test: compare a time range as it sounds now with the sound before cleaning and plugins. Gives a score from 1 to 5, voice preservation per band (bass 100-300 Hz, mid 300-2000, presence 2-5 kHz, treble 5-10 kHz; loss relative to the mid band), noise in pauses before/after, 'musical noise' (short tonal remnants in pauses, per mille), pumping and the reverb tail after words before/after, with an explanation in Danish.
compare_cleanings- Compare up to 6 cleanings on an excerpt (at most 60 s) without changing the project: built-in cleaning with different options and/or plugin chains with presets or parameters. Gives a table: loudness, noise in pauses, quality score and voice preservation (listening test), and how long it took. Each result is also saved as a WAV file to listen to.
analyze_tape- Analyze old tape recordings: per channel noise floor, signal-to-noise and bandwidth, correlation and time offset between channels with a recommended channel choice; mains hum (exact frequency); tape speed from the hum (when the hum is clear); clicks per minute; bandwidth; noise floor; voice pitch; and whether wow/flutter can be corrected. Use the result to choose set_channel_mode, set_clip_speed and clean_audio options.
analyze_channels- Analyze the two channels of stereo recordings (noise, bandwidth, correlation, time offset) and recommend left, right, both or both aligned. Same channel part as analyze_tape.
set_channel_mode- Choose which channels of a stereo recording are heard (e.g. only the better channel of an old tape, or both mixed after correcting a head misalignment). Applies to all clips from the recording. Undoable.
set_clip_speed- Set the playback speed of a recording (varispeed), e.g. to correct a tape that ran too fast or slow. analyze_tape gives the speed from the mains hum when possible. All clips from the recording follow and keep their order. Undoable.
measure_pitch- Measure the voice pitch (fundamental frequency) in a time range as it sounds now (with any speed change): median and spread in Hz.
level_to_target- Even out the level: measure every sentence (after plugins; without a transcript, speech parts found by loudness) and set level points on the clips so each sentence reaches the target loudness, with 50 ms smooth transitions in the pauses. One undo step.
get_waveform- Get the energy curve of a time range: RMS level in dBFS per point (default 200 points, at most 2000), and quiet places (start, end, quietest point, level), e.g. pauses between words where a cut is clean.
add_marker- Put a note marker on the timeline (shown in amber in the ruler; the note appears when the user points at it). Undoable.
remove_marker- Remove a marker (a note or a mark from an earlier automatic edit). Undoable.
list_markers- List all markers in time order: id, seconds, kind (note, or filler/pause/repetition/cut/cleaning/click/dropout for automatic edits), the note or label, and who made it.
redo- Redo the last undone change.
correct_word- Correct a misheard word in the transcript (e.g. a name). Only the text changes, not the audio. Undoable.
rename_speaker- Rename a speaker in the transcript (e.g. 'Taler 1' becomes the host's name).
export_audio- Export the timeline (or a time range) to a WAV file (24 bit, 48 kHz) or an MP3 file (44.1 kHz CBR, 128 kbit/s stereo / 96 mono by default, with ID3 tags) with podcast loudness (-16 LUFS stereo / -19 LUFS mono, peaks below -1 dBTP). Waits at most 20 seconds; long exports continue in the background and answer with a job id for export_status.
start_export- Start exporting to a WAV or MP3 file in the background and answer at once with a job id. Same options as export_audio. Follow it with export_status.
export_status- Progress of an export started with start_export or export_audio: progress 0-1, done, error and the loudness report.
play- Play the timeline so the user can listen, optionally from a time.
pause- Pause playback.
Architecture
Rust from end to end
The audio engine, cleaning, transcription and the editing model are written in Rust, in separate crates. The app is built with Tauri 2 and a React interface.
Local models
Speech to text, speaker recognition, noise removal and the optional language model all run on the Mac. Large language models run in their own process.
One shared layer
Buttons, the built-in assistant and MCP call the same functions. What your AI does is exactly what a button does.
MCP server
Streamable HTTP on 127.0.0.1 with a key, plus a small stdio bridge for Claude Desktop that starts the app when needed. The built-in assistant connects to the same server in memory.
Plugin host
Audio Units are hosted by a separate Swift program that the editor talks to over a socket, apart from the editor itself.
Nothing is lost
Every edit is an action in a history that records who made it: you, the assistant or MCP. Cleaning writes new files and keeps the original.
The cleaning chain
The order matters. Speech is found before the noise removal, and the calm background is applied last, so the levelling cannot lift the pauses unevenly.
- Room echoWPE: STFT 2048/512, delay 3 frames, filter 32 frames, 3 passes, one Cholesky solve per frequency band, on all cores.
- NoiseDeepFilterNet3. The speech is lifted to a fixed level first and lowered again afterwards, so quiet voices are not removed. Limits: 6, 12 or 24 dB.
- VoiceHigh-pass, de-esser, automatic EQ measured against a standard speech spectrum (bass, presence, treble, at most ±4 dB), compressor.
- LevelEach sentence is measured and moved towards the target, at most ±12 dB, with 50 ms transitions in the pauses.
- Calm backgroundSpeech found in 10 ms steps before the noise removal; pauses lowered 6, 10 or 14 dB and filled with comfort noise in the same spectrum.
- LoudnessITU-R BS.1770-4 measurement and a true-peak limiter on export: -16 LUFS stereo, -19 LUFS mono, peaks below -1 dBTP.
Old tape adds, before the noise removal: wow and flutter (experimental), clicks and dropouts (LPC order 32), hum and its harmonics, and afterwards a gentle treble lift.
Measurements
All numbers are our own, measured on a Mac with Apple silicon with the tests in the source code.
Standard chain against leading commercial AI plugins
Three 30 s excerpts (start, middle, end) of a real recording. Commercial chain: noise removal, echo removal and voice enhancement plugins from one vendor, without levelling (so we compare the background relative to the speech, not in dBFS). Character: SayToEdit’s own automatic listening test (1-5). Pumping: how much the level in the pauses varies (dB). Time: per 30 s. The result applies to this recording; other recordings can give other results.
| Excerpt | Chain | Background below speech | Musical noise | Character | Pumping | Time |
|---|---|---|---|---|---|---|
| Start | SayToEdit | -42.0 dB | 10.5 ‰ | 5.0 | 1.8 dB | 4.5 s |
| Commercial | -39.7 dB | 52.5 ‰ | 5.0 | 2.7 dB | 3.2 s | |
| Middle | SayToEdit | -44.5 dB | 2.2 ‰ | 5.0 | 0.9 dB | 5.4 s |
| Commercial | -39.3 dB | 53.3 ‰ | 5.0 | 1.9 dB | 3.4 s | |
| End | SayToEdit | -44.2 dB | 17.6 ‰ | 5.0 | 2.4 dB | 5.8 s |
| Commercial | -40.1 dB | 56.8 ‰ | 4.0 | 1.3 dB | 3.3 s |
SayToEdit is 2-5 dB quieter in the pauses relative to the speech, has a fraction of the musical noise, and keeps character 5.0 in all three excerpts. The commercial chain is faster and pumps less in the last excerpt.
Other measurements
- Room echo removal (WPE, medium)
- Reverb tail lowered 3.5 dB (RT60 0.4 s) and 3.9 dB (RT60 0.8 s) on artificial reverb, character 5.0, tone changed at most 0.1 dB. About 1.2 s per 30 s of mono.
- Format: raw recording to MP3
- 5:22 in 32-36 s; 30:00 in 3:45 with intro, outro and 11 jingles. -16.0 LUFS, peak -1.8 to -1.9 dBTP.
- Meeting transcription
- 2.5 hours with three speakers: 3.0-6.1 minutes per hour of audio, speakers consistent in 99.8-100% of the words, memory at most 2.5-3.0 GB.
- Spoken commands
- From releasing “Talk” to text: 0.3-0.6 s with Whisper small on the Mac.
- Words kept
- A full 5:22 recording before and after cleaning: 692 and 694 words, first and last word the same; -16.0 LUFS, -1.2 dBTP.
Models and licences
The models are downloaded from their official sources on first start and checked. All are under open licences: MIT, Apache 2.0 and, for the MP3 library, LGPL.
| Model | Used for | Licence |
|---|---|---|
| Whisper large-v3 turbo (whisper.cpp) | Speech to text with word timings | MIT |
| 3D-Speaker CAM++ (sherpa-onnx) | Speaker recognition | Apache-2.0 |
| DeepFilterNet3 | Noise removal | MIT / Apache-2.0 |
| Qwen3 4B (llama.cpp, optional) | Local language model: best take, summary offline | Apache-2.0 |
| Whisper small (optional) | Spoken commands | MIT |
| LAME 3.100 | MP3 export, separate dynamic library | LGPL 2 |
The app includes open-source software and models under their own licences. See the licence page.