Building Audia: What It Took to Control Per-App Audio on macOS

The first version of Audia used the same basic trick that many macOS audio utilities have used for years: route audio through a virtual device.

It worked.

I also hated it.

A user would install Audia and suddenly macOS could show an audio-capture indicator. Even knowing the processing was local, the experience felt wrong. If I were a normal user, I would wonder whether some process was recording me or sending audio somewhere.

The engineering experience wasn't much better.

Virtual audio drivers introduced another component that could fail independently of the app. Minor IPC failures could leave the driver disconnected until restart. macOS updates could change behavior. Driver crashes could paralyze the entire routing pipeline. Debugging often meant understanding not only Audia, but also the virtual device sitting between Audia and Core Audio.

We used projects such as BlackHole to understand the established approach, but eventually started asking a more useful question:

Why should changing Spotify's volume require installing another audio device at all?

That question eventually caused us to throw away a large part of Audia's original audio architecture.

What replaced it taught me far more about macOS audio than building the first version ever did.

The rabbit hole: applications were already doing this without drivers

The frustrating part was that I knew driverless capture was possible.

Applications such as Audio Hijack could manipulate application audio without asking users to install the sort of virtual driver we were relying on.

But knowing that something exists and knowing how it exists are very different things.

I searched through Core Audio documentation, examples, headers, and anything else I could find. For a while I couldn't find an API that matched the behavior I was seeing in commercial applications.

Eventually I found AudioCap.

Its license meant we couldn't simply copy its implementation into Audia, but that wasn't the important discovery.

The important discovery was that the mechanism existed.

Modern macOS exposes Core Audio Process Taps through APIs including:

CATapDescription
AudioHardwareCreateProcessTap(...)
AudioHardwareDestroyProcessTap(...)

That changed the direction of the project.

Instead of maintaining a driver and communicating with it, we could stay almost entirely inside native Swift and Core Audio HAL.

The architecture went from roughly:

Application
    ↓
Virtual audio driver
    ↓
Audia
    ↓
Output device

to:

Application
    ↓
Core Audio process tap
    ↓
Audia DSP / routing
    ↓
Physical output device

At least, that's what I thought it would look like.

The real architecture turned out to be stranger.

A process tap doesn't actually give you a replacement output path

Creating the tap itself is pleasantly small.

Conceptually, Audia does something like:

let tapDescription = CATapDescription(
    stereoMixdownOfProcesses: [process.objectID]
)
 
tapDescription.uuid = UUID()
tapDescription.muteBehavior = .mutedWhenTapped
 
var tapID: AudioObjectID = .unknown
 
let status = AudioHardwareCreateProcessTap(
    tapDescription,
    &tapID
)

The process AudioObjectID comes from HAL. Audia discovers audio-producing processes using properties such as:

kAudioHardwarePropertyProcessObjectList
kAudioHardwarePropertyTranslatePIDToProcessObject
kAudioProcessPropertyBundleID
kAudioProcessPropertyIsRunningOutput

CATapDescription tells Core Audio which process we want.

The mute behavior is important. Once Audia takes responsibility for that application's audio, we don't want the original stream reaching the speakers as well or we'd hear both paths simultaneously.

At this point I initially imagined the remaining pipeline would be something like:

tap → callback → speakers

It isn't.

A process tap exposes audio buffers, but the tap itself is not an output device.

To get those buffers into a physical DAC, Audia has to construct another Core Audio object around the tap.

That object is a private Aggregate Device.

The aggregate-device trick was the first big surprise

Aggregate devices normally make sense in studio setups.

Maybe you have two physical interfaces and want macOS to treat them as one logical device.

Audia uses the same Core Audio machinery for a very different purpose: we bind a process tap and one or more physical outputs into a temporary private device graph.

A simplified version of the real configuration looks like this:

let description: [String: Any] = [
    kAudioAggregateDeviceNameKey: "Tap-\(process.id)",
    kAudioAggregateDeviceUIDKey: UUID().uuidString,
 
    // Don't expose Audia's implementation device in System Settings.
    kAudioAggregateDeviceIsPrivateKey: true,
 
    // Send identical audio to all selected outputs.
    kAudioAggregateDeviceIsStackedKey: true,
 
    kAudioAggregateDeviceTapAutoStartKey: true,
 
    kAudioAggregateDeviceMainSubDeviceKey: clockDeviceUID,
    kAudioAggregateDeviceClockDeviceKey: clockDeviceUID,
 
    kAudioAggregateDeviceSubDeviceListKey: [
        [
            kAudioSubDeviceUIDKey: clockDeviceUID,
            kAudioSubDeviceDriftCompensationKey: false
        ],
        [
            kAudioSubDeviceUIDKey: secondaryUID,
            kAudioSubDeviceDriftCompensationKey: true
        ]
    ],
 
    kAudioAggregateDeviceTapListKey: [
        [
            kAudioSubTapUIDKey: tapDesc.uuid.uuidString,
            kAudioSubTapDriftCompensationKey: true
        ]
    ]
]
 
var aggregateID: AudioObjectID = .unknown
 
AudioHardwareCreateAggregateDevice(
    description as CFDictionary,
    &aggregateID
)

This single dictionary contains several lessons that weren't obvious to me before building Audia.

First, audio devices have clocks.

If we send the same stream to two independent DACs, those devices are not guaranteed to advance at exactly the same rate forever. One becomes the clock source; the other needs drift compensation.

Second, the aggregate is private.

Audia is creating Core Audio devices internally, but users shouldn't suddenly see Tap-42891 appear beside their MacBook speakers in Audio MIDI Setup.

And third, the process tap is explicitly inserted into the device using its UUID.

Only after constructing that graph can we register the real-time callback:

AudioDeviceCreateIOProcIDWithBlock(
    &deviceProcID,
    aggregateDeviceID,
    queue
) { _, inInputData, _, outOutputData, _ in
 
    processAudio(inInputData, to: outOutputData)
}
 
AudioDeviceStart(aggregateDeviceID, deviceProcID)

Inside that callback, Audia finally has what we wanted from the beginning:

PCM from one application
        ↓
volume / pan / EQ
        ↓
PCM for the selected physical output

Getting there was not nearly as synchronous as this code makes it look.

`noErr` does not mean the device is ready

One of the most annoying bugs in the rewrite looked impossible at first.

We would call:

AudioHardwareCreateAggregateDevice(...)

and receive:

noErr

Great.

Then we'd immediately install the IOProc and start the device.

Sometimes it worked.

Sometimes we got silence.

Sometimes Core Audio returned errors such as -50.

The same code. The same machine. No obvious difference.

Eventually we discovered the important distinction:

The aggregate-device creation call can succeed before HAL has finished materializing the device's streams internally.

The API call is synchronous.

The underlying Core Audio state isn't completely synchronous.

So Audia now explicitly waits for the device to become usable before attaching the rest of the pipeline.

The relevant helper is roughly:

extension AudioObjectID {
    func waitUntilReady(
        timeout: TimeInterval = 1.0,
        pollInterval: TimeInterval = 0.01
    ) -> Bool {
 
        let deadline = CFAbsoluteTimeGetCurrent() + timeout
 
        while CFAbsoluteTimeGetCurrent() < deadline {
            if isDeviceAlive() {
                return true
            }
 
            CFRunLoopRunInMode(
                .defaultMode,
                pollInterval,
                false
            )
        }
 
        return false
    }
}

The real implementation also validates that the device's stream configuration has appeared.

That little readiness loop exists because an API returning successfully did not imply the state transition I assumed it did.

This became a recurring lesson while working with Core Audio:

Many bugs weren't failures of an operation. They were failures of my mental model of when that operation had actually finished.

Then we broke the audio thread

Once the process-tap pipeline worked, the next version had another problem.

It sounded fine until the system was busy.

Then we'd get pops.

Dropouts.

Occasional UI stutter.

Multiple applications playing simultaneously made it worse.

The first implementation passed audio around using things that are perfectly normal in most application code:

DispatchQueue.async
NSLock
temporary heap allocations
buffers allocated while processing frames

Those choices are reasonable almost everywhere in a macOS application.

A Core Audio render callback is not "almost everywhere."

It runs under a real-time deadline.

If the callback blocks while another thread owns a mutex, audio doesn't wait politely for the lock.

The DAC still needs its next buffer.

Miss that deadline and the user hears the bug.

The particularly nasty failure mode was priority inversion:

high-priority audio thread
        ↓
needs lock
        ↓
lock held by ordinary thread
        ↓
audio thread waits
        ↓
hardware deadline passes
        ↓
POP

Allocations caused similar trouble because the allocator itself can involve synchronization and unpredictable work.

We had built a real-time pipeline using ordinary application-concurrency instincts.

That was wrong.

Replacing synchronization with an SPSC ring buffer

The audio callback has one job:

finish before the hardware asks for the next buffer.

So we removed essentially everything from that path that could block.

For transferring samples from a real-time producer to a lower-priority consumer, Audia uses a Single Producer, Single Consumer ring buffer.

The storage is allocated once:

final class SPSCFloatRingBuffer {
    private let capacity: Int64
 
    private let left: UnsafeMutablePointer<Float>
    private let right: UnsafeMutablePointer<Float>
 
    private var writeIndex: Int64 = 0
    private var readIndex: Int64 = 0
 
    init(capacity: Int) {
        let resolved = max(1, capacity)
 
        self.capacity = Int64(resolved)
 
        self.left = .allocate(capacity: resolved)
        self.right = .allocate(capacity: resolved)
 
        self.left.initialize(repeating: 0, count: resolved)
        self.right.initialize(repeating: 0, count: resolved)
    }
}

The producer and consumer advance independent monotonic indices.

The current implementation uses atomic barriers around those counters:

let read = OSAtomicAdd64Barrier(0, &readIndex)
let write = OSAtomicAdd64Barrier(0, &writeIndex)

Writing never waits for the reader.

If the consumer falls too far behind, we would rather discard old unread samples than block the real-time producer:

let used = write - read
let available = capacity - used
 
if Int64(frames) > available {
    let overflow = Int64(frames) - available
 
    // Advance the reader instead of blocking the producer.
    _ = OSAtomicAdd64Barrier(
        overflow,
        &readIndex
    )
}

That tradeoff looks brutal until you think about what the alternatives mean.

For real-time playback:

drop stale visualization / recording samples

can be survivable.

block the hardware audio callback

is not.

This was one of the moments where Audia stopped feeling like a normal Swift app and started feeling like systems programming.

Correctness wasn't simply:

"Did every requested operation eventually happen?"

It became:

"Did the required work happen before a physical deadline, without anything on the path having unbounded latency?"

DSP made the real-time constraints harder

Per-app volume was only the beginning.

Audia also supports per-app EQ, global EQ, panning, metering, and multiple routing configurations.

Initially, implementing EQ directly seemed simple enough.

A peaking biquad is governed by a second-order IIR recurrence. Chain ten of them together and we have a ten-band graphic EQ.

We use the standard Audio EQ Cookbook coefficient equations. At a high level each band produces coefficients:

b0, b1, b2
a1, a2

and transforms the incoming signal based on both current input and previous filter state.

The problem wasn't understanding the equation.

The problem was executing it across:

10 bands
×
2 channels
×
N active applications
×
tens of thousands of samples per second

while still respecting the audio callback's deadline.

Pure Swift sample-by-sample loops scaled poorly as the number of active apps increased.

So we moved the expensive work to Apple's Accelerate framework and vDSP_biquad.

Audia constructs a cascade setup:

let coefficients = BiquadMath.coefficientsForAllBands(
    gains: clampedGains,
    sampleRate: sampleRate
)
 
let newSetup = coefficients.withUnsafeBufferPointer { ptr in
    vDSP_biquad_CreateSetup(
        ptr.baseAddress!,
        vDSP_Length(EQSettings.bandCount)
    )
}

Then the real-time path can hand blocks of samples to optimized vector routines rather than running twenty independent scalar loops for every frame.

We use Accelerate elsewhere too:

vDSP_maxmgv(...)  // peak magnitude
vDSP_measqv(...)  // RMS / mean-square energy
vDSP_vsmul(...)   // volume gain
vDSP_vadd(...)    // mixing

This isn't just "SIMD because SIMD is faster."

Moving from object-heavy per-sample processing toward contiguous buffers and vector operations changed what workloads we could safely run inside the deadline.

Even changing an EQ slider can break real-time audio

Another bug appeared when users dragged EQ controls.

Changing a band's gain means changing filter coefficients.

Our early instinct was effectively:

slider changes
    ↓
destroy old filter setup
    ↓
calculate coefficients
    ↓
create new setup
    ↓
continue processing

Functionally correct.

Real-time unsafe.

Creating and destroying vDSP_biquad_Setup objects may involve allocation and cleanup. Doing that synchronously while the audio thread needs the setup can cause stalls.

The current architecture instead performs the expensive work away from the callback.

Conceptually:

UI changes EQ
      ↓
background thread computes coefficients
      ↓
background thread creates new vDSP setup
      ↓
pointer changes
      ↓
old setup retired asynchronously

A trimmed version looks like:

let coefficients = BiquadMath.coefficientsForAllBands(
    gains: clampedGains,
    sampleRate: sampleRate
)
 
let newSetup = coefficients.withUnsafeBufferPointer { ptr in
    vDSP_biquad_CreateSetup(
        ptr.baseAddress!,
        vDSP_Length(EQSettings.bandCount)
    )
}
 
os_unfair_lock_lock(&setupLock)
 
let oldSetup = _eqSetup
_eqSetup = newSetup
 
os_unfair_lock_unlock(&setupLock)
 
if let oldSetup {
    retireSetup(oldSetup)
}

The subtle point is where synchronization occurs.

"Never use locks in an audio application" would be an oversimplification.

The rule is closer to:

Don't make the real-time render path wait on work whose completion time you don't control.

Routing state, UI state, setup construction, and destruction can synchronize elsewhere.

The render callback needs a far stricter contract.

Smooth volume isn't just UI polish

Another thing I had underestimated was how physical audio makes seemingly harmless state changes audible.

Suppose the user drags an application's volume from:

0.25 → 0.80

If we simply switch the multiplier between two consecutive samples, we've introduced a discontinuity into the waveform.

That discontinuity can sound like a click or zipper noise.

So Audia doesn't treat volume as:

currentVolume = targetVolume

It approaches the target over time using an exponential smoothing coefficient based on the current sample rate:

α = 1 - exp(-1 / (fs × 0.030))

Then each sample moves toward the target:

current += (target - current) * α

Once the value settles, we can return to a faster vectorized multiply.

This is a tiny implementation detail, but it changed how I thought about audio software:

A UI value can change instantaneously. A physical signal often shouldn't.

Bluetooth taught us that "output device" is not one abstraction

Wired speakers were comparatively predictable.

Bluetooth was not.

Switching to Bluetooth could introduce a few hundred milliseconds of hardware/codec stabilization. Devices could expose different nominal sample rates. AirPods could change behavior again if macOS switched into a bidirectional headset mode.

Our original device-switch logic assumed we could essentially replace one destination with another.

That produced stutters and unreliable transitions.

The eventual implementation explicitly inspects the output transport type and gives Bluetooth a longer warm-up period before considering the new path ready.

It also recalculates DSP state when the sample rate changes.

That matters because a filter designed around:

48,000 samples / second

does not have the same frequency response if the hardware suddenly operates at another rate.

And multi-output routing introduced yet another issue: independent hardware clocks drift.

That's why Audia enables:

kAudioSubDeviceDriftCompensationKey

for devices that are not the selected clock master.

What looked from the UI like:

"send Spotify to these headphones"

had turned into a synchronization problem between physical oscillators.

Seamless switching requires two audio pipelines at once

Audia's signature device-switching behavior came from another failed assumption.

I expected switching output devices to mean:

stop old device
change destination
start new device

Unfortunately, the gap between those operations is exactly where users hear silence, pops, or clicks.

So Audia doesn't immediately destroy the current path.

It temporarily creates another one.

When switching outputs:

                    ┌── old aggregate → old DAC
process tap / audio ┤
                    └── new aggregate → new DAC

The new path warms up while the old one continues playing.

Once the destination is ready, Audia performs an equal-power crossfade.

The old output follows:

cos(progress × π/2)

while the new output follows:

sin(progress × π/2)

For ordinary devices, the transition is around 50 ms.

Bluetooth receives a much larger warm-up window before the fade begins.

Only after the crossfade finishes do we tear down the old Core Audio objects.

The implementation ended up requiring:

create secondary tap
        ↓
create secondary aggregate
        ↓
wait for HAL readiness
        ↓
warm destination clock
        ↓
run both pipelines
        ↓
crossfade in the RT callback
        ↓
promote secondary
        ↓
tear down primary asynchronously

I originally thought device switching would be a property update.

It became orchestration across two concurrently alive audio engines.

One architecture choice I would not make again

There is one decision in Audia that made perfect sense when we started and that I would redesign now.

We gave each tapped application its own ProcessTapController, aggregate device, and HAL IO path.

That made the software beautifully modular:

Spotify
└── ProcessTapController
    ├── tap
    ├── aggregate
    ├── EQ
    └── routing
 
Chrome
└── ProcessTapController
    ├── tap
    ├── aggregate
    ├── EQ
    └── routing

Per-app volume and routing became easy to reason about.

But the abstraction hides a system cost.

Every additional aggregate involves more Core Audio state:

HAL callbacks
clock synchronization
drift compensation
device objects
teardown lifecycle

Five active applications can therefore mean several independent HAL paths all hitting the audio system concurrently.

If I were starting the engine again, I would seriously explore a centralized architecture:

process tap 1 ─→ SPSC ─┐
process tap 2 ─→ SPSC ─┼─→ one master mixer → one HAL output
process tap 3 ─→ SPSC ─┘

Per-app DSP could still happen independently, but a single output graph would own final mixing and hardware delivery.

That trades some of the elegant isolation of the current architecture for fewer Core Audio device objects and a more centralized timing model.

It's the kind of tradeoff I couldn't have understood by reading the API documentation.

I had to build the first architecture far enough to discover what it made expensive.

Another design I would rethink: output switching

Our current crossfade creates a second aggregate device when the user chooses another destination.

It works, and the transition can sound seamless.

But aggregate-device creation involves Core Audio daemon IPC and asynchronous HAL initialization.

That means the time required to construct the new path isn't completely deterministic.

A stronger future architecture could keep likely outputs pre-opened and perform routing changes as an in-memory mixing matrix:

                    ┌─→ Speakers output
master application ─┼─→ Headphones output
audio                └─→ Interface output

Then switching could mean changing gains inside an already-running graph instead of constructing and destroying HAL objects at the moment the user asks for a transition.

Again, the lesson wasn't that our first solution was stupid.

It was that the first solution exposed the next bottleneck.

audia-tap started as a debugging tool

One of the most useful things we built while figuring all of this out wasn't Audia's UI.

It was a command-line program.

audia-tap started as a way to isolate the problem.

Instead of asking:

"Why is the entire SwiftUI application producing no audio?"

we could ask:

"Can I tap PID 4812 and observe its raw PCM?"

That removed:

SwiftUI state updates
AppKit lifecycle
the mixer interface
most product behavior

and left us with Core Audio.

Today the rough pipeline is:

PID / bundle ID
      ↓
Core Audio process AudioObjectID
      ↓
CATapDescription
      ↓
AudioHardwareCreateProcessTap
      ↓
private Aggregate Device
      ↓
HAL IOProc
      ↓
SPSC ring buffer
      ↓
PCM output / downstream consumer

It also turned out to be useful outside Audia.

Because the tool is headless and Unix-friendly, application audio can be piped into things such as:

audia-tap ... | ffmpeg ...

or local speech/audio processing pipelines without opening the full desktop application.

Separating the engine from the product UI made the system easier to debug and produced a useful open-source tool almost accidentally.

There are still ugly corners

Audia works, but I don't want to pretend we've turned macOS audio into a solved problem.

Browser tabs

Core Audio understands processes, not the conceptual browser tab the user sees.

Chrome, Safari, Arc, and other browsers can route media through renderer/helper processes whose lifetimes and mappings are controlled by the browser.

That makes robust tab-level isolation a different problem, likely requiring cooperation from a browser extension or another layer above Core Audio.

Audio-capture permissions

Modern macOS requires permission for system-audio capture.

The permission surface is considerably less pleasant than something like:

AVCaptureDevice.requestAccess(...)

Audia currently has to interact with the relevant TCC machinery to preflight/request system audio capture.

This is one of the areas where relying on private or poorly surfaced system behavior creates obvious maintenance risk across future macOS releases.

AirPods changing modes

Bluetooth headphones don't always remain the same "device" in practical audio terms.

When microphone use forces a headset into a bidirectional mode, available sample rates and codec behavior can change dramatically.

The audio graph has to react instead of assuming the format remains stable forever.

Applications dying underneath a tap

Processes disappear.

Sometimes abruptly.

HAL state does not necessarily vanish at precisely the same instant as the Unix process.

Audia therefore tracks callback freshness and invalidates stale state so a dead application doesn't leave frozen meters or zombie routing behind.

These aren't side issues.

They're a large part of what separates:

"I successfully called the API"

from:

"A normal person can leave this running all day."

What building Audia changed about how I think about systems programming

Before this project, I thought of audio routing mostly in terms of streams:

get samples
change samples
send samples somewhere

That's technically true and almost useless.

The real system includes:

process identity
    ↓
HAL object discovery
    ↓
process taps
    ↓
aggregate-device graphs
    ↓
hardware clocks
    ↓
real-time deadlines
    ↓
lock-free communication
    ↓
DSP state
    ↓
sample-rate changes
    ↓
device lifecycle
    ↓
permissions

And every abstraction leaks eventually.

The biggest lesson wasn't a particular Core Audio function.

It was that systems programming often consists of discovering the assumptions hidden inside an abstraction.

I assumed successful device creation meant a ready device.

It didn't.

I assumed a tap could simply be routed to speakers.

It couldn't.

I assumed locks were acceptable because the critical sections were tiny.

They weren't.

I assumed changing volume was just changing a number.

It wasn't.

I assumed changing devices meant replacing one output identifier.

It definitely wasn't.

Each wrong assumption produced a bug.

Each bug forced us one layer deeper.

That's the part of building Audia that I wish I had documented from the beginning.

The project today

Audia is now a native macOS menu-bar application with:

per-application volume
per-app and global EQ
stereo panning
individual device routing
multi-device output
glitch-reduced device crossfading
recording
media controls
persistent profiles

The part I'm most interested in, however, is still the layer underneath the UI:

Intercept → Process → Route

That engine is the result of repeatedly discovering that the obvious solution wasn't quite how Core Audio actually worked.

Audia is built collaboratively with Arjav Lamsal. We developed the Core Audio tap/aggregate-device and DSP pipeline together. My work has focused especially on real-time audio safety, crossfade behavior, per-app EQ state/blending, and release/distribution infrastructure, while Arjav has focused heavily on mixer routing, interface design, pipeline hardening, and backend services.

And there are still several architectural ideas I'd like to revisit.

That's probably a good sign.

The project started because I wanted a volume slider that macOS didn't provide.

It turned into a reason to learn what happens underneath one.

Command Palette