Past Research

CHAIS is a multi-year, multi-institution research project trying to help people follow one voice in a noisy, multi-speaker room — the "cocktail party problem" — by combining speech-processing algorithms, neuroscience, and the design of a wearable device. This is a plain read on how far the work has gotten so far, organized by category and depth.

1. Groundwork & Orientation — mature¶

Before building anything, I mapped what already exists and what the science says. This is the largest body of finished work so far, and it's thorough rather than cursory.

Literature review across six fronts of the science (real-time processing limits, combining sound with vision, multi-microphone input, reading brain/eye signals, gaze-based listening, rhythmic brain activity), each closed out with an explicit "what this means for us."
Ten candidate hardware platforms for the eventual wearable device compared head-to-head, with specific disqualifying gaps noted for each.
Teardown of nine-plus existing commercial products (hearing aids, captioning glasses, AI glasses) to establish what the project needs to beat.
Team responsibilities, contacts, and resources across the three partner institutions documented.

2. Experimental Infrastructure — designed, not built¶

The project needs a way to run controlled studies of people trying to follow one voice among several. I've designed that environment — a virtual-reality simulation of a crowded room — in real detail, but haven't built or run it yet.

A three-phase build plan, from simple prototype to a full sound-and-vision system.
Test tasks specified down to mechanics (e.g., a listener keeps re-identifying which of several simultaneous speakers just said a cue word), with concrete measures (time-to-identification, listening effort).
A data-generation approach and rough participant plan (students first, patients later).
Nothing implemented yet — no software, no sessions run.

3. Applied Modeling Research — active, iterating¶

My only thread with actual empirical results so far: does giving a speech-isolating AI model a structural bias modeled on rhythmic brain activity — an oscillatory inductive bias — make it better at the task? Three rounds of work in the first few months show a line that's correcting itself rather than stalling.

First round — does an oscillatory bias help at all? I dropped an oscillatory state-space model (a network whose internal state literally behaves like a bank of harmonic oscillators, rather than a generic memory) into an existing speech-enhancement architecture, in place of its standard component, and compared it against non-oscillatory baselines. It did not win: the plain baseline and a non-oscillatory-but-still-adaptive alternative both matched. A negative result, but interesting insight: An LTI oscillatory model can match a time-varying selective SSM on matched parameter counts.
Diagnosis. A follow-up round of experiments narrowed down why: the oscillatory model I'd built was static — its rhythm was fixed once trained, rather than able to adjust itself to what it was currently hearing (what I call "selectivity"). That's a plausible reason it couldn't track things like a shifting pitch.
Fixing it on paper. Making the oscillator selective — letting its rhythm adapt on the fly — turns out to risk numerical instability (the internal state can blow up over long sequences). I worked out a formal stability proof establishing conditions under which a selective oscillatory model stays numerically stable indefinitely, closing the theoretical gap the first round ran into. This is based on a minimal damping for a given frequency, high frequencies require higher damping for the system to remain stable.
Next. A next-generation model that combines the oscillatory bias with this selective, stability-guaranteed mechanism is fully designed and ready to test. Trained runs seem to indicate no better performance than LTI version. For stability, I found empirically that the damping floor isnt as important as restricting the frequencies away from 0 (real decay).