Selective Oscillatory State-Space Models via Scaled Rotations¶

A normal, input-dependent transition that preserves D-LinOSS's full unit-disk spectrum while admitting a structural, schedule-free stability guarantee.

Working name used below: S-LinOSS (Selective Linear Oscillatory State-Space model). The model keeps the second-order, damped-oscillator character of D-LinOSS but replaces the IMEX discretization of the transition matrix with a directly parameterized scaled rotation, and makes the two spectral degrees of freedom — magnitude (damping) and phase (frequency) — input-dependent.

1. Motivation¶

D-LinOSS (Boyer et al.) generalizes LinOSS (Rusch & Rus) by adding learnable damping $G$, so that a layer's recurrent matrix $M$ can reach every stable, uncoupled second-order system — its reachable spectrum is the full complex unit disk, whereas LinOSS-IM and LinOSS-IMEX reach only measure-zero curves in $\mathbb{C}$. Both LinOSS and D-LinOSS are, however, linear time-invariant (LTI): the transition $M$ is fixed across the sequence. The D-LinOSS paper explicitly flags this as the main limitation, noting that LTI layers cannot perform input-dependent operations such as selective copying, and points to time-varying (selective) dynamics as future work.

The obvious route to selectivity — make $(A_k, G_k, \Delta t_k)$ input-dependent and feed them through the IMEX block of Eq. (6) — is problematic. The IMEX transition

\[ M ;=; \begin{pmatrix} S^{-1} & -\Delta t,S^{-1}A \[2pt] \Delta t,S^{-1} & I - \Delta t^{2} S^{-1} A \end{pmatrix}, \qquad S = I + \Delta t,G, \]

is non-normal: its eigenvectors are not orthogonal ($\sigma_{\max}/\rho \approx 1.4$–$2.5$ in the regimes of interest). For an LTI layer this is harmless — stability is governed by the spectral radius alone. But once the transition becomes input-dependent, the state propagates as a product of distinct matrices $\prod_k M_k$, and for non-normal matrices

\[ \Big|\textstyle\prod_k M_k\Big|_2 ;\not\le; \prod_k \rho(M_k), \]

so the state norm can transiently overshoot (empirically by an order of magnitude) even when every $\rho(M_k) \le 1$. Certifying stability of such a product under an arbitrary input-driven schedule requires a common Lyapunov function in a single fixed metric $P$ — which the natural symplectic invariant of the oscillator does not supply.

The fix. Replace the transition by a scaled rotation, which is normal. A normal matrix satisfies $|M|_2 = \rho(M)$ exactly, so the operator norm and spectral radius coincide and submultiplicativity gives a free, structural stability bound for any input-dependent sequence. We keep the full unit-disk eigenvalue expressivity that D-LinOSS was designed for; we give up the literal discretized-forced-oscillator identity of the transition matrix.

2. Continuous-time formulation¶

We retain the D-LinOSS continuous-time system of damped, forced harmonic oscillators (diagonal $A, G \succeq 0$, so the $m$ modes are uncoupled):

\[ \mathbf{x}''(t) = -A\mathbf{x}(t) - G\mathbf{x}'(t) + B\mathbf{u}(t), \qquad \mathbf{y}(t) = C\mathbf{x}(t). \]

Introducing the velocity $\mathbf{z} = \mathbf{x}'$ gives the first-order form

\[ \begin{pmatrix}\mathbf z'(t)\\ \mathbf x'(t)\end{pmatrix} = \begin{pmatrix}-G & -A \\ I & 0\end{pmatrix} \begin{pmatrix}\mathbf z(t)\\ \mathbf x(t)\end{pmatrix} + \begin{pmatrix}B\\ 0\end{pmatrix}\mathbf u(t). \]

This is the same physical object as in both papers. The difference is purely in how we turn it into a discrete recurrence.

3. From discretization to a directly parameterized normal transition¶

In D-LinOSS the discrete transition $M$ is derived from $(A, G, \Delta t)$ via IMEX integration. We instead parameterize the per-mode transition directly in spectral coordinates, choosing the unique normal matrix with the desired eigenvalues.

For a single oscillator mode with state $\mathbf w_k = (z_k, x_k)^\top \in \mathbb R^2$, the transition is a scaled $2\times 2$ rotation:

\[ M(\nu,\theta) ;=; \nu, R(\theta) ;=; \nu \begin{pmatrix} \cos\theta & -\sin\theta \ \sin\theta & \cos\theta \end{pmatrix}, \qquad \nu \in (0,1],\ \theta \in [0, 2\pi). \]

Its eigenvalues are $\lambda_{1,2} = \nu e^{\pm i\theta}$, so it has magnitude $|\lambda| = \nu$ and phase $\arg\lambda = \pm\theta$. As $(\nu,\theta)$ range over $(0,1]\times[0,2\pi)$, the eigenvalue pair sweeps the entire complex unit disk — the same reachable spectrum D-LinOSS achieves through $(A, G)$, recovered here by construction. The decoupling of damping and frequency that D-LinOSS obtained by adding $G$ is now manifest: $\nu$ alone sets $|\lambda|$ and $\theta$ alone sets $\arg\lambda$, with no coupling.

The crucial structural difference from the IMEX block: $M(\nu,\theta)$ is a scaled orthogonal matrix, hence normal — its eigenvectors are orthogonal. We keep D-LinOSS's spectrum and change only its eigenvectors.

4. The selective recurrence¶

We make the two spectral knobs input-dependent. For each mode $i$ and step $k$, with input token $u_k$ (the layer input at step $k$):

$$

\begin{pmatrix} z_k \ x_k \end{pmatrix}¶

\nu_k\, \begin{pmatrix} \cos\theta_k & -\sin\theta_k \ \sin\theta_k & \cos\theta_k \end{pmatrix} \begin{pmatrix} z_{k-1} \ x_{k-1} \end{pmatrix} + b_k\, u_k $$

with selective parameters produced from the input:

\[ \nu_k = \sigma\big(W_\nu u_k + c_\nu\big) \in (0,1] \quad\text{(selective damping / forgetting, } |\lambda_k|), \]

\[ \theta_k = f\big(W_\theta u_k + c_\theta\big) \quad\text{(selective frequency, } \arg\lambda_k), \]

and an input-dependent (or fixed) input gate $b_k$. Here $\sigma$ is the logistic sigmoid (guaranteeing $\nu_k \in (0,1]$ and hence stability — see §6), and $f$ is any smooth map into the angular range (e.g. a scaled sigmoid or a linear map reduced mod $2\pi$).

The readout stays on the position component $x_k$, preserving the twice-integrated, smoothed second-order character:

\[ y_k = C, x_k ;(+, D, u_k). \]

These are exactly the two quantities D-LinOSS decoupled — magnitude and phase — now made time-varying. Compared to LinOSS/D-LinOSS this adds genuine selectivity (input-dependent transition) in the spirit of Mamba/S6, but on a second-order, oscillatory primitive rather than a first-order diagonal one.

5. Equivalent complex (diagonal) form¶

Identify the 2-vector $\mathbf w_k = (z_k, x_k)$ with the complex number $h_k = z_k + i,x_k$. Because a rotation by $\theta$ acting on $(z,x)$ is multiplication by $e^{i\theta}$, the entire recurrence collapses to a scalar complex mode:

\[ h_k = \nu_k e^{i\theta_k}, h_{k-1} + b_k u_k, \qquad \lambda_k := \nu_k e^{i\theta_k}\in\mathbb C,\ |\lambda_k| = \nu_k \le 1, \]

with the position readout recovered as $x_k = \mathrm{Im}(h_k)$ and velocity as $z_k = \mathrm{Re}(h_k)$.

This is precisely a selective Linear Recurrent Unit (LRU) mode: a diagonal complex recurrence with input-dependent coefficient inside the unit disk. Normality is trivial in this form (a $1\times1$ complex multiplier is automatically normal). The 2D real scaled-rotation view and the scalar complex view are exact equivalents; the complex view is convenient for implementation and for the parallel scan (§8), the real view makes the second-order $[\text{velocity}, \text{position}]$ structure explicit.

6. Stability: structural and schedule-free¶

Proposition (monotone state norm). For every step $k$, the homogeneous transition $M_k = \nu_k R(\theta_k)$ satisfies $|M_k|_2 = \nu_k$ exactly, and the unforced state obeys

\[ |\mathbf w_k| = \nu_k,|\mathbf w_{k-1}|. \]

There is therefore no transient overshoot, ever — the worst-case amplification of the non-normal IMEX product (the $\sim10\times$ transient) cannot occur, because $\sigma_{\max} = \rho = \nu_k$ for a normal transition.

Proposition (whole-sequence bound). For any input-dependent sequence ${(\nu_k,\theta_k)}_{k=1}^{n}$ with $\nu_k \le 1$,

\[ \Big|\textstyle\prod_{k=1}^{n} M_k\Big|_2 ;\le; \prod_{k=1}^{n} |M_k|_2 ;=; \prod_{k=1}^{n} \nu_k ;\le; 1. \]

Proof. Submultiplicativity of the operator 2-norm, then the norm identity above. $\square$

The bound holds for arbitrary schedules of $\nu_k,\theta_k$ — exactly what the selective IMEX block cannot give, since a product of non-normal matrices is not controlled by the product of their spectral radii. No LMI, no Lyapunov search, no offline verification is required: enforcing $\nu_k\in(0,1]$ (automatic from the $\sigma(\cdot)$ parameterization) is sufficient on its own. Contrast with certifying the genuine IMEX block under selectivity, which requires a common quadratic Lyapunov function in a fixed metric $P$ that the symplectic invariant fails to provide.

This is the central payoff: stability that in D-LinOSS was a Proposition about a fixed matrix becomes, here, a structural property that survives arbitrary input-dependence.

7. Closed-form dynamics¶

Because rotations commute and compose by adding angles, the homogeneous (unforced) solution factorizes cleanly:

\[ \mathbf w_n = \Big(\prod_{k=1}^{n}\nu_k\Big)\; R\!\Big(\sum_{k=1}^{n}\theta_k\Big)\;\mathbf w_0 . \]

Two readings:

Instantaneous frequency $= \theta_k$; the running phase $\sum_k \theta_k$ acts as an input-controlled chirp.
Instantaneous decay rate $= -\ln\nu_k$; the envelope $\prod_k \nu_k$ contracts at an input-controlled rate — the layer holds memory when $\nu_k \to 1$ and forgets when $\nu_k$ drops.

The full forced solution, in complex form, is the selective-LRU convolution

\[ h_n = \sum_{j=1}^{n} \Big(\textstyle\prod_{k=j+1}^{n}\nu_k\Big)\, e^{\,i\sum_{k=j+1}^{n}\theta_k}\; b_j u_j . \]

The dynamics remain genuinely second-order and oscillatory: a damped oscillator at every instant, and over the sequence a frequency- and amplitude-modulated one.

8. Efficient computation via associative parallel scans¶

The recurrence is a first-order linear recurrence with input-dependent coefficients, $h_k = \lambda_k h_{k-1} + b_k u_k$ with $\lambda_k = \nu_k e^{i\theta_k}$. This is exactly the structure that admits associative parallel scans, as used for LinOSS, S5, the LRU, and the selective Mamba/S6 models. Define elements $e_k = (\lambda_k,\ b_k u_k)$ and the binary operator

\[ (a_1, a_2)\bullet(b_1, b_2) = (b_1 a_1,\ b_1 a_2 + b_2), \]

which is associative. Scanning $[e_1, e_2, \dots, e_N]$ returns the running states $h_n$ in the second component. Each combine is $O(1)$ per mode (scalar complex multiply–add), so the layer runs in $O(\log N)$ parallel depth and $O(N)$ work over the sequence — the same complexity profile as D-LinOSS, now with a time-varying $\lambda_k$. In the real $2\times2$ view the transitions are scaled rotations, whose products are again scaled rotations, so the scan combine stays $O(1)$ there too.

The diagonal/complex transition means no matrix inversion or Schur complement (which the IMEX form needed); the transition is read off directly from $(\nu_k,\theta_k)$.

9. Initialization¶

Match D-LinOSS's known-good starting point so that, at zero input contribution, the base eigenvalue distribution reproduces the radial band $|\lambda| \in [0.9, 1.0]$ with full angular range — the initialization shown best in the D-LinOSS ablation.

Magnitude bias $c_\nu$: set so $\nu_k = \sigma(c_\nu)$ lands in $[0.9, 1.0]$ at baseline (i.e. radially uniform in that band across modes), giving near-unit-norm eigenvalues that retain energy over long horizons.
Phase bias / map $c_\theta, f$: spread $\theta_k$ across the full angular range $[0, 2\pi)$ (equivalently $[0,\pi)$ for a conjugate pair) at baseline.
Selective weights $W_\nu, W_\theta$: initialize small, so selectivity is learned as a perturbation off a working oscillator bank rather than from scratch.

Because the spectral coordinates $(\nu,\theta)$ are the parameters, the inverse map $(\lambda)\mapsto(A,G)$ that D-LinOSS needed for eigenvalue-targeted initialization is unnecessary here — you sample $(\nu,\theta)$ directly.

10. What to make selective — and what to avoid¶

Safe to make selective: $\nu_k$, $\theta_k$ (the transition), and $b_k$ / output gates. The transition stays normal regardless of how wildly $\nu_k,\theta_k$ vary, so the stability guarantee of §6 survives any schedule.
Do not reintroduce a non-normal transition. In particular, do not parameterize via selective $(A_k, G_k, \Delta t_k)$ fed through the real IMEX block — that returns you to the product-of-non-normal-matrices problem and forfeits the structural guarantee.

11. Relationship to existing models¶

LinOSS-IMEX: conservative ($|\lambda| = 1$), fixed transition, measure-zero spectrum. S-LinOSS recovers the energy-preserving regime as the special case $\nu_k \equiv 1$, but with selectable frequency and the full angular range.
LinOSS-IM: dissipative, fixed transition, measure-zero spectrum coupling damping to phase. S-LinOSS decouples them.
D-LinOSS: full unit-disk spectrum, fixed (LTI) non-normal transition. S-LinOSS keeps the spectrum, makes it input-dependent, and trades the non-normal eigenvectors for orthogonal ones.
LRU / Mamba (S6): first-order selective diagonal recurrences. S-LinOSS is the second-order, oscillatory analogue — the complex mode of §5 is a selective LRU, but the real readout on $x_k = \mathrm{Im}(h_k)$ preserves the twice-integrated smoothing that distinguishes oscillatory SSMs.

12. The one honest caveat¶

The scaled rotation is normal; the true IMEX block is not. Forcing the eigenvectors orthogonal is what removes the non-normal transient — but that transient is real representational content, and it may be part of why genuine D-LinOSS performs well on long physical-signal tasks. You keep the spectrum, you change the eigenvectors, and the dynamics depend on both. So S-LinOSS is a principled normal surrogate, not a drop-in equivalent to a selective D-LinOSS.

Recommended validation before committing: hold the LTI genuine-IMEX D-LinOSS block as a baseline and compare S-LinOSS on one or two UEA-MTSCA datasets (e.g. the long-sequence EigenWorms and MotorImagery, where second-order structure and long-range memory both matter), to check empirically whether the non-normality was load-bearing for your task. If S-LinOSS matches or beats the LTI baseline, the added selectivity and the free stability guarantee are pure upside.