Notes on transform domain adaptation (LMS)

Transform domain adaptive filter (e.g. DFT-LMS, DCT-LMS)

• Performance of LMS is sensitive to the eigenvalue spread of the input covariance matrix.
• The smallest eigenvalue contribute to slower convergence and the largest eigenvalue limit the range of allowed step-sizes, thus limiting the learning abilities of the filter.

A DFT/DCT type transformation allows one to whiten the input sequence in the transform domain without the need to know the correlation matrix (e.g. using KLT) of the sequence which may not be stationary.

[Hodgkiss, 1979] has shown the conditions under which time domain and frequency domain processing is equivalent.  

[Beaufays, 1995] showed analytically the eigenvalue distributions of a markov-1 sequence to have asymptotic eigenvalue spread of  [interesting matrix theory]

$$\left( \frac{1+\rho}{1-\rho} \right)^2, \quad \text{before transformation}$$  

$$\left( \frac{1+\rho}{1-\rho} \right), \quad \text{after DFT and power normalization}$$  

$$(1+\rho), \quad \text{after DCT and power normalization}$$