Towards Low-Complexity Vector Quantization

Doctoral thesis, 2004

This thesis is about constructing low-complexity, yet high-performance, vector quantizers (VQs) for 'real-world' sources. Knowledge concerning the source is extracted from a finite training set. In contrast with conventional VQ design procedures, we use the training set to estimate a statistical model for the source. The model is a weighted sum of Gaussian densities, known as Gaussian mixture (GM). Parameters of a GM are estimated by the widely-used expectation-maximization (EM) algorithm. Thus, a complex source density is locally divided into Gaussian densities. To reduce design complexity, a local encoder is designed independently, for each mixture component. Therefore, a part of this thesis is devoted to study Gaussian encoders. It is evident that a global encoder inherits the properties of the local encoders. Hence, we target for Gaussian encoders which have i) a high performance, ii) a low-complexity encoding and indexing, iii) a small memory requirement, and iv) a straightforward design. Such a coder is studied in paper A, where we combine finite dimensional random coding and companding techniques, to construct fixed rate VQs. An entropy constrained dual is proposed in paper D. These two building blocks are brought to the 'real-world' of spectral coding in papers B and D. To obtain a high fidelity spectral coder, independent training of local encoders is proposed in paper C. We employ constrained training, and use the estimated model to generate a synthetic database. The fidelity criterion in spectral coding is a non-difference distortion measure, called spectral distortion (SD). Therefore, in paper C we partially incorporate a non-difference distortion measure into design and encoding procedures, still with a reasonable complexity. In paper E, we address a speech coder that can operate over a wide range of rates, without re-training. Finally, paper F is a continuation of paper A, where we bring trellises into play. 'On-the-fly' design and low-complexity encoding and indexing are features of VQs proposed in paper F. Moreover, such VQs have a high performance over a wide range of dimensions and encoding rates.